Skip to content

feat: implement local inference mode with SAP RPT-1 OSS model#31

Open
KoblerS wants to merge 8 commits into
mainfrom
local-mode
Open

feat: implement local inference mode with SAP RPT-1 OSS model#31
KoblerS wants to merge 8 commits into
mainfrom
local-mode

Conversation

@KoblerS

@KoblerS KoblerS commented Jun 15, 2026

Copy link
Copy Markdown
Member

Add Local Inference Mode with SAP RPT-1 OSS Model

New Feature

✨ Introduces a local inference mode for the CAP AI plugin, allowing developers to run predictions using the open-source SAP RPT-1 OSS model without requiring a full SAP AI Core connection. Two backends are supported: a local Python subprocess (AICore-local) and the HuggingFace Inference API (AICore-hf).

Changes

  • srv/LocalRPTService.js: New service implementation with two backend classes:

    • LocalSubprocessRPTService — spawns a Python subprocess (infer.py), automatically downloads and caches the model checkpoint (~65 MB) and sentence embedder on first startup, and communicates via a JSON stdin/stdout protocol.
    • HFInferenceRPTService — calls the HuggingFace Inference API directly, no local Python required.
    • LocalRPTService (default export) — delegates to either backend based on the local: true config flag.
  • lib/rpt/infer.py: New Python inference server script that reads JSON requests from stdin, runs predictions using SAP_RPT_OSS_Classifier, and writes JSON responses to stdout. Includes dtype coercion, top-3 candidate ranking, and a readiness signal on startup.

  • srv/LocalRPTService.cds: New CDS definition wiring the AICore service to the LocalRPTService.js implementation.

  • package.json: Added AICore-local and AICore-hf CDS service kinds. Changed the default (no-profile) kind from AICore-mocked to AICore-local. Production and hybrid profiles continue to use AICore-btp.

  • README.md: Added a "Local mode (without SAP AI Core)" section documenting both backends, setup steps (licence acceptance, HF token creation, .cdsrc-private.json configuration), and installation instructions for the Python package.

  • .gitignore: Fixed an overly broad resources/ ignore rule to only ignore resources/.cds-private.json.

  • 🔄 Regenerate and Update Summary
PR Bot Information

Version: 1.23.0

  • Summary Prompt: Default Prompt
  • Output Template: Default Template
  • LLM: anthropic--claude-4.6-sonnet
  • File Content Strategy: Full file content
  • Correlation ID: 70c2036e-f7c3-4036-857c-28f2e4a906e8
  • Event Trigger: pull_request.opened

@KoblerS KoblerS self-assigned this Jun 15, 2026
@KoblerS
KoblerS requested a review from a team as a code owner June 15, 2026 20:18
@KoblerS

KoblerS commented Jun 15, 2026

Copy link
Copy Markdown
Member Author

TODO: add tests

Comment thread srv/LocalRPTService.js Fixed

@hyperspace-pr-bot hyperspace-pr-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR introduces a useful local-inference mode but has several correctness issues that need addressing before merge: a typo in .gitignore that will leave sensitive files unignored, a missing module-level pandas import in the Python script, a null-dereference race on the subprocess stdin, a token-resolution inconsistency between _hfToken() and the documented credential path, and — most impactful — changing the plugin's default AICore kind from the safe AICore-mocked to AICore-local, which will silently trigger model downloads and Python subprocess spawning in all downstream projects that don't override the kind.

PR Bot Information

Version: 1.23.0

  • File Content Strategy: Full file content
  • LLM: anthropic--claude-4.6-sonnet
  • Correlation ID: 70c2036e-f7c3-4036-857c-28f2e4a906e8
  • Event Trigger: pull_request.opened

Comment thread .gitignore Outdated
Comment thread lib/rpt/infer.py
Comment thread lib/rpt/infer.py Outdated
Comment thread lib/rpt/infer.py
Comment thread srv/LocalRPTService.js
Comment thread srv/LocalRPTService.js
Comment thread srv/LocalRPTService.js Outdated
Comment thread srv/LocalRPTService.js
Comment thread package.json Outdated
@KoblerS

KoblerS commented Jun 15, 2026

Copy link
Copy Markdown
Member Author

TODO: add huggingface secret for read access to rpt1 oss model

@KoblerS

KoblerS commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Unfortunately the intereference Python script is required, since the model has been only published as PyTorch model and not an ONNX model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant