feat: implement local inference mode with SAP RPT-1 OSS model#31
feat: implement local inference mode with SAP RPT-1 OSS model#31KoblerS wants to merge 8 commits into
Conversation
|
TODO: add tests |
There was a problem hiding this comment.
The PR introduces a useful local-inference mode but has several correctness issues that need addressing before merge: a typo in .gitignore that will leave sensitive files unignored, a missing module-level pandas import in the Python script, a null-dereference race on the subprocess stdin, a token-resolution inconsistency between _hfToken() and the documented credential path, and — most impactful — changing the plugin's default AICore kind from the safe AICore-mocked to AICore-local, which will silently trigger model downloads and Python subprocess spawning in all downstream projects that don't override the kind.
PR Bot Information
Version: 1.23.0
- File Content Strategy: Full file content
- LLM:
anthropic--claude-4.6-sonnet - Correlation ID:
70c2036e-f7c3-4036-857c-28f2e4a906e8 - Event Trigger:
pull_request.opened
|
TODO: add huggingface secret for read access to rpt1 oss model |
|
Unfortunately the intereference Python script is required, since the model has been only published as PyTorch model and not an ONNX model. |
Add Local Inference Mode with SAP RPT-1 OSS Model
New Feature
✨ Introduces a local inference mode for the CAP AI plugin, allowing developers to run predictions using the open-source SAP RPT-1 OSS model without requiring a full SAP AI Core connection. Two backends are supported: a local Python subprocess (
AICore-local) and the HuggingFace Inference API (AICore-hf).Changes
srv/LocalRPTService.js: New service implementation with two backend classes:LocalSubprocessRPTService— spawns a Python subprocess (infer.py), automatically downloads and caches the model checkpoint (~65 MB) and sentence embedder on first startup, and communicates via a JSON stdin/stdout protocol.HFInferenceRPTService— calls the HuggingFace Inference API directly, no local Python required.LocalRPTService(default export) — delegates to either backend based on thelocal: trueconfig flag.lib/rpt/infer.py: New Python inference server script that reads JSON requests from stdin, runs predictions usingSAP_RPT_OSS_Classifier, and writes JSON responses to stdout. Includes dtype coercion, top-3 candidate ranking, and a readiness signal on startup.srv/LocalRPTService.cds: New CDS definition wiring theAICoreservice to theLocalRPTService.jsimplementation.package.json: AddedAICore-localandAICore-hfCDS service kinds. Changed the default (no-profile) kind fromAICore-mockedtoAICore-local. Production and hybrid profiles continue to useAICore-btp.README.md: Added a "Local mode (without SAP AI Core)" section documenting both backends, setup steps (licence acceptance, HF token creation,.cdsrc-private.jsonconfiguration), and installation instructions for the Python package..gitignore: Fixed an overly broadresources/ignore rule to only ignoreresources/.cds-private.json.PR Bot Information
Version:
1.23.0anthropic--claude-4.6-sonnet70c2036e-f7c3-4036-857c-28f2e4a906e8pull_request.opened