Skip to content

build(deps): pin lemonade 11.5.1 from a single source of truth - #177

Merged
rominf merged 4 commits into
mainfrom
feat/lemonade-11-5-1
Aug 12, 2026
Merged

build(deps): pin lemonade 11.5.1 from a single source of truth#177
rominf merged 4 commits into
mainfrom
feat/lemonade-11-5-1

Conversation

@rominf

@rominf rominf commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Bumps the pinned Lemonade embeddable runtime from 10.10.0 to 11.5.1, and removes the reason the pin kept going wrong.

Before this PR the version was a hand-written literal in seven places across four files, with nothing tying them together. They had drifted:

Where Held Should have been
engines/lemonade — the version rocm engines install lemonade downloads 10.10.0 (5 constants) latest pin
rocm-dash-tui — offline fallback when the GitHub releases API is unreachable 10.6.0 same as the engine
apps/rocm — the runtime-id label recorded in user config 10.6.0 the env_id the adapter reports

So a host that could not reach the releases API installed a runtime five minor versions behind the one the engine pinned, and a doc comment in the same block had been wrong since 10.10.0. Nothing caught any of it.

What changes

1. One pin, in a config file. runtime-deps.toml at the repository root now holds the version:

[runtime.lemonade]
version = "11.5.1"

One [runtime.<name>] table per managed runtime, each with one or more string fields — a runtime that needs more than a version (a version plus an ABI tag, say) adds a field rather than reshaping the schema.

2. Everything else is derived. A new rocm-deps crate has a build.rs that turns each field into a constant (runtime.lemonade.versionLEMONADE_VERSION) and provides the helpers that build archive names (lemonade-embeddable-<ver>-<os-arch>.{tar.gz|zip}) and download URLs from it. The engine adapter, the dashboard fallback, and the runtime-id label all call those helpers, so there is no second place a version can be written. Bumping Lemonade is now a one-line edit to the TOML.

Generating at build time rather than committing a generated .rs means there is nothing that can go stale, so this needs no cargo xtask ... --check gate the way MANIFEST.md and THIRD_PARTY_NOTICES.txt do. A missing or malformed pin is a build error, and the two crates that used to disagree now cannot: they read the same constant.

Behaviour changes

  • The Lemonade runtime installed by rocm engines install lemonade moves to 11.5.1.
  • The offline fallback and the runtime-id label recorded in user config now match that pin instead of naming 10.6.0. The label is an opaque identifier, not a lookup key, but it is persisted, so it is called out rather than buried.
  • Only windows-x64 and ubuntu-x64 remain wired up in the engine adapter. The release also publishes ubuntu-arm64 and macos-arm64; the adapter still does not select them, and the doc comment that claimed otherwise is corrected.

A bug the bump would otherwise have hit

Reviewing the bump surfaced a pre-existing defect that this PR would have made live for every user: upgrading did nothing.

The embeddable runtime directory is not version-scoped, so prepare_embeddable downloaded the new archive and then skipped extraction because lemond was already present from the previous version — leaving the old binaries in place while reporting the new version. One layer up, the CLI asked the adapter whether a runtime was installed and ignored which version came back, so a bump never reached the install path at all.

Both now compare versions. The adapter re-extracts when the recorded version differs from the one being installed, and the CLI requires the detected env id to match the one it asked for — which is meaningful only because that id is now derived from the pin. The adapter's decision is a pure function, so the upgrade case is covered by a test without needing a real archive.

Non-goal: a version override variable

An earlier revision of this PR added ROCM_CLI_LEMONADE_VERSION to try an untested release without rebuilding. It has been dropped, because it could not work as written and the fix does not belong in a version bump.

The archive name and URL are derived from the resolved version, but the SHA-256 digest cannot be — every release has its own. prepare_embeddable would have fetched the overridden release and then verified it against the pinned version's digest, so any real override failed with a checksum mismatch. (The original verification used v99.0.0, which 404s before reaching the digest check, which is why it looked fine.)

Making it work means pinning digests per version and deciding what an override without a known digest should do — either carrying digests in runtime-deps.toml alongside the version, or letting the caller supply one. That is a design question about supply-chain verification, not part of bumping a pin, so it is left for a follow-up. Nothing here blocks it: the schema already takes extra fields per runtime.

What 11.5.1 brings over 10.10.0

  • Ten MiniCPM text and vision GGUF models added to the built-in llama.cpp catalog.
  • Router Builder gains a Test Prompt tab and a POST /routing/validate endpoint that runs a routing policy against a sample prompt and returns the decision plus a step-by-step trace.
  • lemonade bench adds an image-generation benchmark mode with capability-aware scenario/model filtering and a --timeout flag.
  • Tool-calling requests to llama.cpp with large JSON schema bounds are now accepted (previously rejected by a grammar limit).
  • Upstream breaking changes worth knowing: llama.cpp non-streaming responses return the requested model id instead of the local .gguf absolute path; registered/imported collections list under their canonical prefixed id; auto backend selection now prefers Vulkan on AMD systems across all engines.

Asset naming is identical between the two releases, so the bump itself is a pure version substitution.

Tests

The old pins had no test coverage at all — nothing failed if a constant was wrong. Added:

  • Derived archive names and URLs match the published shape for the configured version, and a v-prefixed version is accepted anywhere a version is.
  • The dashboard fallback resolves to the same version the engine installs, and its artifact matches the derived archive name — a regression that reintroduced a second constant would fail here.
  • A version change forces re-extraction over an existing runtime tree, while an unchanged version does not.

A stale lemonade version string in an e2e-report test fixture is refreshed alongside; it is self-contained so it never failed, but it kept drifting.

Verification

Ran locally on Linux x86_64 (no GPU):

  • cargo test --workspace --all-targets --no-fail-fast, cargo clippy --locked --workspace --all-targets -- -D warnings, cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings, cargo fmt --all --check.
  • cargo xtask manifest --check and cargo xtask tpn --check — both pass with the new crate. The MANIFEST table lists external crates only, and rocm-deps's only dependency (toml, build-only) is already in the tree.

End to end, against the real release: with no override, rocm-engine-lemonade install downloaded lemonade-embeddable-11.5.1-ubuntu-x64.tar.gz (6512136 bytes) from the upstream release and passed digest verification — so the bumped URL and SHA-256 resolve for real, not just in a unit test.

Temporarily adding two pin entries whose names differ only by separator (lemonade-x and lemonade_x) failed the build with runtime.lemonade-x.version and runtime.lemonade_x.version both generate LEMONADE_X_VERSION; rename one, confirming the generator's collision guard and that editing the pin file rebuilds the crate.

On CI, the self-hosted GPU lanes (MI300X, Strix Halo Ubuntu, Strix Halo Windows) are green on this branch, including the lemonade serve and inference scenarios, which exercise the bumped runtime on real hardware. Note the adapter's own Preparing Lemonade embeddable ... line is not surfaced in the job log, so the log does not independently evidence the 11.5.1 archive download — that is what the local run above covers.

Three tests fail identically on an unmodified origin/main checkout in this environment and are unrelated to this change: rocm-core's proc_lifecycle::tree_stop_waits_for_descendants and tree_forced_kill_reaches_sigterm_ignoring_descendant, and rocm's therock::extracting_the_sdk_archive_removes_it (which passes when run alone, and passes in CI).

Risk: medium. The pin mechanism itself is low risk — a bad pin is a build or install failure, not a silent wrong install — but 11.5.1 is a ten-minor-version jump with the upstream breaking changes listed above.

@rominf rominf changed the title build(deps): bump lemonade embeddable to 11.5.1 build(deps): pin lemonade 11.5.1 from a single source of truth Aug 4, 2026
@rominf
rominf force-pushed the feat/lemonade-11-5-1 branch from 8236118 to 1b8f628 Compare August 11, 2026 16:15
rominf added 4 commits August 12, 2026 10:06
Move the pinned Lemonade embeddable runtime from 10.10.0 to 11.5.1.
Asset naming is unchanged between the two releases, so the download
URLs only need the version substituted.

Also refresh a stale lemonade version in an e2e-report test fixture so
it stops drifting from the pinned version.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The dash offline fallback version is documented to track the latest
release ("Bump deliberately"), but it still named 10.6.0 while the
engine pin moved to 11.5.1. A host whose GitHub releases API call fails
would silently install a runtime several minor versions behind the pin.

Also correct the host-triple mapping doc, which claimed the release
ships only the three archives we select; it also publishes ubuntu-arm64.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The Lemonade embeddable version was written out seven times across five
constants in the engine adapter, and again as an independent constant in
the dashboard TUI. Nothing tied those together, so they drifted: the two
constants sat five minor versions apart, the runtime-id label in the CLI
was four behind both, and a doc comment had been stale since 10.10.0.

Move the pin into `runtime-deps.toml` at the repository root and add a
small `rocm-deps` crate whose build script turns each `[runtime.<name>]`
field into a constant. Archive names, download URLs, the dashboard's
offline fallback, and the managed runtime-id label are all derived from
that one value, so a bump is a one-line edit and the tree can no longer
hold two different Lemonade versions. Generating at build time rather
than committing generated source means there is nothing to drift and no
`--check` gate to keep honest.

The schema is a table per runtime with one or more string fields, so a
runtime needing more than a version (a version plus an ABI tag, say) is
additive rather than a reshape.

No behaviour change beyond the runtime-id label, which now matches the
`env_id` the adapter actually reports instead of naming an older release.

Signed-off-by: Roman Inflianskas <roman.inflianskas@amd.com>
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The embeddable runtime directory is not version-scoped, so after a bump
`prepare_embeddable` downloaded the new archive but skipped extraction
because `lemond` was already there from the previous version. The old
binaries stayed in place and were then reported as the new version. The
CLI made the same mistake one layer up: it asked the adapter whether a
runtime was installed and ignored which version came back, so a bump
never even reached the install path.

Compare versions in both places. The adapter re-extracts when the
recorded version differs from the one being installed, and the CLI now
requires the detected env id to be the one it asked for — which the
previous commit made meaningful by deriving that id from the pin. The
decision in the adapter is a pure function so the upgrade case is tested
without a real archive.

Also guard the pin generator against two entries whose names differ only
by separator (`rocm-abi` and `rocm_abi` both fold to `ROCM_ABI`), and
escape generated literals with Debug formatting instead of a hand-written
rule about which characters are allowed.

Signed-off-by: Roman Inflianskas <roman.inflianskas@amd.com>
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf force-pushed the feat/lemonade-11-5-1 branch from 1b8f628 to 1b7d6e6 Compare August 12, 2026 10:19
@rominf
rominf marked this pull request as ready for review August 12, 2026 11:09
@rominf
rominf requested a review from a team as a code owner August 12, 2026 11:09
@juhovainio

Copy link
Copy Markdown
Collaborator

I reviewed this PR and checked it against a live checkout of the branch, not just the diff.

The core claim holds up: before this PR the Lemonade version was hand-written 7 times across 4 files, and I confirmed the specific drift the PR calls out (engine adapter at 10.10.0, dashboard offline fallback at 10.6.0, runtime-id label in apps/rocm at 10.6.0). The remaining 10.6.0/10.10.0 literals left in the tree are all test fixtures for behavior that does not depend on the actual pin (parsing, string formatting, request/response plumbing), not stale copies of the version.

I traced the new single-source-of-truth mechanism end to end:

  • crates/rocm-deps/build.rs resolves its pin file as ../../runtime-deps.toml relative to CARGO_MANIFEST_DIR, which correctly lands on the repo-root runtime-deps.toml from crates/rocm-deps.
  • The workspace sets publish = false at the root and rocm-deps opts in via publish.workspace = true, so a build.rs that reads a file outside its own crate directory is not a packaging problem here, since the crate is never published.
  • The collision guard (two differently-spelled TOML keys folding to the same constant name) and the switch to {:?} for literal escaping both look correct.
  • managed_engine_runtime_id's return type change from &'static str to String is threaded through both call sites (resolve_engine_install_runtime_id and ensure_self_managed_engine_ready) without a leftover .to_owned() mismatch.
  • The real bug fix (embeddable runtime dir not being version-scoped) is sound: prepare_embeddable now reads the previously-installed version from the manifest, guarded by runtime_dir == manifest.runtime_dir so a manifest from a different env_root can't be mistaken for a match, and needs_extraction correctly forces re-extraction on a version change. Separately, ensure_self_managed_engine_ready now also requires detect.env_id to match the pinned runtime_id; I confirmed detect_response actually sources env_id from the manifest's env_id field (lemonade-embeddable-), so this comparison will actually catch a stale install rather than always matching or never matching.

I also built and ran the affected crates (rocm-deps, rocm-engine-lemonade, rocm-dash-tui, and the rocm binary's lemonade-related tests) rather than just reading the diff — everything compiles and all tests pass, including the new coverage for version-triggered re-extraction and the build.rs collision guard.

I didn't find anything blocking. CI is green and this looks safe to approve pending final validation of the live 11.5.1 install path itself, which the PR description says was already tested manually against the real release.

@rominf
rominf added this pull request to the merge queue Aug 12, 2026
Merged via the queue into main with commit 12ae31e Aug 12, 2026
23 checks passed
@rominf
rominf deleted the feat/lemonade-11-5-1 branch August 12, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants