feat: Add unified system for handling runtime assets like pseudopotentials and model weights - #169
Open
sigilmakes wants to merge 26 commits into
Open
feat: Add unified system for handling runtime assets like pseudopotentials and model weights#169sigilmakes wants to merge 26 commits into
sigilmakes wants to merge 26 commits into
Conversation
sigilmakes
force-pushed
the
feat/168-runtime-assets
branch
from
August 14, 2026 11:40
312fdab to
eaa0b69
Compare
Replace mutable PseudoMetadata (library, source_set, is_sssp, sssp_recommended_cutoff) with frozen PseudoMetadata carrying PseudoCutoffs (ecutwfc_ry, ecutrho_ry) and typed table_id. Rename PseudopotentialAdvice to PseudopotentialRequirements and pseudo_mode to pseudo_accuracy. Add PseudoAccuracy, PseudoType, and RelativisticTreatment literal types. Add table_id to PseudoMetadata replacing the stringly-typed pseudo_info dict lookup. Remove AvailablePseudos (source and table_id fields were write-only).
Add PseudoDojo and SSSP importers that validate report/header/table agreement, verify MD5 per UPF, and produce one normalized installed manifest. Add pseudo/registry.toml with 15 validated table declarations. Replace pp_selector.py with source.py: source_for_request returns a Callable resolving one of three sources (explicit metadata, local root, installed table). PseudoTableMismatch reports the specific unsatisfied requirement and matching alternatives. Remove write-only PseudoCutoffs.provenance and hints_ry (PseudoDojo hint levels remain in the installed manifest per #168). Remove unused PseudoTable fields (upstream_url, transfer_bytes, installed_bytes, record, note, covers method).
…tional consistency Merge the Resolve Pseudos graph stage into Select — the stage adapter calls the source resolver and passes metadata to select_pseudopotentials in one step. Select is pure: takes PseudopotentialRequirements and Sequence[PseudoMetadata], returns SelectionRecord with specific diagnostics for each unsatisfied requirement. Add functional consistency enforcement at QE generation: requirement functional must match intent functional, selection coverage must match structure elements, every selected pseudo must have filename and both cutoffs. Rename advise_pseudopotentials to advise_pseudopotential_requirements. CoreRuntime gains asset_store and pseudo_registry_path parameters.
Add --pseudo-table, --pseudo-accuracy, and --fetch-missing CLI options. Add assets install/status/verify lifecycle commands. Map PseudoTableMismatch (422), AssetCorrupt (424), and AssetNotInstalled (424) to structured HTTP responses. Wrap PseudoTableMismatch, PseudoImportError, AssetCorrupt, and AssetNotInstalled in MCP ToolError for structured tool failures. Update server request deserializer and MCP pydantic schemas for the new PseudoMetadata and PseudoCutoffs shapes.
…acts Update conftest, server conftest, and all test fixtures to use PseudoCutoffs (ecutwfc_ry, ecutrho_ry) without provenance/hints_ry. Update PseudoTable fixtures without dead fields. Update selection, generation, runtime, CLI, transport, and integration tests for the renamed types and merged graph stage. Add HTTP tests for PseudoTableMismatch (422) and AssetCorrupt (424). Add MCP tests for table mismatch and missing-asset ToolError. Strengthen asset store: checksum validation, AssetInstallation/ AssetPreparer protocol, atomic corrupt repair.
Document the normalized table lifecycle, source resolution model, --pseudo-table/--pseudo-accuracy CLI options, asset store root, and the simplified SCF pipeline. Update changelog, conventions, tutorial, pipeline, and README examples.
sigilmakes
force-pushed
the
feat/168-runtime-assets
branch
from
August 14, 2026 11:47
eaa0b69 to
b9ce02f
Compare
urllib has no timeout, no retry, and no streaming control — it will hang silently on a slow connection and fail without retry on a transient 503. Replace with requests (stream=True, timeout=300s). Drop certifi as a direct dependency; requests bundles its own trust store.
sigilmakes
force-pushed
the
feat/168-runtime-assets
branch
from
August 14, 2026 12:41
0c70e44 to
dbd5720
Compare
sigilmakes
marked this pull request as ready for review
August 14, 2026 12:43
The Resolve Pseudos stage was merged into Select. Tutorial updated to use pseudo_root instead of the removed load_pseudo_metadata + tuple pattern.
Selection now filters lanthanide/actinide elements to SSSP candidates unconditionally, and ranks SSSP ahead of PseudoDojo for equal candidates, restoring the scientific behavior lost in the asset-store unification. Fallback warnings for these elements explain the 4f-freezing and actinide coverage reasons and name the install and select commands to run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR unifies model weight downloading and the new pseudopotential download system behind a single asset store, rather than having duplicating systems that could end up downloading assets to different locations on the disk.
To do this, the PR introduces a new
assetspackage with anAssetStoreclass which handles downloads, asset integrity checks, and other various responsibilities of a store. All weights, pseudopotentials and any future assets will be stored inXDG_DATA_HOME/goldilocks/assets(which will usually resolve to~/.local/share/goldilocks/assets). Fallback and manually specified locations for the asset store are also available.The model registry has then been moved into the
mlpackage, and refactored to make use of this new asset store. The pseudopotential registry has also been added topseudos.Other minor changes include a rename of the cli to
goldilocksfromgoldilocks-core, the addition of the newgoldilocks assetscommands for acquiring assets, and improvements and rewrites to documentation as needed.(Pseudopotential asset system originally written by Junwen for #160, based on the existing model weights registry system. This is a unification and simplification of the two systems into something reusable and maintainable)
N.B. The MCP/HTTP and python API surfaces do expose the asset store to the user/client, but don't currently expose tools for downloading assets. I'm deferring that until we've built the frontend so I have a proper idea of how I want to handle that. For now,
goldilocks assetsis the only entrypoint for that.Closes #168