feat: overridable results_dir/cache_dir roots; anchor SWE artifacts and setup trees on them

[michal2409]

Rewritten per review discussion — the original opt-in results_root_dir field carved out only one of the three kinds of state this server anchors at Path(__file__).parent:

1. packaged read-only assets (setup scripts, prompts, configs) — install-relative is correct;
2. reusable multi-GB setup trees (clones, venvs, toolchains) — a cache, reused across runs and bind-mounted into eval containers by host path;
3. per-run results — written by the server and read across the run, from other nodes on multinode deployments.

Only (1) belongs next to the code, and (2)/(3) often need opposite placement (results on a shared filesystem, caches on fast local or baked container storage) — so one knob can't cover both.

What this does

• nemo_gym: exposes the already-existing RESULTS_DIR / CACHE_DIR constants as overridable top-level config keys (results_dir, cache_dir), defaulted with the same setdefault pattern as uv_cache_dir three lines above, and added to the reserved-keys list. Any server can use them; nothing else changes unless a run overrides them.
• swe_agents: derives both roots from the global config instead of the install dir — run artifacts under results_dir, all setup trees under cache_dir/swe_agents/ via a setup_root property on the shared processor base. parent_dir remains for packaged assets. Both reads fall back to the module constants when the config dict bypassed the parser defaults (env-provided dicts, tests).
• Companion race fix: the OpenHands setup dir is now keyed by agent_framework_commit. file_lock is only held during setup(), so an unkeyed shared checkout could be reset --hard to a different commit by a later run while an earlier one was still executing from it.
• Unit tests: parser defaults + override preservation; the existing setup-cache-hit test now patches setup_root.

Behavior note

Default locations move from the package directory to WORKING_DIR/{results,cache} — the layout the nemo_gym module header documents. Deployments that relied on in-package paths can pin the old locations via the new keys; otherwise setup trees rebuild once at the new default.

We run the multinode configuration in production (64-node Ray clusters with the SWE agents), where results must live on a shared filesystem while the setup trees are baked into the container image — the two-key split is load-bearing for that.

Scope. The keys cover directories consulted through the global config: the SWE agents' artifacts and setup trees, the uv_cache_dir default, and the W&B run dir. Import-time environment defaults (e.g. HF_HOME/HF_DATASETS_CACHE, set in nemo_gym/__init__.py before any config is parsed) intentionally keep using the module constants.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-mfutrega-swe-agents-base-results-dir.docs.buildwithfern.com/nemo/gym

Here are the markdown pages you've updated:

• nemo/gym/reference/configuration