learning integrity: sandbox leak closure, observable-only goal + settle certificate, prompt surfaces - #150
Merged
Merged
Conversation
yichao-liang
force-pushed
the
bridge-learning-integrity
branch
from
August 20, 2026 21:04
d6736cb to
f5738c1
Compare
yichao-liang
force-pushed
the
engaging-slurm-config
branch
2 times, most recently
from
August 20, 2026 21:22
255b4c6 to
8589ce3
Compare
yichao-liang
force-pushed
the
bridge-learning-integrity
branch
from
August 20, 2026 21:22
f5738c1 to
619cd5a
Compare
The first bridge runs died at the common 500-step cap mid-plan (12 of 19 options already cost ~483 steps), before any cure Wait could complete, so no episode ever produced glue evidence. Override the cap per-env next to the other bridge step budgets, and un-park the predicate-invention arm for the next sweep.
Sessions now carry a three-way phase tag (solve/explore/synthesis)
instead of two: get_interaction_requests flips _explore_phase around
explorer creation, so explore sessions log their system prompt
separately instead of overwriting the solve copy. The mixin records
the phase a live session was built with and rebuilds on mismatch, so
prompt tag, tool surface, and CLAUDE.md always match the active phase.
Drop the redundant system_prompt_{phase}.md copies: sandbox sessions
already write full_system_prompt_{phase}.md during setup, and
in-process sessions now write the same name (their approach prompt IS
the full prompt) - exactly one prompt file per phase.
Deduplicate across the three always-in-context surfaces (system prompt, CLAUDE.md, query message) so each instruction has one home: - Sandbox mechanics live in CLAUDE.md only; the system-prompt suffix shrinks to the tool list plus a pointer. - The explore query's four overlapping sections (Exploration Setting, Belief-Model Simulator, delivery paragraph, probe guidance) merge into one Exploration Setting section covering loop context and early stopping, belief-model semantics, and experiment design (front-load the unpredictable step, make its outcome observable). - The query Instructions no longer restate the system prompt's Subgoal Annotations contract, Wait rule, or parameter-effort guidance; a one-paragraph binder plus the format block remain. - The synthesis prompt defines the recurrent rule signature once (the Recurrent rules section now references it), defers the threshold-fitting protocol to CLAUDE.md's canonical copy from both restatement sites, and trims the learn message's duplicated snapshot/probe/goal-check text to pointers. - Fix a coherence gap: the synthesis Workflow list now tells invention arms that step 4's sketches need predicates.py written and loaded first (new _synthesis_workflow_extra hook). - Render '(none)' for an empty Initial State Atoms section.
The 20260819 bridge runs read ground truth through three holes instead of learning it: seed1 dumped the env's cure counters off recorded states' State.privileged; seed2/seed0 reached the live env via sim._ctx.env, read cure_threshold/apply_glue_radius, and seed2 flipped _skip_domain_specific_dynamics to run true dynamics in its belief sim; and reference/options.py spelled out the wetting law in prose. - sandbox_guard: screen executed code for privileged / _ctx / _skip_domain_specific_dynamics (word-bounded), with a deny message pointing back at the observable surface; covered by tests. - CLAUDE.md rules: state upfront that harness internals are off-limits and that conclusions derived from them are invalid results. - bridge options.py + skill_factories: scrub hidden-mechanics prose (wetting radius/streak, cure gates, welds) from the files copied verbatim into the sandbox; the glue dwell now comes from a neutrally-named env classmethod instead of wet_streak_steps + 1.
…stop Remove the two Attached atoms from the bridge goal. They were the only goal atoms whose classifier reads state.privileged, which made the goal unreachable for any learned belief model by construction: residual rules can only write observable features, so evaluate_option_plan could never report success, mental_model_solved was unattainable, early stopping could never fire, and the 20260819 runs burned ~4 of 5 test attempts per seed re-proving that. The welds stay physically required (the unwelded middle span cannot stay in the NextToEnd window), so the geometric goal implies them. The naive removal opens an exploit: the base sim removes the grasp constraint AFTER an action's substeps, so the release step's observation shows an unheld DRY span at its exact placement pose and cogman's per-step goal check would score a dry bridge SOLVED. Close it with a settle certificate: check_episode_trajectory steps raw physics (no actuation, no tack, no cure) at episode end and re-checks the goal, so the structure must stand by its cured welds alone. Covered by a test that builds the dry and welded layouts and asserts reject/accept. Also add wait_option_max_steps as a backstop for TARGETED Waits. The cap previously existed only in the untargeted any-atom-change branch; a Wait annotated with an atom the world never produces (e.g. a learned latent-only predicate) spun until the episode budget killed it - seed1 lost ~250 steps and its co-motion probes this way. Default (inf) is unchanged. Also cap the parallel hill-climbing pool at min(cpus, items, 32): mp.cpu_count() on a 448-core shared node exceeds the per-user process rlimit and test_run_hill_climbing died with fork EAGAIN (pre-existing failure, reproduced on a clean tree).
The demonstrator's episodes go through check_episode_trajectory now, so assert the genuinely-built bridge (welded butt joints, unwelded seat joints resting by gravity) survives the settle certificate too, not just the per-step goal check.
Explore-mode prompts no longer open with "You are solving a task": they open with an exploration framing where achieving the goal is the most informative experiment, so solving is part of information gathering. The early-stop note now credits the exploration plans (not "the learned model") with solving training, matching main.py's criterion, and the Exploration Setting states the real refine-then- truncate mechanism: refinement drops steps beyond the deepest annotated subgoal the belief model cannot establish, which is why the unknown step should come early.
Sessions are per-phase (_ensure_agent_session rebuilds on phase change), so an explore session can never receive a solve query; the dual "Queries arrive in two kinds" framing was stale. The bilevel system prompt now branches on _explore_phase: solve sessions get the capture-gate contract only, explore sessions get the experiment-text contract only, and the identity line matches (exploration agent vs planning agent). The shared evaluate_option_plan mechanics stay in both since both phases carry the tool.
The explore/solve prompts already rendered unwrapped paragraphs, but the sandbox suffix, the synthesis system prompt template, and the sandbox CLAUDE.md carried source-width manual line breaks. The sandbox suffix is now authored unwrapped, and unwrap_prose_lines() joins hard-wrapped prose at render time for the synthesis template and CLAUDE.md while preserving code fences, headings, lists, tables, and indented lines verbatim.
run_python/explore_python only invoked the escape screen when a sandbox_dir existed, so an in-process session without a sandbox copy got NO screening at all - while its exec namespace still binds states whose privileged channel carries hidden ground truth and a probe whose _ctx.env is the live env. The hidden-state, introspection, and hidden-module-import screens do not depend on a sandbox directory existing; only the path screen has no boundary to enforce without one. Screen unconditionally: _screen_text_for_sandbox_escape now accepts sandbox_dir=None and skips just the path screen. The screen remains a best-effort heuristic (in-process exec has no hard boundary); stripping privileged from exec-visible states is the robust follow-up.
The note telling editors not to document the wetting/curing mechanics (radii, streaks, thresholds) is itself copied verbatim into the agent sandbox as reference/options.py - handing the agent the hidden mechanism's structure as a head start. Say 'any hidden dynamics' instead.
Replace the em dashes this branch introduced in sandbox_prompts, sketch_prompts, and sandbox_guard with plain dashes (repo policy), and apply the formatter fixes CI would demand: yapf reflows in pybullet_bridge and two test files, an isort fix in test_sandbox_prompts, and docformatter reflows across the files this range touched.
yichao-liang
force-pushed
the
bridge-learning-integrity
branch
from
August 20, 2026 21:51
619cd5a to
4ca936c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 5/6 of the bridge-domain stack (based on #149).
Review fixes folded in on top:
🤖 Generated with Claude Code