Skip to content

Add trap bundle collection/ablation utilities, tests, and bump version to 0.8.4#368

Open
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/add-deterministic-trap-analysis-support-0qh4de
Open

Add trap bundle collection/ablation utilities, tests, and bump version to 0.8.4#368
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/add-deterministic-trap-analysis-support-0qh4de

Conversation

@charlesmartin14
Copy link
Copy Markdown
Member

Motivation

  • Provide a way to capture, persist, and reuse per-layer trap analysis artifacts (SVD, permutations, original/permute matrices) to enable deterministic ablation experiments and reproducible analysis.
  • Expose high-level helpers to remove individual traps or sets of traps from models using precomputed decompositions, and to run ablation experiments across checkpoints.

Description

  • Introduces a new weightwatcher/trap_bundles.py module with TrapAnalysisBundle dataclass and functions analyze_traps_bundle, save_trap_bundle, load_trap_bundle, remove_single_trap_from_bundle, remove_traps_from_bundle, and run_trap_bundle_ablation_experiment to collect, serialize, and apply bundle-based ablations.
  • Extends weightwatcher/trap_analysis.py to accept an optional bundle_collector and to populate it with SVD and permutation artifacts for each analyzed layer, using svd_full and safe permute_ids handling.
  • Adds tests/test_trap_bundles.py with unit tests covering round-trip bundle save/load, seed-independence of ablation vs fresh analysis, mismatch validation, and legacy API behavior; tests are skipped when torch is not available.
  • Bumps package version in weightwatcher/__init__.py from 0.8.3 to 0.8.4.

Testing

  • Ran pytest -q tests/test_trap_bundles.py, and tests passed on environments with torch installed and were skipped when torch was not available.
  • The new bundle collection and ablation code paths exercised by the tests completed without uncaught exceptions under the test conditions.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant