test: anchor ImputationDiD leave-one-out SE against Stata did_imputation (2nd Stata arm)

[igerber]

Summary

• Second Stata parity arm (the first, LPDiD RA-SE, merged as test: anchor LPDiD regression-adjustment SE against Stata teffects (first Stata parity arm) #709), and the first SSC-dependent one. Anchors ImputationDiD(leave_one_out=True) — the Borusyak-Jaravel-Spiess (2024) Supp. App. A.9 leave-one-out variance — which had no runnable R reference (R didimputation omits LOO) and was validated only by an internal psi-identity + hand-calc + MC coverage.
• benchmarks/stata/generate_imputation_loo_golden.do runs the authors' own did_imputation ..., leaveout avgeffectsby(Ei t) cluster(unit) on the committed R-arm panel didimputation_test_panel.csv (no clean-sample reconstruction; the only mapping is Ei = first_treat, missing for never-treated).
• Result: the library LOO SE matches to ~1e-9 (cross-implementation: did_imputation goes through reghdfe, the library through its own sparse IF solver; the point to ~2e-8) across the overall ATT and all 6 event-study horizons. The non-LOO cluster SE is confirmed three-way (library = R = Stata on the same panel).
• First SSC-dependent arm: the chain did_imputation → reghdfe → require + ftools is not pinned by version 19 (SSC has no version history), so the generator does not auto-install (that would fetch latest every run) — install once via benchmarks/stata/requirements.do — and it records each package's version/date in meta.ssc_versions (asserted to be a real version line) for drift detection.
• No diff_diff/ source changed.

Methodology references

• Method name(s): ImputationDiD leave-one-out finite-sample variance (leave_one_out=True).
• Paper / source link(s): Borusyak, Jaravel & Spiess (2024), Revisiting Event-Study Designs, Supplementary Appendix A.9 "Leave-Out Conservative Variance Estimation" (arXiv:2108.12419v5). Reference implementation: Stata did_imputation, leaveout (Borusyak, SSC).
• Intentional deviations from the source: none introduced. REGISTRY's LOO Note is upgraded from inferred ("the authors' Stata ships the same option; there is no CI-runnable anchor") to measured. avgeffectsby(Ei t) is pinned explicitly (== aux_partition="cohort_horizon", also did_imputation's current default) to keep the estimand self-describing and robust to a future default change.

Validation

• Tests added: tests/test_imputation_loo_stata_parity.py (23 cases) — per horizon + overall at abs=1e-7, rel=0 (the repo's imputation SE-parity tolerance): point, LOO SE anchor, non-LOO SE, non-LOO three-way vs R, and a no-A.9-singleton-warning gate proving the LOO rescale is genuinely exercised.
• Golden regenerates byte-identically; existing imputation suite (TestB2024AppendixA9LeaveOneOut) and doc-IA / doc-deps / refresh gates all pass. Stata is node-locked so — like the R arm — the golden is committed and the test skips when absent.
• Notebook evidence: N/A.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01X9m4hRxwqRnEJsv5ZhHY3i

[github-actions]

Overall assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive summary

• Correctly targets ImputationDiD’s BJS Appendix A.9 leave-one-out variance; estimator behavior and defaults are unchanged.
• Stata uses the matching partition and clustering: avgeffectsby(Ei t) cluster(unit).
• Golden values coherently match the library and existing R anchor within documented tolerances.
• SSC reproducibility limitations are transparently recorded through package-version metadata.
• Coarser auxiliary partitions remain unanchored but are properly tracked in TODO.md.

Methodology

No defects. The benchmark matches the Registry’s documented LOO residual rescaling and Stata convention (docs/methodology/REGISTRY.md:L1669).

Code Quality

No findings.

Performance

No findings. The module-scoped fixture limits execution to two estimator fits.

Maintainability

No findings. SSC package drift is explicitly documented and recorded in the golden metadata.

Tech Debt

• Severity: P3 — informational
  Impact: External Stata validation currently covers only the default cohort_horizon partition.
  Concrete fix: None required; extension to cohort and horizon is tracked in TODO.md:L51.

Security

No findings. No secrets or unexpected runtime network operations were introduced.

Documentation/Tests

No findings. Tests cover overall and all six horizons for point estimates, LOO SEs, and non-LOO SEs (tests/test_imputation_loo_stata_parity.py:L134-L209). JSON and Python syntax validated; full pytest execution was unavailable because the review environment lacks project dependencies.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: de9a9975ac6e88c239aa591cd4faad353cf8c84a

---

Overall assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive summary

• ImputationDiD behavior and defaults are unchanged.
• Stata uses the correct LOO option, unit clustering, and cohort-horizon partition.
• Golden values and all-horizon LOO/non-LOO tests are coherent.
• Coarser partition coverage remains properly tracked in TODO.md.
• One minor test-description inconsistency does not affect correctness.

Methodology

No findings. The affected method is ImputationDiD(leave_one_out=True). The command leaveout avgeffectsby(Ei t) cluster(unit) matches the Appendix A.9 implementation and Registry contract (docs/methodology/REGISTRY.md:L1669).

Code Quality

No findings.

Performance

No findings. Module-scoped fixtures limit execution to two estimator fits.

Maintainability

No findings. SSC versions are recorded in the golden for drift detection.

Tech Debt

• Severity: P3 — informational
  Impact: External validation covers only the default cohort_horizon partition.
  Concrete fix: None required; coarser partitions are tracked in TODO.md:L54.

Security

No findings. No secrets or unexpected runtime network operations were introduced.

Documentation/Tests

• Severity: P3 — minor
  Impact: The module overview says every gate covers each horizon, but the direct Stata-versus-R Gate 4 checks only the overall SE (tests/test_imputation_loo_stata_parity.py:L22-L28, L186-L193). The generator also describes the new point gate as comparing against the R golden, while it compares library output to the Stata golden (benchmarks/stata/generate_imputation_loo_golden.do:L39-L40). Core LOO coverage is unaffected.
  Concrete fix: Reword these comments to describe the actual coverage, or parameterize Gate 4 over the six R horizons.

JSON, YAML, and Python syntax validated. Full pytest execution was unavailable because the environment lacks pytest and scientific Python dependencies.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: ed1f0a4ef8b3e3e14efcc5365ebb9fd8ce972b06

---

Overall assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive summary

• No estimator behavior, defaults, weighting, or inference code changed.
• The Stata command matches the Registry’s Appendix A.9 LOO contract and default cohort-horizon partition.
• Golden values, panel dimensions, and LOO/non-LOO comparisons are coherent.
• Prior review findings were resolved: Gate 4 now covers every horizon and generator comments are accurate.
• Coarser partition validation remains properly tracked.

Methodology

No findings. leaveout avgeffectsby(Ei t) cluster(unit) correctly anchors ImputationDiD(leave_one_out=True) against BJS Appendix A.9 (benchmarks/stata/generate_imputation_loo_golden.do:L153-L176, docs/methodology/REGISTRY.md:L1669).

Code Quality

No findings.

Performance

No findings. Module-scoped fixtures limit execution to two estimator fits.

Maintainability

No findings. SSC versions are captured and validated during regeneration (benchmarks/stata/generate_imputation_loo_golden.do:L105-L128).

Tech Debt

• Severity: P3 — informational
  Impact: External validation currently covers only aux_partition="cohort_horizon".
  Concrete fix: None required; coarser partitions are tracked in TODO.md:L54.

Security

No findings. No secrets or automatic runtime package installation were introduced.

Documentation/Tests

No findings. The previous Gate 4 coverage issue is resolved by per-horizon R/Stata checks (tests/test_imputation_loo_stata_parity.py:L196-L201).

Python AST, JSON schema, panel shape, and numerical sanity checks passed. Full pytest execution was unavailable because the environment lacks pytest and scientific Python dependencies.