Skip to content

test: anchor LPDiD regression-adjustment SE against Stata teffects (first Stata parity arm)#709

Merged
igerber merged 1 commit into
mainfrom
stata-lpdid-ra-teffects-anchor
Jul 19, 2026
Merged

test: anchor LPDiD regression-adjustment SE against Stata teffects (first Stata parity arm)#709
igerber merged 1 commit into
mainfrom
stata-lpdid-ra-teffects-anchor

Conversation

@igerber

@igerber igerber commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds the repo's first Stata parity arm to externally anchor the LP-DiD (Dube, Girardi, Jorda & Taylor 2025) regression-adjustment (RA) standard error, which had no runnable R reference and was previously pinned against itself.
  • benchmarks/stata/generate_lpdid_ra_golden.do independently reconstructs each horizon's clean sample (porting the R prep/clean_h recipe to tsset + L./F.; tsfill mirrors R fill_gaps() across the deliberate interior gap) and runs teffects ra (Dy x i.time) (tdiff), atet vce(cluster unit).
  • Result: the library RA IF SE matches teffects to ~1e-16 at all 7 event-study horizons; the SE ratio is a flat 1.0 (no G/(G-1), no (n-1)/(n-k)) — confirming the no-finite-sample-factor convention.
  • teffects is native to Stata (no SSC dependency), so version 19 pins numerical behavior. Stata is node-locked and cannot run in CI, so — exactly like the R arm — the golden is committed and the test skips when absent; the CI-runnable guards (RA_SE_PIN + coverage_lpdid_ra.py) are unchanged.
  • No diff_diff/ source changed.

Methodology references

  • Method name(s): LPDiD regression-adjustment event-study SE (covariate path, reweight=True).
  • Paper / source link(s): Dube, Girardi, Jordà & Taylor (2025), Local Projections Difference-in-Differences — footnote-9 Stata recipe teffects ra (Dhy i.time) (dtreat) if D.treat==1 | Fh.treat==0, atet vce(cluster unit). Reference implementation: Stata teffects ra (native).
  • Intentional deviations from the source: none introduced. REGISTRY LPDiD Deviation 2 is upgraded from inferred (read off the authors' .do comments) to measured: the no-finite-sample-factor RA convention is now confirmed against a runnable teffects to machine precision.

Validation

  • Tests added: tests/test_lpdid_ra_stata_parity.py (22 cases) — three gates per horizon, each a purely absolute bound: point vs the R-anchored ra_cov[h][0] (1e-10), Stata (e(N), e(N_clust)) vs the library (n_obs, n_clusters) (exact), and library SE vs teffects SE (1e-7 cross-platform; ~1e-16 local). Strong corroboration of the same realized clean sample (not a formal membership proof).
  • Golden regenerates byte-identically; existing LPDiD R-parity suite (RA_SE_PIN) and doc-IA / doc-deps / refresh gates all pass.
  • Notebook evidence: N/A.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

https://claude.ai/code/session_01X9m4hRxwqRnEJsv5ZhHY3i

@github-actions

Copy link
Copy Markdown

Overall assessment: ✅ Looks good

Executive summary

  • No estimator or default behavior changed.
  • Stata sample construction matches the paper, Registry, and existing R recipe.
  • Golden ATETs, sample sizes, clusters, and IF SEs independently reproduce within (10^{-15}).
  • No P0/P1 findings.
  • Full pytest execution was unavailable because the review image lacks dependencies.

Methodology

  • P3 — Documented implementation choice
    • Impact: The no-finite-sample-factor RA variance is explicitly documented and externally anchored against Stata at REGISTRY.md:L2130. It is not a defect.
    • Concrete fix: None required.

The post/pre clean-control filters and teffects ra ... atet vce(cluster unit) specification at generate_lpdid_ra_golden.do:L85 conform to the cited methodology.

Code Quality

No findings.

Performance

No findings. The library fit is module-scoped and reused across parameterized tests.

Maintainability

No findings.

Tech Debt

  • P3 — Tracked
    • Impact: The optional R reference remains explicitly tracked in DEFERRED.md:L60; it does not block approval.
    • Concrete fix: None required.

Security

No secrets, unsafe external commands, or sensitive data found.

Documentation/Tests

  • P3 — Minor test hardening
    • Impact: The new point gate compares Stata against R, while library_fit retains only SE/counts at test_lpdid_ra_stata_parity.py:L88. Direct Python–Stata point agreement relies on the existing, looser R-parity test.
    • Concrete fix: Optionally retain each library coefficient and compare it directly with the Stata ATET at POINT_ATOL.

Static syntax, JSON validity, diff hygiene, and documentation dependency paths passed.

…irst Stata parity arm)

The LP-DiD (Dube, Girardi, Jorda & Taylor 2025) regression-adjustment SE carries
no finite-sample factor and has no runnable R reference (alexCardazzi/lpdid does
direct covariate inclusion, not RA). It was previously pinned against itself and
MC-coverage-validated; REGISTRY LPDiD Deviation 2 justified the no-factor
convention by *inferring* from comments in the authors' .do files that Stata
`teffects` applies no factor.

This adds the repo's first Stata parity arm to convert that inference into a
measurement. `benchmarks/stata/generate_lpdid_ra_golden.do` independently
reconstructs each horizon's clean sample (porting the R prep/clean_h recipe to
tsset + L./F.; tsfill mirrors R fill_gaps() across the deliberate interior gap)
and runs `teffects ra (Dy x i.time) (tdiff), atet vce(cluster unit)`. The library
RA IF SE matches teffects to ~1e-16 at all 7 event-study horizons; the SE ratio is
a flat 1.0 (no G/(G-1), no (n-1)/(n-k)), confirming the convention.

teffects is native to Stata (no SSC dependency), so `version 19` pins the numerical
behavior; the running edition is derived from c(MP)/c(SE). Stata is node-locked and
cannot run in CI, so -- exactly like the R arm -- the golden is committed and the
test skips when absent; the CI-runnable guards (RA_SE_PIN + coverage_lpdid_ra.py)
are unchanged.

tests/test_lpdid_ra_stata_parity.py (29 cases) gates per horizon, each as an
absolute bound (rel=0): Stata point vs the R-anchored ra_cov[h][0] (1e-10) AND a
direct library-ATET vs Stata-ATET check (1e-6 cross-platform, so Python<->Stata
point agreement is direct, not transitive through R); Stata (e(N), e(N_clust)) vs
the library (n_obs, n_clusters) (exact, to narrow the "point matches on a different
sample" hole); and library SE vs teffects SE (1e-7 cross-platform; local ~1e-16).
Together these strongly corroborate -- not formally prove -- the same clean sample.

No diff_diff/ source changed. Docs updated: REGISTRY Deviation 2 (inferred ->
measured), DEFERRED.md row (resolved; optional R follow-up only),
dube-2025-review.md, benchmarks.rst + benchmarks/README.md (Stata arm),
doc-deps.yaml (register benchmarks/stata/README.md), CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9m4hRxwqRnEJsv5ZhHY3i
@igerber
igerber force-pushed the stata-lpdid-ra-teffects-anchor branch from 6e9ece3 to cf1cd76 Compare July 19, 2026 21:57
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: cf1cd76cc1a098306a6788c3968395f5bb821c42


Overall assessment: ✅ Looks good

Executive summary

  • No estimator or default behavior changed.
  • Stata sample construction matches the paper, Registry, and R recipe.
  • Golden points, sample counts, clusters, and SEs are internally consistent.
  • The prior direct Python–Stata point-comparison gap is resolved.
  • No P0/P1 findings.

Methodology

  • P3 — Documented implementation choice
    • Impact: The no-finite-sample-factor RA variance is explicitly documented at docs/methodology/REGISTRY.md:L2130 and externally anchored across seven horizons. The post/pre filters at benchmarks/stata/generate_lpdid_ra_golden.do:L105-L130 match the cited clean-sample recipe.
    • Concrete fix: None required.

Code Quality

No findings.

Performance

No findings. The library fit is module-scoped and reused.

Maintainability

No findings.

Tech Debt

  • P3 — Tracked
    • Impact: The optional R implementation remains tracked at DEFERRED.md:L60.
    • Concrete fix: None required.

Security

No secrets, unsafe external commands, or sensitive data found.

Documentation/Tests

  • P3 — Prior informational item resolved
    • Impact: tests/test_lpdid_ra_stata_parity.py:L159-L171 now directly compares recomputed library coefficients with Stata ATETs.
    • Concrete fix: None required.

Static Python syntax, JSON validity, diff hygiene, sample counts, and committed numerical anchors passed. Full pytest execution was unavailable because the review environment lacks NumPy, pandas, SciPy, and pytest.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 19, 2026
@igerber
igerber merged commit 4dedf5b into main Jul 19, 2026
35 of 36 checks passed
@igerber
igerber deleted the stata-lpdid-ra-teffects-anchor branch July 19, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant