docs: Tutorial 28 - The Scholarship Illusion (sharp and fuzzy RDD)

[igerber]

Summary

• New Tutorial 28 (docs/tutorials/28_rdd_scholarship_illusion.ipynb): the regression discontinuity walkthrough. A merit scholarship is awarded at an exam-score cutoff; the naive above-vs-below comparison claims $14,779 while the designed truth is a $2,640 offer ITT - the tutorial recovers it with sharp RD, then the $4,000 complier LATE with fuzzy RD.
• Acts: DiD-vs-RDD positioning (when an assignment rule replaces a panel), the naive comparison, RDPlot (full-support + zoomed p=2, ci=95 views), sharp RD with robust bias-corrected inference and the three-row summary explained, the validity toolkit (estimator-as-balance-test, illustrative within-side-median placebo cutoffs, bandwidth sweep - all framed as falsification/supporting evidence with continuity itself untestable), fuzzy RD via takeup= with the full local-IV assumption bundle, and CCFT 2019 covariate adjustment (testable sufficient condition; simulation-specific 27% CI shrink).
• New drift suite tests/test_t28_rdd_scholarship_illusion_drift.py (12 tests): pins every headline number, executes the notebook's actual DGP cell and compares the full DataFrame against the test's faithful copy, covers both RDPlot fits (bin counts 123/163 and 72/76, finite means/CIs, Agg render smoke test), pins bandwidth-sweep CI endpoints with truth coverage, and locks the load-bearing prose qualifiers on the rendered surface.
• Registrations: docs/tutorials/index.rst (Advanced Methods card + toctree), CHANGELOG.md (Unreleased/Added), docs/doc-deps.yaml (tutorial rows under diff_diff/rdd.py and diff_diff/rdplot.py).

Methodology references (required if estimator / math changes)

• Method name(s): N/A - no estimator/math changes (documentation + tests only; tutorial teaches existing RegressionDiscontinuity and RDPlot)
• Paper / source link(s): Tutorial cites CCT 2014 (Econometrica), CCT 2015 (JASA), CCFT 2019 (REStat), Cattaneo-Idrobo-Titiunik 2020, McCrary 2008, Cattaneo-Jansson-Ma 2020, Thistlethwaite & Campbell 1960
• Any intentional deviations from the source (and why): None; the tutorial's description of the robust-row canonical binding and the absent density test matches the documented REGISTRY.md notes

Validation

• Tests added/updated: tests/test_t28_rdd_scholarship_illusion_drift.py (12 tests, all passing; deterministic single fits, no slow markers needed)
• Backtest / simulation / notebook evidence (if applicable): Notebook executed end-to-end with 0 errors and 0 stderr blocks against this branch; simulated DGP has known truth by construction (sharp ITT $2,629 vs designed $2,640; fuzzy LATE $3,765 vs designed $4,000; first stage 0.701 vs designed 0.66 jump); sphinx -b html -W docs build passes; test_docs_ia.py + test_doc_deps_integrity.py green

Security / privacy

• Confirm no secrets/PII in this PR: Yes

[github-actions]

PR Review

Overall assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive summary

• No estimator, weighting, variance, or default behavior changes.
• Sharp/fuzzy RD, RBC inference, RDPlot, and covariate adjustment agree with the Registry and cited papers.
• Two P2 tutorial-clarity issues should be considered.
• Documented robust-row and density-test deviations are not defects.
• No security or untracked technical-debt concerns found.

Methodology

P2 — Fuzzy treatment is enrollment, not scholarship receipt

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L134-L160, L686-L703
• Impact: takeup="enrolled" estimates the complier LATE of enrollment/program participation. The DGP also gives the $4,000 effect to below-cutoff students using outside aid, yet several labels call it the effect of “using the scholarship” or “the scholarship itself.” Readers could report the wrong causal estimand.
• Concrete fix: Rename the section and constant/comments to “effect of enrollment/program participation,” or model scholarship receipt as the actual take-up variable.

P2 — Bandwidth sweep implicitly changes both h and b

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L586-L592, L630-L650
• Impact: Passing only h sets b=h (REGISTRY.md:L4231). With q=p+1, CCT Remark 7 makes these RBC results numerically equivalent to the higher-order conventional fit (REGISTRY.md:L4444-L4447). The sweep therefore is not sensitivity solely to the main local-linear bandwidth and is not directly comparable to the default fit’s selected (h,b) pair.
• Concrete fix: Explain the b=h implication. If the intent is proportional or fixed bias bandwidth sensitivity, pass an explicit b or rho.

P3 — Documented deviations

• Location: REGISTRY.md:L4254, L4404
• Impact: Canonical fields using the robust row and the missing packaged density test are explicitly documented deviations/seams.
• Concrete fix: None required.

Code Quality

No findings.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No untracked deferred work found.

Security

No findings; no secrets or executable HTML/JavaScript outputs detected.

Documentation/Tests

The drift suite has strong coverage of headline estimates, configurations, rendered prose, and notebook synchronization. Notebook JSON and Python syntax validated successfully.

Runtime tests could not be executed locally because pytest, NumPy, and pandas are unavailable in the review environment; required CI should confirm them.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: 8ca72e33d559529b64d6694d5910f002dbb4cc88

---

PR Review

Overall assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive summary

• Both previous P2 findings are resolved.
• Sharp/fuzzy RD, RBC inference, RDPlot, and CCFT covariate adjustment match the Registry and cited papers.
• No estimator or statistical behavior changes.
• One minor naming issue remains; it does not affect results.
• Static notebook/ Python validation passed; runtime tests were unavailable locally.

Methodology

P3 — Documented deviations

• Location: docs/methodology/REGISTRY.md:L4126-L4625
• Impact: Robust-row canonical binding and the absent packaged density test are documented behavior, not defects. The tutorial now correctly explains b=h and funded-enrollment LATE.
• Concrete fix: None required.

Code Quality

P2 — P_COMPLY is not the complier proportion

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L137-L177, tests/test_t28_rdd_scholarship_illusion_drift.py:L52-L74
• Impact: 0.72 is offered-side enrollment probability; the complier share is 0.72−0.06=0.66. The current name could confuse future maintenance, although calculations and prose are correct.
• Concrete fix: Rename it to P_ENROLL_IF_OFFERED, optionally defining P_COMPLIER = P_ENROLL_IF_OFFERED - P_ALWAYS.

Performance

No findings.

Maintainability

No additional findings.

Tech Debt

No untracked deferred work found.

Security

No findings or exposed secrets detected.

Documentation/Tests

The 12-test drift suite covers headline estimates, notebook synchronization, configurations, plots, and rendered claims. Notebook JSON and test syntax validated successfully.

Runtime tests could not be run because pytest, NumPy, and pandas are unavailable in the review environment.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: d362796019ca533f98d5178006b3abc37245f166

---

PR Review

Overall assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive summary

• The previous P_COMPLY naming concern is resolved.
• No estimator, variance, inference, or default behavior changes.
• Sharp/fuzzy RD, RDPlot, RBC inference, and covariate adjustment match the Registry and cited papers.
• No edge-case, security, or untracked-debt findings.
• Static notebook validation passed; runtime tests were unavailable because scientific Python dependencies are not installed.

Methodology

P3 — Documented implementation differences

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L593-L710, docs/methodology/REGISTRY.md:L4231-L4254, docs/methodology/REGISTRY.md:L4400-L4406
• Impact: Robust-row canonical binding, h implying b=h, and the absent density test are accurately disclosed Registry notes, not defects. Fuzzy-IV assumptions and CCFT covariate adjustment are described correctly.
• Concrete fix: None required.

Code Quality

P3 — Previous naming finding resolved

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L137-L177, tests/test_t28_rdd_scholarship_illusion_drift.py:L52-L74
• Impact: P_ENROLL_IF_OFFERED now correctly denotes 0.72, while the 0.66 complier share is separately documented.
• Concrete fix: None required.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No untracked deferred work found.

Security

No findings or exposed secrets detected.

Documentation/Tests

No findings. Notebook JSON, code-cell syntax, committed outputs, rendered claims, configuration locks, and dependency-map links passed static validation. Runtime tests could not run because pytest, NumPy, pandas, SciPy, and matplotlib are unavailable.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: d2d18605e31aa59ad70ab3e50625d12faef6dea3

---

PR Review

Overall assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive summary

• The previous P_COMPLY naming concern is resolved.
• No estimator, inference, variance, or default behavior changed.
• Sharp/fuzzy RD, RDPlot, RBC inference, and CCFT adjustment match the Methodology Registry.
• Static notebook and drift-surface validation passed.
• Runtime tests could not run because scientific Python dependencies are unavailable.

Methodology

P3 — Documented implementation differences

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L385-L425, :L593-L685; docs/methodology/REGISTRY.md:L4231-L4265, :L4400-L4406
• Impact: Robust-row canonical binding, h implying b=h, and the unavailable density test are accurately disclosed Registry notes, not defects. Fuzzy-IV assumptions and CCFT covariate adjustment also match the cited methods.
• Concrete fix: None required.

Code Quality

P3 — Previous naming finding resolved

• Location: docs/tutorials/28_rdd_scholarship_illusion.ipynb:L132-L177; tests/test_t28_rdd_scholarship_illusion_drift.py:L48-L54
• Impact: P_ENROLL_IF_OFFERED correctly denotes 0.72, while the 0.66 complier share is separately identified.
• Concrete fix: None required.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No untracked deferred work. The absent manipulation-density diagnostic is explicitly documented in the Registry.

Security

No findings or exposed secrets detected.

Documentation/Tests

No findings. Notebook JSON, all 11 code-cell ASTs, execution order/output health, 33 drift strings/configurations, and tutorial registration passed static validation. Runtime pytest was unavailable because NumPy, pandas, SciPy, matplotlib, and pytest are not installed.