Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7 event-study horizons, confirming the no-finite-sample-factor convention (REGISTRY
LPDiD Deviation 2). This is the repo's first Stata parity arm — `teffects` is native
(no SSC dependency) and goldens are committed, so CI needs no Stata.
- **4.0 API-unification program, Phase 2 results-contract foundation.** Three new
public symbols in `diff_diff.results_base`: `BaseResults` (shared estimator-results
base, behaviorally inert in 3.9), `Diagnostic` (marker base applied to the diagnostic
result roster - Bacon, RDPlot, HonestDiD, pre-trends/power, placebo, HAD pretests,
DiagnosticReport results - which carry no inference row), and `EventStudyResults` (the
unified per-event-time representation, with package-internal builders over all 14
event-study producers; public exposure rides `aggregate(type="event_study")` in a
later Phase 2 PR). Seven estimator results classes gained `to_dict()`
(CallawaySantAnna, StackedDiD, ContinuousDiD, SunAbraham, Wooldridge,
ChaisemartinDHaultfoeuille, Bacon). Ledger row M-091 flips to `done`; M-092 (unified
event-study surface) and M-093 (4.0 sentinel retirement, phase 5) are added.
**Estimator equations, weighting, variance, and numerical output are unchanged.**

### Changed
- **Diagnostic input validation in the report consumers (4.0 program Phase 2).**
`BusinessReport`, `practitioner_next_steps`, and `DiagnosticReport` now route
diagnostics by the new `Diagnostic` marker instead of by result-class name. Passing a
marked non-Bacon diagnostic (e.g. `HonestDiDResults`, `PlaceboTestResults`) as the
primary estimator input now raises `TypeError` - previously `BusinessReport`
special-cased only Bacon by name, and `DiagnosticReport` silently produced a
zero-check report. Estimator results are unaffected; Bacon's existing read-out is
retained.
- **Internal tracking docs reorganized (no library behavior change).** `TODO.md` is now
the actionable backlog only; blocked / parked work and won't-fix decisions moved to the
new root-level `DEFERRED.md` (deferral & decision registry, same blocker sections plus
Expand Down
1 change: 1 addition & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Related tracking surfaces:
|-------|----------|--------|--------|----------|
| `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
| `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low |
| `EventStudyResults` full-vcov + df round-trip for more producers (4.0 program Phase 2 follow-up). StackedDiD and TwoStageDiD compute a full event-study VCV (regression / Gardner GMM) internally but retain only marginal SEs on their result containers, so their unified surfaces carry `vcov=None`; persist the event-time submatrix + ordered horizon index on those containers and thread it in. Likewise thread producer-specific inference df into `EventStudyResults.df` (MPD `inference_df`, explicit-survey CS `survey_metadata.df_survey`, HC2-BM horizon-specific contrast df). The adapter is faithful to what containers expose today; this is additive container work, not a defect. | `diff_diff/results_base.py`, `diff_diff/stacked_did_results.py`, `diff_diff/two_stage_results.py` | #M-092 | Mid | Low |

### Performance

Expand Down
9 changes: 9 additions & 0 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,11 @@
SpilloverDiDResults, # re-export
SyntheticDiDResults,
)
from diff_diff.results_base import (
BaseResults,
Diagnostic,
EventStudyResults,
)
from diff_diff.spillover import (
SpilloverDiD,
)
Expand Down Expand Up @@ -577,6 +582,10 @@
"MeridianROIPrior",
# LLM guide accessor
"get_llm_guide",
# Results-contract foundations (4.0 program Phase 2, spec sections 3.5/5)
"BaseResults",
"Diagnostic",
"EventStudyResults",
]

# Agent-facing entrypoints surface first in dir(diff_diff). LLM agents
Expand Down
22 changes: 22 additions & 0 deletions diff_diff/_reporting_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@

from typing import Any, Dict

from diff_diff.results_base import Diagnostic


def describe_target_parameter(results: Any) -> Dict[str, Any]:
"""Return the target-parameter block for a fitted result.
Expand Down Expand Up @@ -655,6 +657,26 @@ def describe_target_parameter(results: Any) -> Dict[str, Any]:
"reference": "Butts (2021); REGISTRY.md Sec. SpilloverDiD",
}

# Marked diagnostic results (spec section 3.5): no causal-effect
# inference row exists, so no ATT-style target parameter applies.
# The named ``BaconDecompositionResults`` branch above returns first
# (its block documents the TWFE coefficient being decomposed). This
# branch is defensive depth for direct callers: BR / DR reject
# marked non-Bacon diagnostics before ever calling this helper.
if isinstance(results, Diagnostic):
return {
"name": f"Diagnostic result ({name})",
"definition": (
f"``{name}`` is a diagnostic result: it assesses a design, "
"an identifying assumption, or robustness, and carries no "
"causal-effect inference row. There is no target parameter; "
"interpret it alongside the primary estimator's results."
),
"aggregation": "diagnostic",
"headline_attribute": "",
"reference": "docs/v4-design.md Sec. 3.5",
}

# Default: unrecognized result class. Fall through with a neutral
# block — agents / downstream consumers can still dispatch on
# ``aggregation="unknown"`` and fall back to generic ATT narration.
Expand Down
33 changes: 32 additions & 1 deletion diff_diff/bacon.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import pandas as pd

from diff_diff.results import _format_survey_block
from diff_diff.results_base import Diagnostic
from diff_diff.utils import pre_demean_norms, snap_absorbed_regressors
from diff_diff.utils import within_transform as _within_transform_util

Expand Down Expand Up @@ -76,7 +77,7 @@ def __repr__(self) -> str:


@dataclass
class BaconDecompositionResults:
class BaconDecompositionResults(Diagnostic):
"""
Results from Goodman-Bacon decomposition of TWFE.

Expand Down Expand Up @@ -270,6 +271,36 @@ def _total_weight(self) -> float:
"""Calculate total weight (should be 1.0)."""
return sum(c.weight for c in self.comparisons)

def to_dict(self) -> Dict[str, Any]:
"""
Convert the decomposition headline to a dictionary.

Returns
-------
Dict[str, Any]
TWFE estimate, per-comparison-type weights and averages, and
counts. This is a diagnostic container - there is no causal
inference row. The per-comparison table is available via
``to_dataframe()``.
"""
result = {
"twfe_estimate": self.twfe_estimate,
"total_weight_treated_vs_never": self.total_weight_treated_vs_never,
"total_weight_earlier_vs_later": self.total_weight_earlier_vs_later,
"total_weight_later_vs_earlier": self.total_weight_later_vs_earlier,
"weighted_avg_treated_vs_never": self.weighted_avg_treated_vs_never,
"weighted_avg_earlier_vs_later": self.weighted_avg_earlier_vs_later,
"weighted_avg_later_vs_earlier": self.weighted_avg_later_vs_earlier,
"n_timing_groups": self.n_timing_groups,
"n_never_treated": self.n_never_treated,
"n_obs": self.n_obs,
"n_comparisons": len(self.comparisons),
"decomposition_error": self.decomposition_error,
}
if self.n_always_treated_remapped:
result["n_always_treated_remapped"] = self.n_always_treated_remapped
return result

def to_dataframe(self) -> pd.DataFrame:
"""
Convert comparisons to a DataFrame.
Expand Down
28 changes: 21 additions & 7 deletions diff_diff/business_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@

from diff_diff._reporting_helpers import describe_target_parameter
from diff_diff.diagnostic_report import DiagnosticReport, DiagnosticReportResults
from diff_diff.results_base import Diagnostic

BUSINESS_REPORT_SCHEMA_VERSION = "2.0"

Expand Down Expand Up @@ -92,9 +93,11 @@ class BusinessReport:
Parameters
----------
results : Any
A fitted diff-diff results object. Any of the 16 result types is
accepted. ``BaconDecompositionResults`` is not a valid input — Bacon
is a diagnostic, not an estimator; use ``DiagnosticReport`` for that.
A fitted diff-diff ESTIMATOR results object. Diagnostic results
(anything subclassing ``diff_diff.Diagnostic``, e.g.
``BaconDecompositionResults``, ``HonestDiDResults``) are rejected
by type — pass diagnostics via ``diagnostics=`` or
``DiagnosticReport(precomputed=...)`` instead.
outcome_label : str, optional
Stakeholder-friendly outcome name (e.g. ``"Revenue per user"``).
outcome_unit : str, optional
Expand Down Expand Up @@ -174,11 +177,22 @@ def __init__(
survey_design: Optional[Any] = None,
precomputed: Optional[Dict[str, Any]] = None,
):
if type(results).__name__ == "BaconDecompositionResults":
# Marked diagnostic results are rejected BY TYPE (spec section
# 3.5, ledger row M-091): BusinessReport's primary input is a
# fitted ESTIMATOR result carrying the canonical inference row.
if isinstance(results, Diagnostic):
if type(results).__name__ == "BaconDecompositionResults":
raise TypeError(
"BaconDecompositionResults is a diagnostic, not an estimator; "
"wrap the underlying estimator with BusinessReport and pass the "
"Bacon object to DiagnosticReport(precomputed={'bacon': ...})."
)
raise TypeError(
"BaconDecompositionResults is a diagnostic, not an estimator; "
"wrap the underlying estimator with BusinessReport and pass the "
"Bacon object to DiagnosticReport(precomputed={'bacon': ...})."
f"{type(results).__name__} is a diagnostic result, not an "
"estimator result; BusinessReport takes the fitted "
"estimator's results as its primary input. Pass diagnostic "
"objects via the diagnostics= parameter (as a "
"DiagnosticReport) or interpret them alongside the report."
)

if diagnostics is not None and not isinstance(
Expand Down
39 changes: 38 additions & 1 deletion diff_diff/chaisemartin_dhaultfoeuille_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import pandas as pd

from diff_diff.results import _get_significance_stars
from diff_diff.results_base import BaseResults

__all__ = [
"ChaisemartinDHaultfoeuilleResults",
Expand Down Expand Up @@ -180,7 +181,7 @@ class DCDHBootstrapResults:


@dataclass
class ChaisemartinDHaultfoeuilleResults:
class ChaisemartinDHaultfoeuilleResults(BaseResults):
"""
Results from de Chaisemartin-D'Haultfoeuille (dCDH) Phase 1 estimation.

Expand Down Expand Up @@ -1504,6 +1505,42 @@ def _render_honest_did_section(self, lines: List[str], width: int, thin: str) ->
# to_dataframe
# ------------------------------------------------------------------

def to_dict(self) -> Dict[str, Any]:
"""
Convert headline results to a dictionary.

Returns
-------
Dict[str, Any]
Canonical inference row plus scalar metadata; joiner / leaver /
placebo decompositions are included only when available.
Detailed tables are available via ``to_dataframe(level=...)``.
"""
result = {
"att": self.att,
"se": self.se,
"t_stat": self.t_stat,
"p_value": self.p_value,
"conf_int_lower": self.overall_conf_int[0],
"conf_int_upper": self.overall_conf_int[1],
"n_obs": self.n_obs,
"n_treated_obs": self.n_treated_obs,
"n_switcher_cells": self.n_switcher_cells,
"n_cohorts": self.n_cohorts,
"L_max": self.L_max,
"alpha": self.alpha,
}
if self.joiners_available:
result["joiners_att"] = self.joiners_att
result["joiners_se"] = self.joiners_se
if self.leavers_available:
result["leavers_att"] = self.leavers_att
result["leavers_se"] = self.leavers_se
if self.placebo_available:
result["placebo_effect"] = self.placebo_effect
result["placebo_se"] = self.placebo_se
return result

def to_dataframe(self, level: str = "overall") -> pd.DataFrame:
"""
Convert results to a DataFrame at the requested level of aggregation.
Expand Down
4 changes: 3 additions & 1 deletion diff_diff/changes_in_changes_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@
import numpy as np
import pandas as pd

from diff_diff.results_base import BaseResults

_ESTIMATOR_TITLES = {
"cic": "Changes-in-Changes (Athey & Imbens 2006) Results",
"qdid": "Quantile Difference-in-Differences (QDiD) Results",
}


@dataclass
class ChangesInChangesResults:
class ChangesInChangesResults(BaseResults):
"""Results for :class:`~diff_diff.changes_in_changes.ChangesInChanges` and
:class:`~diff_diff.changes_in_changes.QDiD`.

Expand Down
41 changes: 40 additions & 1 deletion diff_diff/continuous_did_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import pandas as pd

from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults

__all__ = ["ContinuousDiDResults", "DoseResponseCurve"]

Expand Down Expand Up @@ -78,7 +79,7 @@ def to_dataframe(self) -> pd.DataFrame:


@dataclass
class ContinuousDiDResults:
class ContinuousDiDResults(BaseResults):
"""
Results from Continuous Difference-in-Differences estimation.

Expand Down Expand Up @@ -369,6 +370,44 @@ def print_summary(self, alpha: Optional[float] = None) -> None:
"""Print summary to stdout."""
print(self.summary(alpha))

def to_dict(self) -> Dict[str, Any]:
"""
Convert headline results to a dictionary.

Returns
-------
Dict[str, Any]
Canonical ATT inference row, the ACRT companion estimand, and
scalar metadata. Detailed dose-response / event-study tables
are available via ``to_dataframe(level=...)``.
"""
return {
"att": self.att,
"se": self.se,
"t_stat": self.t_stat,
"p_value": self.p_value,
"conf_int_lower": self.overall_att_conf_int[0],
"conf_int_upper": self.overall_att_conf_int[1],
"acrt": self.overall_acrt,
"acrt_se": self.overall_acrt_se,
"acrt_t_stat": self.overall_acrt_t_stat,
"acrt_p_value": self.overall_acrt_p_value,
"acrt_conf_int_lower": self.overall_acrt_conf_int[0],
"acrt_conf_int_upper": self.overall_acrt_conf_int[1],
"n_obs": self.n_obs,
"n_treated_units": self.n_treated_units,
"n_control_units": self.n_control_units,
"control_group": self.control_group,
"treatment_type": self.treatment_type,
"estimation_method": self.estimation_method,
"degree": self.degree,
"num_knots": self.num_knots,
"base_period": self.base_period,
"anticipation": self.anticipation,
"n_bootstrap": self.n_bootstrap,
"alpha": self.alpha,
}

def to_dataframe(self, level: str = "dose_response") -> pd.DataFrame:
"""
Convert results to DataFrame.
Expand Down
Loading
Loading