Skip to content

feat(analytics): add cc-economics --audit for cache-lifecycle savings#2953

Open
thebigjc wants to merge 3 commits into
rtk-ai:developfrom
thebigjc:feat/analytics-cc-economics-audit
Open

feat(analytics): add cc-economics --audit for cache-lifecycle savings#2953
thebigjc wants to merge 3 commits into
rtk-ai:developfrom
thebigjc:feat/analytics-cc-economics-audit

Conversation

@thebigjc

@thebigjc thebigjc commented Jul 11, 2026

Copy link
Copy Markdown

Summary

  • Adds an opt-in --audit mode to rtk cc-economics that attributes token savings by reading Claude Code / pi session logs (~/.claude/projects/**/*.jsonl and ~/.pi/agent/sessions/**/*.jsonl) instead of using a flat weighted-CPT estimate.
  • Credits each compressed command's saved tokens across subsequent turns based on the turn's actual Anthropic prompt caching tier:
    • Cache write (1.25x): tool-result appearance + cache-eviction rebuilds
    • Cache read (0.10x): steady cached turns
  • Scopes matching to the same project (cwd) with a 60-second window, capped by the authoritative count of actual rtk commands invoked.
  • Removes the vestigial $0 input savings row from the audited output since tool results structurally always enter the cached prefix via cache creation (writes), leaving plain input savings at $0.
  • Fixes a parallel-test environment/database race condition in src/core/tracking.rs tests by synchronizing database-touching tests with a shared Mutex.

Example Output

Running rtk cc-economics --audit displays the session-log audited prompt caching savings breakdown:

[cost] Claude Code Economics
════════════════════════════════════════════════════

  Spent (ccusage):              $8728.40
  Token breakdown:
    Input:                      42.7M
    Output:                     51.0M
    Cache writes:               168.6M
    Cache reads:                9255.0M

  RTK commands:                 7893
  Tokens saved:                 1421.1M

  Audited Savings (Prompt Caching):
  ┌─────────────────────────────────────────────────┐
  │ Cache write savings:   $242.01  (2.8%)          │
  │ Cache read savings:    $925.66  (10.6%)         │
  │ ─────────────────────────────────────────────── │
  │ Total savings:         $1167.67  (13.4%)        │
  │ Claude rtk calls:      2236/2260 matched        │
  │ Derived input CPT:     $6.09/MTok               │
  └─────────────────────────────────────────────────┘

  How it works:
  RTK compresses CLI outputs before they enter Claude's context.
  Savings derived using API price ratios (out=5x, cache_w=1.25x, cache_r=0.1x).

Test plan

  • cargo fmt --all && cargo clippy --all-targets && cargo test (all 2437+ tests pass successfully)
  • Manual testing:
    • Verified that rtk cc-economics --audit parses session logs correctly, maps rtk commands to Claude turns, and displays precise caching savings.

Adds an opt-in `--audit` mode to `rtk cc-economics` that attributes token
savings by reading Claude Code session logs (~/.claude/projects/**/*.jsonl)
instead of the flat weighted-CPT estimate.

Each rtk-compressed command's saved tokens are credited across the session
turns that follow it, bucketed by each turn's actual Anthropic pricing tier:
  - cache write (1.25x): tool-result appearance + cache-eviction rebuilds
  - cache read  (0.10x): steady cached turns

Matching is bounded by an authoritative cap: only Claude Bash events invoked
as `rtk ...` can match, scoped to the same project (cwd), so matched count can
never exceed the number of rtk commands Claude actually drove. The cap is
surfaced in the output (`Claude rtk calls: X/Y matched`) for self-validation.

Notes:
- rtk records its timestamp after command completion, so the match window is
  60s (recovers ~99.9% of invocations; the rtk-invocation + project filters
  keep it unambiguous).
- Unmatched commands (terminal/codex) earn no savings: they never entered an
  LLM context.
- Tool results always enter the cached prefix via cache_creation (validated
  with `claude -p`), so there is no separate input-rate savings bucket.

Tracker gains `get_raw_commands()` (timestamp, project_path, saved_tokens) for
the audit; unparseable timestamp rows are skipped rather than coerced.
@CLAassistant

CLAassistant commented Jul 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@thebigjc thebigjc marked this pull request as ready for review July 11, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants