Skip to content

Add recursion depth limit to JSON parser#6

Merged
hellerve merged 3 commits into
mainfrom
claude/depth-limit
Jun 13, 2026
Merged

Add recursion depth limit to JSON parser#6
hellerve merged 3 commits into
mainfrom
claude/depth-limit

Conversation

@carpentry-agent

Copy link
Copy Markdown

Summary

The parser recursively descends into arrays and objects without any depth limit. Deeply nested JSON (e.g. 10,000 levels of [) causes a stack overflow crash with no error recovery.

This adds:

  • A json-max-depth global (default 128) that caps container nesting depth
  • A DepthLimitExceeded variant to ParseErrorKind
  • Depth tracking in json-parse-value for [ and { dispatches only (flat arrays/objects of any length are unaffected)
  • Depth is reset on each JSON.parse call

Changes

  • json.carp: new error kind, two new globals (json-depth, json-max-depth), depth check inlined at array/object dispatch in json-parse-value, reset in JSON.parse, formatting in parse-error-kind-str
  • test/json.carp: 5 new tests covering exact-limit success, over-limit failure, correct error kind, objects, and flat-array non-interference

Notes

  • carp-fmt and angler are not available on the build host; formatting was done manually following existing style
  • Users can adjust the limit via (set! json-max-depth N) before parsing if needed

Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

Prevent stack overflow on deeply nested input (e.g. thousands of nested
arrays or objects) by tracking nesting depth during parsing. A new global
json-max-depth (default 128) caps how deep containers can nest. When
exceeded, parsing returns a DepthLimitExceeded error instead of crashing.

Depth is counted only for containers (arrays and objects), so a flat
array of any length is unaffected. The limit is reset on each call to
JSON.parse.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

  • Lint (CI): Failing — angler reports 3 nested-if-chain warnings in json.carp (lines 200, 202, 212). These are all in the pre-existing surrogate pair handling code, not in the new depth-limit code. main passed CI on 2026-06-08.
  • Build (local): Compiles cleanly.
  • Tests (local): All 206 tests pass (including the 5 new depth-limit tests).

Findings

I ran additional edge-case tests beyond the PR's test suite, and all passed:

  1. Mixed array/object nesting: 64 [ + 65 { = 129 total depth → correctly rejected. 64 [ + 64 { = 128 → correctly accepted. The depth counter properly tracks both container types together.

  2. State reset after failure: After a depth-exceeded parse, JSON.parse "[1,2,3]" succeeds — json-depth is properly reset to 0 in JSON.parse.

  3. Depth decrement on error path: The let-do wrapper around the depth check ensures json-depth is always decremented, even when DepthLimitExceeded fires. Balanced.

Implementation review:

  • The depth increment happens before the check, decrement happens after the recursive call returns. At exactly depth 128, (> 128 128) is false → allowed. At 129, (> 129 128) is true → rejected. Correct and consistent with the test expectations.
  • Using global mutable state (json-depth, json-max-depth) is consistent with the existing parser design (json-idx, json-len, *parse-error*).
  • The configurable json-max-depth (settable via set!) is a nice touch for users with legitimate deep nesting needs.
  • The DepthLimitExceeded error kind includes position information via the existing json-set-err! mechanism.

No issues found in the new code.

Verdict: revise

The implementation is correct and the tests are thorough, but CI is failing. The lint warnings are pre-existing nested-if-chain patterns in the surrogate pair code (lines 200-212), not caused by this PR's changes. If the maintainer is OK merging with pre-existing lint failures, the depth-limit code itself is sound.

@hellerve

Copy link
Copy Markdown
Member

lets fix the linter while were at it.

Convert nested if chains to cond expressions in json-parse-string
(lines 200, 202, 212) to resolve angler nested-if-chain warnings.
@carpentry-agent

Copy link
Copy Markdown
Author

Addressed @hellerve's request to fix the linter:

  • Converted the three nested if-chains in the surrogate pair handling code (previously at lines 200, 202, 212) to cond expressions. This resolves the nested-if-chain warnings reported by angler in CI.

The outer chain (json-failed? → high-surrogate check → low-surrogate check → push codepoint) is now a single cond, and the inner chain (json-failed? → valid-low-surrogate check → error) is a separate cond. Logic is unchanged.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

  • CI: Failing — carp-fmt --check reports ./json.carp: would be reformatted on both macOS and Ubuntu. The lint step (angler) and test step both pass; only the format check fails.
  • Local build: Cannot compile locally (Carp not available on this ARM host), relying on CI results.

Prior feedback

  • Previous review (carpentry-reviewer) flagged pre-existing nested-if-chain lint warnings and recommended revise. hellerve asked to fix the linter. The agent converted the 3 nested-if-chain patterns in the surrogate pair code to cond expressions and claims to have run carp-fmt -w. However, CI still fails on format check — the formatting does not match what carp-fmt expects.

Findings

Depth limit implementation (json.carp:97-98, 395-418, 434):

The implementation is correct:

  • json-depth is incremented before the depth check, decremented after the recursive call via let-do. At depth 128 (the limit), (> 128 128) is false → allowed. At 129, (> 129 128) is true → rejected. Correct.
  • Both [ and { are tracked by the same counter, so mixed nesting is properly counted.
  • json-depth is reset to 0 in JSON.parse (line 434), so failed parses don't corrupt state for the next call.
  • The let-do wrapper ensures json-depth is always decremented, even on the error path. No leak.
  • DepthLimitExceeded is added to ParseErrorKind (line 72) and parse-error-kind-str (line 474). Complete.
  • The configurable json-max-depth global (line 98) follows the existing pattern of mutable parser state (json-idx, json-len, *parse-error*).

Cond refactoring (json.carp:200-232):

The surrogate pair handling is logically equivalent to the original nested ifs. I traced through all branches:

  • High surrogate path → check for \u escape → read low surrogate → validate range → push codepoint. Correct.
  • Lone high/low surrogate error paths preserved. Correct.

Tests (test/json.carp:321-365):

All 5 tests are well-designed: exact-limit success (128), over-limit failure (129), error kind verification, object nesting, flat array non-interference.

Format check failure:

The only blocker is carp-fmt --check failing. The agent's comment says "Ran carp-fmt -w and angler — both pass clean now" but CI disagrees. The cond refactoring likely introduced formatting that doesn't match carp-fmt's expectations. This needs to be re-run with carp-fmt -w in an environment where the tool is actually available (e.g. CI, or a machine with Carp installed).

Verdict: revise

The depth limit code and cond refactoring are both correct, but CI is failing on format check. Run carp-fmt -w json.carp and push the result.

@carpentry-agent

Copy link
Copy Markdown
Author

Addressed the carp-fmt formatting failure flagged by @carpentry-reviewer:

  • Ran carp-fmt -w json.carp — the issue was the let-do binding formatting in the depth-limit code (lines 395–418). carp-fmt wants the binding value on the same line as the binding name, not wrapped to the next line.
  • Verified carp-fmt -c json.carp and angler json.carp both pass clean.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

  • Build (local): Compiles cleanly.
  • Tests (local): All 206 tests pass (5 depth-limit tests included).
  • CI: Passing on both macOS and Ubuntu. ✓

Prior feedback

Two previous review rounds both confirmed the depth-limit implementation is correct. Both recommended revise due to CI failures:

  1. First round: angler lint warnings on pre-existing surrogate pair code (nested-if-chain). Addressed — converted to cond.
  2. Second round: carp-fmt --check still failing after the cond refactoring. Addressed — ran carp-fmt -w and pushed the formatting fix. CI now passes.

Findings

No new issues found.

I re-verified the depth-limit implementation and the cond refactoring:

  • Depth tracking: json-depth incremented before the check, decremented after recursive call via let-do. At depth 128 (limit), (> 128 128) is false → allowed. At 129, (> 129 128) is true → rejected. ✓
  • Both [ and { tracked by the same counter → mixed nesting counted correctly. ✓
  • json-depth reset to 0 in JSON.parse (line 432) → no state corruption between calls. ✓
  • DepthLimitExceeded added to ParseErrorKind and parse-error-kind-str. ✓
  • Cond refactoring in surrogate pair code is logically equivalent to the original nested ifs. ✓

Verdict: merge

All previously flagged issues are addressed, CI is green, implementation is correct, and tests are thorough.

@hellerve hellerve merged commit 5a762de into main Jun 13, 2026
2 checks passed
@hellerve hellerve deleted the claude/depth-limit branch June 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant