Skip to content

Guard against cyclic trees in process_tag/process_element (#256) - #274

Open
chiliec wants to merge 1 commit into
matthewwithanm:developfrom
chiliec:fix/issue-256-cyclic-tree-recursion
Open

Guard against cyclic trees in process_tag/process_element (#256)#274
chiliec wants to merge 1 commit into
matthewwithanm:developfrom
chiliec:fix/issue-256-cyclic-tree-recursion

Conversation

@chiliec

@chiliec chiliec commented Aug 27, 2026

Copy link
Copy Markdown

Fixes #256.

Problem

The mutual recursion introduced in 1.2.2 between process_tag and process_element has no cycle guard. A well-formed BeautifulSoup tree is acyclic, but some HTML producers — notably PDF-to-HTML pipelines (the reporter hit this via marker-pdf) — can yield a graph where a descendant node references an ancestor. Converting such input recurses without bound and crashes:

RecursionError: maximum recursion depth exceeded

Because it aborts with an unhandled exception, a single malformed document takes down the whole conversion.

Fix

Thread an optional _visited set of tag id()s through the descent. When a tag already on the current path is re-encountered, return `` to break the cycle instead of recursing into it.

  • _visited defaults to None, so existing callers and any custom converters that call process_tag / process_element directly are unaffected.
  • A fresh copy is passed per level (_visited | {node_id}), so it only detects a node repeating on the current descent path — a node legitimately reached via two separate sibling branches is still processed each time.

Test

Adds test_cyclic_tree_does_not_recurse, which builds a cyclic tree (a <p> whose contents include its own ancestor <div>), lowers the recursion limit, and asserts conversion completes without RecursionError.

Verification (local, Python 3.11)

  • RED→GREEN: with the fix reverted the new test fails with RecursionError; with the fix it passes.
  • pytest tests/ — full suite passes (84 tests, up from 83), no regressions.
  • flake8 --ignore=E501,W503 markdownify tests (as tox runs it) is clean.

…hanm#256)

The mutual recursion between process_tag and process_element has no cycle
guard. A well-formed BeautifulSoup tree is acyclic, but some HTML producers
(e.g. PDF-to-HTML pipelines feeding marker-pdf) can yield a graph where a
descendant references an ancestor. Converting such input recurses without
bound and crashes with RecursionError.

Track the ids of the tags on the current descent path (an optional _visited
argument, defaulted to None so existing callers and custom converters are
unaffected) and return '' when a tag is re-encountered, breaking the cycle.
A fresh copy is threaded per level, so a node legitimately reached via two
sibling paths is still processed each time.

Adds a regression test.

Fixes matthewwithanm#256.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recursion Error: Process_element / process_tag mutual recursion (infinite loop)

1 participant