Skip to content

fix(extract): keep external imports on unchanged scans - #3342

Open
VasuBansal7576 wants to merge 2 commits into
Graphify-Labs:v8from
VasuBansal7576:fix/incremental-external-imports
Open

fix(extract): keep external imports on unchanged scans#3342
VasuBansal7576 wants to merge 2 commits into
Graphify-Labs:v8from
VasuBansal7576:fix/incremental-external-imports

Conversation

@VasuBansal7576

Copy link
Copy Markdown
Contributor

Problem

Running graphify extract twice on an unchanged TypeScript file can delete an external dependency and its import edge.
The rescued import stub stores the package name as source_file, so the incremental stale-file check mistakes it for a deleted local file.
With --no-cluster, the command even reports "outputs left untouched" after modifying graph.json.

For a two-file fixture, the raw graph shrinks from 5 nodes / 4 edges to 4 nodes / 3 edges, with zero changed or deleted source files.

Solution

Emit the existing unowned-source representation for external package stubs, while leaving relative imports and tsconfig path resolution unchanged.
Add raw-graph cleanup when an update removes a stub's last reference, preserving shared dependencies, standalone nodes, and both supported hyperedge layouts.

Existing graphs need one rebuild after upgrading: run graphify extract <corpus> --code-only --force --out <same-output-root> with your usual clustering options.
This replaces pre-fix metadata; an unchanged incremental run alone cannot repair an existing graph.

Before and after

Actual Graphify viewer screenshots rendered from the second run's saved graph using build_from_json and to_html.
Same fixture and viewport; the viewer generates the layout.
Normal graph assembly drops an unresolved ref_ edge, so viewer totals differ from raw JSON totals.

Before: dependency missing After: dependency preserved
Before After

Fixture, CLI receipts, and saved graphs.
No API calls or private corpus data are involved.

Validation

  • 196 focused tests pass; Ruff and git diff --check pass.
  • Real CLI regression covers bare/scoped packages, raw/clustered graphs, unchanged scans, unrelated edits, shared importers, deletion, and exclusion.
  • Pruning regression preserves nested graph.hyperedges members.
  • Required graphify update . completed; optional SQL, DM, and Robot fixture parsers were unavailable.
  • Independent review checked standards and lifecycle behavior; the nested-hyperedge finding was fixed and the rebuild requirement verified.

Related reports checked

No matching issue or PR found in open/closed searches before submission.
Unlike #3084 / #3092, this requires no package manifest or ID collision.
Unlike #1711 / #3328, no source is re-extracted before the loss.
#2807 / #2809 cover orphan cleanup after a real deletion; this failure deletes a still-referenced dependency during an unchanged scan.
Also inspected #880, #2878, #3262, and the resolved path-normalization report #2210.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Fixes external import stubs being wrongly pruned during incremental re-extraction: merge_raw_extraction and _prune_graph_json_sources now run a new _sweep_raw_orphans that drops only unowned nodes whose last edge reference this update actually removed, while preserving already-isolated nodes and any node still referenced through a hyperedge (including nested graph.hyperedges). Rescued package specifiers via _resolve_rescued_specifier now carry empty source metadata instead of a fake local path, so a package name is no longer mistaken for a deleted file. Adds coverage for the sweep semantics and for end-to-end survival of external imports across no-op, edit, add, delete, and exclusion-only extractions.

No blocking issues surfaced. 8 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2686 functions depend on the 368 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract() — 520 callers, 43 callees
  • new: _rebuild_code() — 113 callers, 50 callees
  • new: build_from_json() — 192 callers, 18 callees
  • new: build_merge() — 62 callers, 13 callees
  • new: to_obsidian() — 36 callers, 13 callees
  • new: extract_files_direct() — 17 callers, 20 callees
  • new: extract_js() — 85 callers, 4 callees
  • new: extract_xaml() — 19 callers, 17 callees
  • …and 65 more — each is listed as a finding

Verification — 2686 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2479 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify merge\_raw\_extraction.

The verifier did not have enough to check merge\_raw\_extraction, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `graph_path` is annotated `str | Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_prune\_graph\_json\_sources.

The verifier did not have enough to check \_prune\_graph\_json\_sources, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `graph_path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_resolve\_rescued\_specifier.

The verifier did not have enough to check \_resolve\_rescued\_specifier, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 1 grounded finding(s) anchored inline below; 72 more finding(s) on lines outside this diff (see the check run).

Comment thread graphify/build.py
]


def merge_raw_extraction(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressionmerge_raw_extraction()

fans out to 10 callees (efferent coupling); 8 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added callee is _sweep_raw_orphans, in the same module. Both raw-merge and exclusion-only pruning need the same last-reference rule, so the shared helper keeps their behavior consistent. The regression tests cover shared importers, removal of the last importer, standalone nodes, and both hyperedge layouts. I checked the diff and am keeping the shared helper; this coupling-count increase does not identify a correctness regression.

@VasuBansal7576

Copy link
Copy Markdown
Contributor Author

The Graphify review check passes on 7818da9, and I have replied to its inline coupling advisory. The CI workflow has not executed any jobs: run https://github.com/Graphify-Labs/graphify/actions/runs/33923399562 ended with action_required. Could a maintainer approve the fork workflow if approval is required? The 196 focused local tests and lint pass, but I am not treating those as a replacement for the repository CI matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant