Skip to content

Bug: OpenGraph relationship ingest cannot merge into a pre-existing node from a different source (fails outright when using match_by: "property" + kind) #3003

Description

@JVBotelho

Description:

An OpenGraph-sourced relationship whose endpoint references an existing node created by a
different ingest source (for example, a base AD Group node from a SharpHound/RustHound-CE
import) cannot attach to that node. Two approaches were tested against a live BloodHound CE
instance, and both fail: one silently, by creating a duplicate node, and the other with a hard
ingest error. Root cause: ingestibleRelationshipsToUpdates
(cmd/api/src/services/graphify/ingestrelationships.go) sets StartIdentityKind/EndIdentityKind
to the ingest batch's own sourceKind unconditionally, regardless of the endpoint's resolved
Kind or match strategy, so the write targets a node under the current ingest's identity space
instead of the node it just resolved. resolveIngestibleEndpoint
(cmd/api/src/services/graphify/endpoint/fetch.go, called via endpoint.ResolveAll from
IngestRelationships) does find the correct node. The write step afterward is what loses track of
it.

Are you intending to fix this bug?

Yes, if a fix along these lines is the right direction. ingestibleRelationshipsToUpdates is
shared by every OpenGraph ingest, not just this one, so I'd rather settle on an approach during
triage before sending a PR that presumes a specific fix.

Component(s) Affected:

  • API
  • Neo4j

Steps to Reproduce:

  1. Ingest a standard SharpHound/RustHound-CE collection so a base :Group node with a known
    objectid exists (e.g. Domain Admins, ...-512).
  2. Ingest an OpenGraph payload (custom source_kind) containing a relationship whose end
    endpoint is {"match_by": "id", "value": "<same objectid>"}.
  3. Query MATCH (x) WHERE x.objectid = '<same objectid>' RETURN x, labels(x): two distinct nodes
    come back instead of one, the original Active Directory | Group node and a second
    bare/untyped node under the OpenGraph source.
  4. Instead, ingest an OpenGraph payload whose end endpoint is {"match_by": "property", "kind": "Group", "property_matchers": [{"key": "objectid", "operator": "equals", "value": "<same objectid>"}]}.
  5. See the ingest task fail entirely with a ConstraintValidationFailed error (full text below),
    even though resolveIngestibleEndpoint correctly found the real node moments earlier in the
    same request.

Expected Behavior:

Per the OpenGraph docs, match_by: "property" (with an optional kind filter) is documented as
the mechanism for resolving relationship endpoints against nodes the payload didn't itself declare.
A successfully resolved endpoint should attach the relationship to the real, existing node. It
should not throw a constraint error immediately after correctly finding that same node.

Actual Behavior:

  • A bare match_by: "id" reference to another source's node creates a disconnected duplicate
    under the current ingest's own source-kind identity space, instead of attaching to the original.
  • Adding kind to get a genuine graph-wide match (via match_by: "property") still writes under
    the current source's identity. Doing so with a base kind's label present collides with that
    kind's uniqueness constraint and fails the whole ingest task, not just the one relationship.

Separately, analysis.FetchNodeByObjectIDIncludeOpenGraph (packages/go/analysis/analysis.go)
falls back to its openGraphNodeByIndexedKindProperty helper when resolving OpenGraph nodes, which
explicitly excludes ad.Entity/azure.Entity-kinded nodes (query.Not(query.Kind(query.Node(), ad.Entity, azure.Entity))). That segregation between built-in collector data and third-party
OpenGraph data looks intentional at the node level. The same boundary blocks a relationship from
legitimately referencing a base node without either duplicating it or crashing ingest.

Screenshots/Code Snippets/Sample Files:

Duplicate-node check after the match_by: "id" ingest:

MATCH (x) WHERE x.objectid = '<sid>' RETURN x, labels(x)
// -> two rows: one `Active Directory | Group`, one bare/untyped OpenGraph node

Error from the match_by: "property" + kind ingest:

Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(48) already exists with
label `Group` and property `objectid` = '...-512')

Environment Information:

BloodHound: 9.4.0 (installed via bloodhound-cli)

Collector: RustHound-CE 2.4.91 (-c DCOnly)

OS: N/A (server-side ingest behavior, not UI)

Browser (if UI related): N/A

Node.js (if UI related): N/A

Go (if API related): N/A (behavior confirmed by reading source, not by building from Go toolchain)

Database (if persistence related): Neo4j 4.4.48 (community, bundled)

Docker (if using Docker): 29.6.2

Additional Information:

Current workaround: reference the node with match_by: "id" (accepting the harmless duplicate),
then run a one-time Cypher script directly against Neo4j (not through BloodHound's own Cypher
search bar, which rejects updating clauses) that MERGEs a relationship between the duplicate and
the real node, matched by their shared objectid. This only creates a relationship between two
already-existing nodes, with no node creation involved, so it doesn't touch the uniqueness
constraint and restores normal shortestPath(...)-style traversal through the bridge. Full
writeup: docs/adr/0006-opengraph-cross-source-node-identity.md and
crates/ad-tombstone/bridge_shadow_nodes.cypher in this repo (github.com/JVBotelho/ghosthound).

Potential Solution (optional):

Not proposing a specific fix yet; see "Are you intending to fix this bug?" above. At a glance, the
identity used to locate the write target in ingestibleRelationshipsToUpdates would need to come
from the endpoint's resolved Kind (when one was found via match_by: "property" + kind)
rather than unconditionally from the ingest's own sourceKind. That has implications for every
other OpenGraph ingest path, which is exactly what I'd want maintainer input on before writing it.

Related Issues:

This area has seen recent, active work, but on a different axis of the same file:

That work is about whether ingest mutates the kind labels on a resolved endpoint node. This report
is about a separate step in the same function: ingestibleRelationshipsToUpdates (current main)
still hardcodes StartIdentityKind/EndIdentityKind to the ingest's own sourceKind when building
the update, regardless of the endpoint's resolved kind or match strategy. That's the identity used
to locate which node the write targets, upstream of whatever kind-merging behavior is currently in
effect. None of #2712/#2725/#2749 touch that identity restriction, so the behavior described above
is unaffected by the revert and still reproduces on current main (re-confirmed by reading
ingestrelationships.go directly, independent of the live repro).

Also worth noting: match_by: "property" itself was added in #2422 (BED-7451) explicitly to
support "hybrid environment paths," the same use case of an OpenGraph edge resolving against a
node from another source. The behavior above means that stated goal doesn't hold once the endpoint
resolves to a base AD/Azure node with a real kind label.

Contributor Checklist:

  • I have searched the issue tracker to ensure this bug hasn't been reported before or is not
    already being addressed.
  • I have provided clear steps to reproduce the issue.
  • I have included relevant environment information details.
  • I have attached necessary supporting documents.
  • I have checked that any JSON files I am attempting to upload to BloodHound are valid.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriageThis issue requires triaging

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions