feat: evidence source provenance (EvidenceMapping) support by marcusburghardt · Pull Request #840 · complytime/complyctl

marcusburghardt · 2026-09-04T15:10:47Z

Summary

Plumb the Gemara #EvidenceMapping (source field) through complyctl's
entire evidence pipeline, enabling providers to report where evidence
was collected from.

Fixes #841

Context

The Gemara CUE schema (evaluationlog.cue line 73, auditlog.cue
lines 78-97) defines #Evidence with a source field of type
#EvidenceMapping that records evidence provenance -- which artifact,
which location within it, and a content digest. The go-gemara SDK already
exposes this as Evidence.Source. complyctl's proto API currently defers
this field with a comment referencing an unwritten "ADR 0023".

Changes

Proto API: Add EvidenceMapping message (reference_id,
coordinate, entry_id, digest, remarks) and source field
(field 6) on Evidence. Remove stale ADR 0023 comment.
Provider SDK: Add EvidenceSource struct and
Source *EvidenceSource field on Evidence in pkg/provider/.
Proto mapping: internalEvidenceSourceToProto() in server.go
and protoEvidenceMappingToInternal() in client.go.
Evaluator: Map provider.EvidenceSource to
gemara.EvidenceMapping when constructing evaluation log entries.
Markdown: Render source provenance as source: <ref> @ <coord>
in formatEvidenceMeta().
Test provider: Emit evidence with Source for E2E coverage.
Tests: Comprehensive coverage for both proto mapping directions,
evaluator serialization (YAML + JSON), and markdown rendering.

Backward-compatible: providers that don't send source are unaffected.

OpenSpec Artifacts

openspec/changes/evidence-source/proposal.md
openspec/changes/evidence-source/specs/evidence-source/spec.md
openspec/changes/evidence-source/design.md
openspec/changes/evidence-source/tasks.md

Add OpenSpec change artifacts for plumbing the Gemara EvidenceMapping (source field) through complyctl's evidence pipeline. This enables providers to report evidence provenance -- which artifact, location, and content digest each evidence entry was collected from. Artifacts: - proposal.md: motivation and scope - specs/evidence-source/spec.md: behavioral requirements - design.md: technical decisions and approach - tasks.md: implementation checklist Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

Refine Non-Goals to acknowledge go-gemara#127 tracks evidence-to-OSCAL and evidence-to-SARIF conversion upstream. SARIF already delegates to gemaraconv.ToSARIF() and will benefit automatically. Local OSCAL formatter migration tracked in a separate complyctl issue. Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

Apply auto-fixes for findings from 6 Divisor spec reviewers: - Normalize SHALL to MUST per AGENTS.md spec convention - Add scenario for partial source fields (common real-world case) - Add scenario for coordinate/entry_id passthrough (Gemara mutual exclusivity documented but not enforced by complyctl) - Qualify YAML omission scenario with JSON serialization edge case (encoding/json does not omit zero-value structs) - Add D5 design decision for passthrough model + digest format note - Expand D3 with gemara value-type serialization note - Task 1.1: add buf breaking check for backward-compat verification - Task 2.2: acknowledge missing internalEvidenceToProto test coverage - Task 3.2: add JSON serialization and YAML omission assertions - Task 5.2: add schema validation and specific E2E assertions - Add documentation tasks (CHANGELOG.md, AGENTS.md Recent Changes) Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

Add EvidenceMapping message to proto API and plumb the source field through the entire evidence pipeline, enabling providers to report where evidence was collected from. Changes: - Proto: add EvidenceMapping message with reference_id, coordinate, entry_id, digest, remarks fields; add source field (field 6) to Evidence message; remove stale ADR 0023 comment - SDK: add EvidenceSource struct and Source *EvidenceSource field to provider.Evidence in pkg/provider/client.go - Mapping: add internalEvidenceSourceToProto() in server.go and protoEvidenceMappingToInternal() in client.go for gRPC boundary - Evaluator: map provider.EvidenceSource to gemara.EvidenceMapping when constructing evaluation log entries - Markdown: render source provenance as 'source: <ref> @ <coord>' in formatEvidenceMeta() - Test provider: emit evidence with Source for E2E coverage - Tests: comprehensive coverage for both proto mapping directions, evaluator serialization (YAML + JSON), and markdown rendering Backward-compatible: providers without source are unaffected. Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

gxmiranda

Clean, well-specified feature implementation that plumbs the Gemara EvidenceMapping through complyctl's entire evidence pipeline -- from proto definition through gRPC mapping, evaluator, and Markdown rendering. Comprehensive test coverage (both mapping directions, nil/partial/full cases, YAML/JSON serialization), backward-compatible design, and proper spec-first workflow with OpenSpec artifacts. All CI checks pass.

This review was generated by /review-pr (AI-assisted).

jpower432 · 2026-09-11T18:40:26Z

+// evidence was collected. Mirrors go-gemara EvidenceMapping.
+message EvidenceMapping {
+  // Ties this evidence to a mapping-reference in the artifact's metadata
+  string reference_id = 1;


Do plugins have awareness of a mapping reference? Would we prefer that they send us a list of their sources too?

Great question. I explored this in depth, including running through some possible scenarios with AI assistance against existing policy content in the repo.

Current state: This PR implements the transport layer — reference_id flows through from provider → proto → evaluator → output. If the provider sets it, it appears in the evaluation log and Markdown report. If not, it's empty.

The gap you identified: Providers may not know about mapping-references from the policy metadata. The host (complyctl) has this information (via the DependencyGraph), but currently doesn't pass it to the evaluator.

Host-side enrichment would require:

Preserve MappingReferences from parsePolicyLayer() in the resolver (currently discarded)

Build a reqToMappingRef map (requirement → catalog mapping-reference id)

Pass it to the Evaluator and enrich evidence that has empty reference_id

Populate EvaluationLog.Metadata.MappingReferences in the output (currently empty) — without this, reference_id values would be dangling references

Trade-offs:

The matching heuristic (requirement → catalog → mapping-reference) works for the common case but reference_id is generic — evidence could relate to guidance or threats, not just catalogs

Both enrichment AND metadata output must ship together for the reference to be meaningful

It touches the resolver (core pipeline), which needs its own design consideration

Options:

Keep this PR as transport — providers can populate reference_id if they have the info; follow up with host-side enrichment in a dedicated PR with proper spec

Expand this PR — add host-side enrichment now (scope expansion: resolver + evaluator + metadata output changes)

I'm leaning toward option 1 (transport now, enrichment follow-up) to keep this PR focused, but I'd value your input on whether you think the enrichment should block this PR or can be a fast follow-up.

jpower432

Implementation looks good, but I see an issue with regard to reference-id. There doesn't seem to be a way to collect this actual reference from plugins and I don't think the host would have that information either.

@jpower432

Remove documentation claiming coordinate and entry_id are mutually exclusive in EvidenceMapping. The Gemara maintainer (@jpower432) confirmed this was an upstream documentation bug, now fixed in the next release. Addresses PR complytime#840 review feedback from @jpower432. Assisted-by: OpenCode (claude-opus-4-6) Signed-off-by: Marcus Burghardt <maburgha@redhat.com>

This was referenced Sep 4, 2026

feat: support Evidence source provenance (EvidenceMapping) in provider API #841

Open

feat: populate Evidence.Source (EvidenceMapping) in provider scan responses complytime/complytime-providers#174

Open

jpower432 reviewed Sep 5, 2026

View reviewed changes

Comment thread openspec/changes/evidence-source/design.md Outdated

marcusburghardt mentioned this pull request Sep 10, 2026

refactor: migrate OSCAL formatter to gemaraconv for evidence and relevant_evidence support #851

Open

marcusburghardt added 4 commits September 11, 2026 10:30

marcusburghardt force-pushed the opsx/evidence-source branch from ee3915f to ddb1674 Compare September 11, 2026 08:37

marcusburghardt marked this pull request as ready for review September 11, 2026 08:48

marcusburghardt requested a review from a team as a code owner September 11, 2026 08:48

marcusburghardt requested review from gxmiranda and hbraswelrh September 11, 2026 08:48

marcusburghardt added this to the ComplyTime improvements milestone Sep 11, 2026

marcusburghardt added the enhancement New feature or request label Sep 11, 2026

marcusburghardt requested a review from jpower432 September 11, 2026 09:50

gxmiranda previously approved these changes Sep 11, 2026

View reviewed changes

jpower432 reviewed Sep 11, 2026

View reviewed changes

Comment thread api/plugin/plugin.proto Outdated

jpower432 reviewed Sep 11, 2026

View reviewed changes

jpower432 requested changes Sep 11, 2026

View reviewed changes

marcusburghardt dismissed gxmiranda’s stale review via cd21489 September 14, 2026 11:53

Navigation Menu

Uh oh!

feat: evidence source provenance (EvidenceMapping) support - #840

feat: evidence source provenance (EvidenceMapping) support#840
marcusburghardt wants to merge 5 commits into
complytime:mainfrom
marcusburghardt:opsx/evidence-source

marcusburghardt commented Sep 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

gxmiranda left a comment

Uh oh!

Uh oh!

jpower432 Sep 11, 2026

Uh oh!

marcusburghardt Sep 14, 2026

Uh oh!

jpower432 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

marcusburghardt commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Changes

OpenSpec Artifacts

Related

Uh oh!

Uh oh!

gxmiranda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jpower432 Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

marcusburghardt Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

jpower432 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

marcusburghardt commented Sep 4, 2026 •

edited

Loading