feat(opa-proto): support proto-format plan bundles (opa build --format=proto) - #110
Draft
sspaink wants to merge 11 commits into
Draft
feat(opa-proto): support proto-format plan bundles (opa build --format=proto)#110sspaink wants to merge 11 commits into
sspaink wants to merge 11 commits into
Conversation
…t=proto) Add a new opa-proto module that decodes protobuf-format plan bundles (plan.pb / .manifest.pb) into the existing IR model, so bundles built with `opa build --format=proto` evaluate identically to their JSON counterparts. Proto bundles keep data.json as JSON, so opa-proto is an optional add-on alongside a JSON provider rather than a replacement. opa-proto module: - Vendored plan.proto / manifest.proto (Java bindings generated via protobuf-gradle-plugin + protobuf-java 4.x). - PlanMapper (all 34 IR statement types), ManifestMapper, and StructConverter. - ProtoBundleReader implementing the new optional ProtoBundleDecoder SPI. opa-evaluator: - ProtoBundleDecoder SPI (optional, zero-or-one). - BundleFormat: filename constants + mixed-format rejection matching OPA. - InputStreamSource + BundleAssembler.loadPlanAndManifest: shared detect/validate/route logic so loaders only do file discovery. - Services.loadAtMostOne: one shared ServiceLoader helper (dedups three copies). - PROTO_DECODER resolved lazily so proto misconfiguration cannot break JSON-only bundle loading. Loaders/CLI: - FileSystemBundleLoader and TarballBundleLoader detect .pb and route through the shared path. - CLI depends on opa-proto at runtime so it can read proto bundles. Version pinning: - Pin the OPA version in tools/generate-compliance-tests/go.mod (single source of truth) and re-vendor .proto via the native vendorProtoSchemas Gradle task; CI verify-proto-vendor guards against drift. Tests: proto-vs-JSON evaluation parity, mixed-format rejection, manifest omitempty fidelity, numeric-type parity, and malformed-plan handling. Closes open-policy-agent#101 Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
…-bundles Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com> # Conflicts: # opa-evaluator/src/main/java/io/github/open_policy_agent/opa/bundle/BundleAssembler.java # opa-evaluator/src/main/java/io/github/open_policy_agent/opa/mapper/AnnotationIntrospectors.java
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…ndor Address zizmor findings on the verify-proto-vendor job: pin all four actions to commit SHAs (blanket-pinning policy) and set persist-credentials: false on checkout, matching the rest of the workflow. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
…-bundles # Conflicts: # .github/workflows/pull-request.yml # opa-services/src/main/java/io/github/open_policy_agent/opa/bundle/TarballBundleLoader.java # tools/generate-compliance-tests/go.mod # tools/generate-compliance-tests/go.sum
johanfylling
approved these changes
Jul 24, 2026
- Drop the misleading '.manifest.pb before .manifest' comment; branches use exact equals() matching, so order is irrelevant. - Replace the redundant 'else if (!entry.isDirectory())' with a plain else; directories already continue at the top of the loop. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
sspaink
force-pushed
the
worktree-proto-plan-bundles
branch
from
July 24, 2026 16:27
28999b7 to
34eb669
Compare
Replace the lazy ProtoDecoderHolder with a static final PROTO_DECODER field, matching POLICY_READER and BUNDLE_PARSER. loadOptional still returns null for JSON-only classpaths, so the only behavioral change is that a duplicate-provider misconfig now fails at class-init rather than on first proto load — a classpath error we want to surface early. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Per review: fold the three StructConverter number-conversion cases into a @ParameterizedTest, and the two ProtoBundleParity per-entrypoint methods into one parameterized test over (entrypoint, input). Structurally distinct cases are left as their own @test methods. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The other dependencies carry no rationale; opa-proto's purpose is clear from the module name and documented in ProtoBundleDecoder's javadoc. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
ProtoBundleParityTest only exercised the proto decoder on one authz bundle. This extends coverage to the full OPA compliance corpus so the decoder is validated against the same breadth of IR the JSON reader is. - generate-compliance-tests: emit a base64 `plan_proto` per case next to the JSON `plan`, produced by OPA's own ir.PolicyToProto (the same path as `opa build --format=proto`), so the proto fixtures are faithful to OPA rather than a Java-side re-encoding. - ProtoComplianceTest: for every case, decode both the JSON and proto plans and assert they evaluate identically. Evaluation parity (not a structural Policy comparison) is the contract that matters — the JSON reader and proto decoder legitimately differ on how absent repeated fields are represented (null vs empty list), which does not affect evaluation. - Regenerate the fixtures. They were already stale against the pinned OPA version (the compliance-fixtures CI check was failing); regenerating syncs them and adds the plan_proto column. - exceptions.yaml: exclude two new logic_op short-circuit cases whose builtin-error surfacing the Java evaluator does not yet match. - opa-proto: add jackson-databind as a test dependency for reading the fixtures. Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The opa-proto module was never wired into CI, so its tests (including the new ProtoComplianceTest) never ran on PRs. Add a test-opa-proto job gated by the change filter, extend the pr-check Rego (and its tests) with an opa-proto rule, and add the job to the summary's required set. opa-proto's filter also triggers on tools/generate-compliance-tests/ changes, since ProtoComplianceTest decodes the plan_proto fixtures that generator emits (an OPA version bump there changes them). Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
CodeQL flagged the leading String parameter of two parameterized tests as
unused (it fed only the {0} display name). Use it in the AssertJ
description instead, which resolves the finding and improves the failure
message.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for reading protobuf-format plan bundles produced by
opa build --format=proto(plan.pb/.manifest.pb), decoding them into the existing IR model so they evaluate identically to their JSON counterparts.Proto bundles keep
data.jsonas JSON, so proto support is an optional add-on alongside a JSON provider (e.g. opa-jackson), not a replacement.Closes #101
opa build --format=protoisn't in a tagged OPA release yet, sogo.modis pinned to a temporary commit (f6092b9ce, ~v1.19.0-dev; PRs open-policy-agent/opa#8825 and #8775).