Skip to content

rules_go: add registry-backed multi-version fork support#174

Open
tonyredondo wants to merge 3 commits into
mainfrom
feat/rules-go-multi-version-patch-architecture
Open

rules_go: add registry-backed multi-version fork support#174
tonyredondo wants to merge 3 commits into
mainfrom
feat/rules-go-multi-version-patch-architecture

Conversation

@tonyredondo

@tonyredondo tonyredondo commented Jun 24, 2026

Copy link
Copy Markdown
Member

Summary

  • Add a registry-backed rules_go fork architecture with v0_60_0 as the default upstream and v0_61_1 as the first additional supported upstream. Materialized base trees live under third_party/rgo/<upstream>/base.
  • Move the old default base tree into third_party/rgo/v0_60_0/base, add third_party/rgo/v0_61_1/base, remove the public complete variant, and make rules_go_variant = "complete" fail with a migration message.
  • Add maintainer patch series for each registered upstream under third_party/rules_go_orchestrion/patches/<upstream>/, plus generated fork-selection maps, materialization/drift checks, release archive checks, CI updates, Go onboarding pins, bootstrap tooling, docs, and integration harness updates.
  • Add public consumer patch-profile support through the workspace_runtime profile, including generator/verifier tooling for consumers that already own a rules_go patch stack.
  • Add uploader execution-log cache filtering so CI can avoid uploading Bazel-cached test outputs. The uploader now supports execution-log modes (auto, required, optional, disabled), default .topt/bazel-execution-log.json discovery, and the explicit --allow-cached-payload-uploads opt-out.
  • Fix validation-discovered edge cases around default variant normalization, legacy complete strip-prefix migration errors, missing registry/profile contract fields, Darwin pure/internal link handling, direct-vs-transitive CGo toolchain propagation, and the multi-service aggregate export.bzl shim.

Validation

  • python3 tools/dev/generate_rules_go_fork_maps.py --check
  • RULES_GO_ORCHESTRION_CACHE=/private/tmp/rules_go_orchestrion_cache python3 tools/dev/materialize_rules_go_fork.py check --all
  • python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt
  • python3 tools/dev/check_release_archive_contents.py --source worktree
  • git diff --check origin/main...HEAD -- . ':(exclude)third_party/rules_go_orchestrion/patches/**' ':(exclude)third_party/rgo/**'
  • ./bazelw test //... --noexperimental_split_xml_generation
  • cd modules/go && ../../bazelw test //tools/onboardingpins:onboardingpins_test //tools/dd_topt_go_bootstrap:bootstrap_test --override_module=datadog-rules-test-optimization=../..

The raw patch files and materialized upstream rules_go trees intentionally preserve upstream/patch whitespace, so the whitespace check is scoped to repository-owned code, docs, and generated registry/profile metadata outside those raw data trees.

Follow-up validation

  • Cross-repo Go fixture validation is covered by DataDog/rules_test_optimization_tests#86. That PR now pins 7aa157b2ad0b9f3ed7d4e84f02d4ac03a58948e8, the current head of this PR, across the root module and fixtures.
  • DataDog/dd-source#480054 validates the workspace_runtime profile applied after a repository-owned rules_go v0.60.0 patch stack. It now pins 7aa157b2ad0b9f3ed7d4e84f02d4ac03a58948e8 and includes enabled pilot tests, doctor validation, execution-log-required dry-run validation, cached-output filtering, and a real uploader run from its functional validation pass.

Deferred / out of scope

  • Top-level ../rules_test_optimization_tests ./runtests is deferred because the current failure is in the Python fixture path: a ddtrace-4.8.0rc4 wheel URL returned 404. This PR is focused on Go / rules_go support; the Go fixture matrix is covered by DataDog/rules_test_optimization_tests#86.

@datadog-datadog-prod-us1-2

This comment has been minimized.

@tonyredondo tonyredondo force-pushed the feat/rules-go-multi-version-patch-architecture branch 2 times, most recently from 3041aaf to 20682d6 Compare June 26, 2026 08:55
@tonyredondo tonyredondo force-pushed the feat/rules-go-multi-version-patch-architecture branch from 20682d6 to f3dc118 Compare June 26, 2026 09:19
@tonyredondo tonyredondo marked this pull request as ready for review June 26, 2026 13:12
@tonyredondo tonyredondo requested a review from a team as a code owner June 26, 2026 13:12

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: FAIL

Users who previously set --rules-go-variant=complete had strip_prefix = "third_party/rules_go_orchestrion_complete" written to their MODULE.bazel. Re-running bootstrap without flags hits hydrateManagedRulesGoVariantRulesGoSelectionForStripPrefix, which returns a generic "not in registry" error with no migration guidance. The fix adds the missing legacy-complete guard in RulesGoSelectionForStripPrefix so users get the same actionable "complete" is no longer supported. Use "base". message as the explicit-flag path.

View proposed fix
📊 Validated against 6 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 04770ab · What is Autotest? · Any feedback? Reach out in #autotest

Comment thread modules/go/tools/onboardingpins/pins.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant