Skip to content

Gate PRs with an automated review instead of a human one - #2230

Closed
defangdevs wants to merge 1 commit into
mainfrom
chore/coderabbit-review-gate
Closed

Gate PRs with an automated review instead of a human one#2230
defangdevs wants to merge 1 commit into
mainfrom
chore/coderabbit-review-gate

Conversation

@defangdevs

@defangdevs defangdevs commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Why

Every PR in this org needs 1 approving review (org ruleset "Protect Default Branches"), and in practice @lionello is the only approver. Most PRs are now agent-authored, so human review is the bottleneck on velocity.

This makes CodeRabbit the approving reviewer instead:

  • request_changes_workflow: true — CodeRabbit requests changes when it finds a problem and approves when the problem is fixed. Today it only comments, so it gates nothing.
  • pre_merge_checks.override_requested_reviewers_only: true — the PR author can not dismiss a failed check, so an agent can not wave through its own PR.
  • Built-in checks start at warning. Raise one to error when we want it to block a merge.

An app approval already satisfies the rule today: dependabot-automerge.yml merges with an approval from github-actions[bot]. Nothing in the ruleset has to change for this to work.

The ruleset itself keeps requiredReviewThreadResolution: true, so CodeRabbit comments must still be resolved before a merge.

Scope

The same file goes into every active DefangLabs repo. A CI gate (one standard ci check per repo, required through the org ruleset) is the second layer and lands separately — the require-ci-gate ruleset in defang-mvp/pulumi/infrastructure/githubx.ts.

Verify

After merge, open a throw-away PR and confirm coderabbitai[bot] posts a review with state CHANGES_REQUESTED or APPROVED (not just COMMENTED), and that the approval satisfies the branch rule.

Summary by CodeRabbit

  • Chores
    • Added shared review workflow configuration.
    • Configured standard review checks to issue warnings by default.
    • Restricted pre-merge check overrides to requested reviewers.
    • Disabled docstring checks.

@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

This pull request changes a CodeRabbit configuration file. Because it comes from a fork or its author is not a repository collaborator, reviews use only the configuration from the target branch. The proposed configuration will take effect after it is merged.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 06607183-4245-498b-b5b1-65f7fafaa96c

📥 Commits

Reviewing files that changed from the base of the PR and between bd55787 and 41b8111.

📒 Files selected for processing (1)
  • .coderabbit.yaml

Included review availability: 3 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.


📝 Walkthrough

Walkthrough

Added a shared CodeRabbit configuration. It enables request-based change workflows, limits pre-merge overrides to requested reviewers, sets selected checks to warnings, and disables docstring checks.

Changes

CodeRabbit configuration

Layer / File(s) Summary
Review policy configuration
.coderabbit.yaml
Configures request-based changes, reviewer-only pre-merge overrides, warning defaults for title, description, and issue-assessment checks, and disabled docstring checks.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 41b81

This change enables automated review gating for pull requests; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: jordanstephens

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: using CodeRabbit as an automated pull request reviewer and approver.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@defangdevs

Copy link
Copy Markdown
Contributor Author

Superseded by CodeRabbit Organization → Global Overrides, which now carries this policy org-wide. That layer outranks a repo .coderabbit.yaml, so it also closes the hole this PR would have left: CodeRabbit reads a repo file from the PR's head branch, so a per-repo file can be relaxed by the very PR it is meant to gate.

Verified on DefangLabs/defang-github-action#58 with @coderabbitai configurationoverride_requested_reviewers_only, title.mode, description.mode and issue_assessment.mode all resolve from Global overrides.

The CI half of the change is unaffected: DefangLabs/defang-github-action#58 and DefangLabs/defang-mvp#3203 stay open.

@defangdevs defangdevs closed this Aug 21, 2026
@defangdevs
defangdevs deleted the chore/coderabbit-review-gate branch August 21, 2026 21:05
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@defangdevs Thanks for the clarification. The Organization → Global Overrides policy makes this repository-level configuration redundant and removes the head-branch configuration bypass risk.

The separate CI work remains tracked in DefangLabs/defang-github-action#58 and DefangLabs/defang-mvp#3203.


✏️ Learnings added
Learnt from: defangdevs
URL: https://github.com/DefangLabs/defang/pull/2230

Timestamp: 2026-08-21T21:05:54.842Z
Learning: For DefangLabs repositories, CodeRabbit review policy is configured through Organization → Global Overrides. This layer outranks a repository `.coderabbit.yaml` file, including a file supplied by a pull request head branch. The global configuration sets `override_requested_reviewers_only`, `title.mode`, `description.mode`, and `issue_assessment.mode`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant