Skip to content

feat(iorails): Support blocking rails - #2264

Draft
tgasser-nv wants to merge 5 commits into
refactor/iorails-compiledrail-migrationfrom
feat/iorails-blocking-actions
Draft

feat(iorails): Support blocking rails#2264
tgasser-nv wants to merge 5 commits into
refactor/iorails-compiledrail-migrationfrom
feat/iorails-blocking-actions

Conversation

@tgasser-nv

@tgasser-nv tgasser-nv commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Description

Related Issue(s)

Verification

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

@github-actions github-actions Bot added size: XL status: needs triage New issues that have not yet been reviewed or categorized. labels Aug 7, 2026
@tgasser-nv
tgasser-nv changed the base branch from develop to refactor/iorails-compiledrail-migration August 7, 2026 04:48
@tgasser-nv
tgasser-nv force-pushed the feat/iorails-blocking-actions branch from 126f981 to e31419e Compare August 7, 2026 05:02
@tgasser-nv tgasser-nv added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Aug 7, 2026
@tgasser-nv tgasser-nv self-assigned this Aug 7, 2026
@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.70115% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
nemoguardrails/guardrails/compiled_rail.py 97.40% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment on lines +558 to +563
# The rails this engine runs. Compilation decides whether a flow is *servable*; this
# decides whether it is in scope. Listed rather than derived from the catalog on purpose:
# enabling a rail is a decision, and a surface added to the catalog later should not turn
# itself on. Every block-only input/output surface is here except the seven whose actions
# read retrieval evidence IORails has no source for; transform surfaces wait on rewrite
# support.

@Pouyanpi Pouyanpi Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m concerned about intentionally coupling IORails to every individual surface name here. The manifest already describes the surface, its direction, bindings, context inputs, and transform target. Repeating those names in IORails creates a second source of truth, and every rail addition or rename now requires an engine change as well. That partly defeats the purpose of the manifest work, which was to let engines consume the library’s declared execution contract instead of maintaining their own knowledge of each rail.

I understand the concern that adding a surface to the catalog shouldn’t automatically enable unsupported behavior. Could we make that capability-driven instead? IORails can declare what it supports, and compilation can explicitly reject surfaces whose manifest requirements it cannot satisfy. If we still need an intentional rollout flag, I think it should live alongside the surface in the manifest/catalog rather than in a parallel list here. Should be a separate PR if it has to touches manifests.

Also, I wouldn’t connect relevant_chunks specifically to having a KB. From the rail’s perspective, it is another context input, similar to user_message and bot_message. The current limitation is that IORails cannot supply that context value yet.

Comment on lines +171 to +184
# Verdict fields a blocked caller may see.
_CLIENT_EVIDENCE_KEYS = frozenset(
{
"policy_violations", # content safety, llama guard
"triggered_violation", # activefence, gcp text moderation
"max_risk_score", # activefence, gcp text moderation
"trustworthiness_score", # cleanlab
"assessment", # policyai
"category", # policyai
"severity", # policyai
"score", # autoalign
"threshold", # autoalign
}
)

@Pouyanpi Pouyanpi Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think this is another smell.

If IORails needs to inspect provider-specific metadata to construct the caller facing explanation, then the shared outcome boundary is incomplete. The actions already have the information needed to construct that explanation. We should define whether RailOutcome.reason is caller safe and require blocking actions to populate it consistently. If structured evidence also needs to be exposed, that should be represented explicitly in the shared RailOutcome contract. either way, I don’t think IORails should maintain knowledge of each integration’s metadata fields.

I checked the actions behind these fields. ActiveFence, GCP moderation, and PolicyAI already provide a reason, so those entries appear redundant. Content Safety, Llama Guard, Cleanlab, and AutoAlign do not provide one, but they already have enough information to construct it at the point where they create the blocking outcome.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a consequence of the RailResult boundary discussed in #2261. If RailResult carried the complete RailOutcome, IORails would not need to maintain a list of provider-specific metadata fields here. I’d prefer fixing that shared boundary

Comment on lines +535 to +536
if "http_client" in _accepted_parameters(action):
return

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think accepting an http_client parameter is a reliable signal that the action does not need its optional dependency. It only says what the function can receive; it does not tell us which backend the configuration selected.

For example, an action may accept http_client and still use its local backend depending on its configuration. If we have conditional or alternative dependencies, I think that needs to be expressed in the manifest or validated against the selected backend rather than inferred from the Python signature.


# The parameter library actions resolve against ``llms``, by convention: an action needing a
# model declares ``model_name`` and indexes ``llms[model_name]``.
_MODEL_NAME_PARAM = "model_name"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here model dependencies are inferred from the action parameter name model_name, even though the manifest already has RailRequirements.models. If that information is not sufficient to connect a binding to a configured model, I think we should extend the manifest/binding contract. Hard-coding a parameter name means renaming it could silently disable validation.

It also looks like compile-time capabilities and runtime dependencies are being mixed together. We create an llms mapping containing None values only to communicate the configured model types.

Could compilation receive the available model types explicitly and validate them against the manifest requirements? That would avoid coupling validation to an action parameter naming convention and keep RailDependencies.llms for actual runtime models.

Comment on lines +688 to 690
compile_rail(flow, direction, deps)
except RailCompilationError as exc:
return str(exc)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also looks like a sign that compile-time capabilities and runtime dependencies are being mixed together.

we create an llms mapping containing None values just to communicate the configured model types, and then infer that a binding is a model dependency because its action parameter happens to be named model_name.

Could compilation receive the available model types explicitly and validate them against the manifest’s model requirements? that would avoid coupling the compiler to an action parameter naming convention and keep RailDependencies.llms for actual runtime models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: XL status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants