perf(imports): resolve top-level and rails exports lazily - #2256
Draft
Pouyanpi wants to merge 5 commits into
Draft
perf(imports): resolve top-level and rails exports lazily#2256Pouyanpi wants to merge 5 commits into
Pouyanpi wants to merge 5 commits into
Conversation
…ing Colang at import-time. Decouples actions from Colang
Resolve the public names exported from `nemoguardrails` and `nemoguardrails.rails` lazily via PEP 562 `__getattr__` instead of importing them eagerly at package import time. A bare `import nemoguardrails` previously booted ~700 modules (Colang parsers, tracing/OpenTelemetry, aiohttp, jinja2, ...) because the top-level package eagerly imported `RailsConfig`, `LLMRails`, and `Guardrails`. Lazy resolution defers that cost to first use of a heavy symbol, so lightweight consumers (version probes, type imports, submodule imports, built-in rail actions) no longer pay it. Making `nemoguardrails.rails` lazy also lets `rails.llm.options` initialize the package without pulling in the full runtime, which removes the circular import that made isolated rail-action imports fail, and removes the need for the explicit `nemoguardrails.rails` preload workaround. Behavior preserved: `__all__`, `dir()`, type-checker exports (via `TYPE_CHECKING`), unknown-attribute `AttributeError`, and the `NEMO_GUARDRAILS_IORAILS_ENGINE` alias of `LLMRails` to `Guardrails`. Resolution is intentionally not cached into module globals so that `importlib.reload(nemoguardrails)` re-reads the env alias. Also drops a now-redundant `cast(str, chunk)` in cli/chat.py that stricter type resolution exposes. Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
tgasser-nv
force-pushed
the
refactor/iorails-actions-decouple-colang
branch
from
August 6, 2026 15:16
ff09623 to
428d84d
Compare
Base automatically changed from
refactor/iorails-actions-decouple-colang
to
develop
August 6, 2026 15:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
import nemoguardrailseagerly importedRailsConfig,LLMRails, andGuardrailsat package-import time. Because those pull in the Colang parsers,the tracing/OpenTelemetry stack,
aiohttp,jinja2, and more, a bareimport nemoguardrailsbooted ~870 modules (~0.5s) even for callers thatonly needed a type or a submodule.
This PR resolves the public names of
nemoguardrailsandnemoguardrails.railslazily via PEP 562 module
__getattr__, pointing each name at its narrowestmodule (
RailsConfig -> rails.llm.config,LLMRails -> rails.llm.llmrails,Guardrails -> guardrails.guardrails). The heavy runtime is loaded only on firstuse of a heavy symbol.
Making
nemoguardrails.railslazy has a second effect: submodules such asnemoguardrails.rails.llm.optionscan now initialize therailspackage withoutdragging in
LLMRailsand its Colang runtime. That removes a circular import(
base_guardrails<->rails.llm.llmrails) which previously made isolatedrail-action imports fail, and removes the need for an explicit
nemoguardrails.railspreload workaround in the top-level__getattr__.Performance
Measured in fresh interpreters (median of 7). "Colang" is the number of
nemoguardrails.colang.*modules resident insys.modulesafter the import;"Modules" is total
sys.modules. Absolute times vary with machine load; themodule counts do not.
import nemoguardrailsfrom nemoguardrails import ChatMessagefrom nemoguardrails import RailsConfigimport nemoguardrails.llm.callimport nemoguardrails.library.content_safety.actionsfrom nemoguardrails import LLMRailsTakeaways:
import nemoguardrails,ChatMessage) drop from ~0.5s and ~870 modules to ~30 ms and ~200 modules,with zero Colang modules loaded.
llm_call, and built-in rail-action paths load zeroColang modules (the invariant refactor(iorails): Decouple llm_call and rail actions from unrelated Colang modules #2241 established, now reachable end-to-end
through the real package entry points).
LLMRailsstill loads the full Colang runtime, by design: it is thelegacy Colang engine. It is even slightly lighter than before (784 vs 874
modules) because it no longer eagerly loads the
Guardrails/tracing branch.Behavior preserved
__all__,dir(nemoguardrails), and unknown-attributeAttributeError.if TYPE_CHECKING:re-exports.NEMO_GUARDRAILS_IORAILS_ENGINEalias of top-levelLLMRailstoGuardrails.importlib.reload(nemoguardrails)re-evaluates that env alias: resolution isintentionally not cached into module globals, so a reload re-reads the
environment rather than returning a stale binding.
Reviewer notes
cli/chat.py(1 line): stricter type resolution through the newTYPE_CHECKINGre-exports letstyinferchunk: strin the streaming loop,which makes an existing
cast(str, chunk)redundant and trips thetygate.The value is always
strthere, so the redundant cast is removed (thecastimport is still used elsewhere in the file).
tests/llm/test_call_import_graph.py(from refactor(iorails): Decouple llm_call and rail actions from unrelated Colang modules #2241): its docstrings statethat the runtime property "could never be observed" because
nemoguardrails/__init__"imports the world." That premise no longer holdsafter this PR. Those static import-graph tests still pass and remain valid;
the new
tests/test_lazy_imports.pyis the runtime complement. The stalewording was left untouched here to keep the stack diff clean and can be
refreshed when the stack settles.
Verification
make test(fullpytest.initestpaths): 6299 passed, 178 skipped.pre-commit run --files ...on every changed file: all hooks pass, includingruff,ruff format, license insertion, andty.tests/test_lazy_imports.pyruns each assertion in an isolatedsubprocess so Colang loaded by one case (for example, resolving
LLMRails)cannot leak into another and mask a regression. It covers: Colang-free
import nemoguardrails,ChatMessage,RailsConfig, a built-in rail action,and
nemoguardrails.llm.call;LLMRailsloading the legacy runtime;Guardrailsresolving; the env-var alias and its re-evaluation on reload; andthe
dir/__all__/invalid-attribute contracts.AI Assistance
Checklist