Skip to content

Latest commit

 

History

History
399 lines (331 loc) · 24.7 KB

File metadata and controls

399 lines (331 loc) · 24.7 KB

validation-engine - Public API

Validating a Template

validate_bytes_with_path is the main entry point - pass raw template bytes and get a complete report. It requires an engine and a SchemaValidator, both of which should be created once and reused:

use rego_engine::RegoEngine;
// or cel_engine::CelEngine
use schema_validator::{SchemaValidator, SchemaValidatorConfig};
use validation_engine::{validate_bytes_with_path, EngineConfig, ValidateConfig};

// One-time setup (reuse across validations)
let schema_validator = SchemaValidator::default();
let engine = RegoEngine::new(EngineConfig::default())?;

// Validate
let bytes = std::fs::read("template.yaml")?;
let report = validate_bytes_with_path(
    &engine,
    &schema_validator,
    &bytes,
    ValidateConfig::default(),
    "template.yaml".to_string(),
)?;

for d in &report.diagnostics {
    println!("[{}] {} - {}", d.severity, d.rule_id, d.message);
}

On parse failure, validate_bytes_with_path returns Ok(report) with a synthetic F1101 diagnostic and status=Error rather than returning Err. This ensures callers always get a structured report.

Validating an AWS CLI Command

validate_aws_cli_command accepts raw service, operation, HTTP, trait, and request-parameter context. It owns operation classification, deterministic CloudFormation resource-type selection, request-to-template modeling, schema-backed property mapping, and diagnostic scoping to explicitly modeled properties:

use rego_engine::RegoEngine;
use schema_validator::SchemaValidator;
use validation_engine::{AwsCliCommand, AwsCliValue, EngineConfig, validate_aws_cli_command};

let engine = RegoEngine::new(EngineConfig::default())?;
let schema_validator = SchemaValidator::default();
let request = AwsCliCommand::new(
    "s3",
    "CreateBucket",
    [
        ("Bucket".into(), AwsCliValue::String { value: "example-bucket".into() }),
    ],
)
.with_service_prefix("s3")
.with_http_method("PUT");

let result = validate_aws_cli_command(&engine, &schema_validator, &request)?;
if let Some(report) = &result.report {
    for diagnostic in &report.diagnostics {
        println!("{}: {}", diagnostic.rule_id, diagnostic.message);
    }
} else {
    println!("{:?}: {}", result.status, result.reason);
}

AwsCliValue preserves bytes and 64-bit integer widths and explicitly marks unsupported values. Exact TemplateBody bytes are validated without rewriting; TemplateURL is skipped because validation is offline. Every result includes an operation kind, validation status, optional template source, resource candidates, and reason. Validated means the modeled template reached the normal validation pipeline; Skipped has no report and explains why. The modeled template runs through the normal template-validation pipeline (validate_bytes_with_path) with a fixed configuration that callers cannot tune, so every embedding reports the same findings for the same command: the default ValidateConfig at the STANDARD detail level — detailed enrichment is not supported for synthesized API-request templates because there is no user-authored source to annotate with context — gated at Warn severity, because a command is a deployment action rather than template authoring and Info-level guidance (best practices, replacement-on-update notes) has nothing for the caller to act on. AwsCliCommandValidation carries the resulting Option<diagnostics::output::ValidationReport>. The template field carries the exact bytes that were validated — the caller's original TemplateBody without reserializing, or the synthesized JSON template for adapter-mapped requests — so consumers can display the modeled template that produced the diagnostics. It is None when the request was skipped. Use validate_aws_cli_command_with_path when the embedding application needs a custom report path.

Deterministic closed-adapter contract. Operation-to-resource mapping uses a generated adapter catalog keyed by case-normalized canonical service_name and exact operation name. The catalog is produced by data-source/scripts/generate_aws_cli_catalog.py from each resource type's own provider handler metadata, resolved against botocore service models and structurally verified against the compiled CloudFormation schemas; it covers create and delete lifecycles for roughly seventy percent of all resource types plus curated update entries. Each adapter declares one CloudFormation resource type with explicit request-parameter-to-property pairs. Unregistered operations never receive an inferred resource type and are classified as UnmappedMutation (or DataPlaneMutation for data-plane verbs) with Skipped status.

Strict all-supplied-state mapping. Template synthesis is all-or-nothing: every request parameter the caller supplies must either (a) map to a resource property with a representable value, or (b) be an explicitly safe-to-ignore field (idempotency tokens, DryRun, or a declared primary identifier on update operations). If any supplied parameter fails both conditions — because it has no mapping, or its value cannot be type-matched to the target property — synthesis is SKIPPED and the reason names the offending parameter. This guarantees that validated templates faithfully represent the full caller-supplied state: no parameter is ever silently omitted from the synthesized template.

A CloudFormation constraint the API does not enforce is never reported against a command. A same-named API input and CloudFormation property can differ in value domain: the botocore model may declare a wider domain than the CloudFormation schema (an extra enum member, a looser numeric bound, string length, list or tag-map size, or a different pattern), or it may declare no corresponding constraint at all, in which case nothing proves that the service rejects what CloudFormation rejects. The generator records every such constraint on the mapping as unrepresentable — the CloudFormation enum, bounds, sizes, and pattern (paired with the anchored API pattern when the API declares a different one) — and the runtime settles it per value: a value outside the CloudFormation constraint skips synthesis, naming the parameter, the value, and the constraint the API does not enforce, so a command the service may accept is never modeled as a template that CloudFormation would reject. A value inside every CloudFormation constraint validates fully, and a constraint the API enforces at least as strictly keeps its CloudFormation finding. Same-named inputs whose meaning differs from the property (for example an API resource ID where the CloudFormation property carries the resource ARN) are removed from the catalog by a reviewed denylist and therefore skip synthesis as unmapped parameters.

Template-authoring advice is not reported for modeled state. Synthesized templates and wrapped Cloud Control desired state never had a template author, so rules whose only remediation is a template construct — replace a literal ARN, AMI ID, account ID, availability zone, password, or pseudo-parameter lookalike with a parameter, Ref/GetAtt, mapping, or dynamic reference — are dropped from those reports and counted as suppressed. They still apply to TemplateBody requests, which validate the caller's real template unchanged. Rules that judge the values themselves (schema constraints, deprecations, security posture such as a publicly accessible database) are always kept.

Cloud Control UpdateResource and DeleteResource may report a known TypeName supplied explicitly by the request, but they never synthesize state. There is no fuzzy inference, substring matching, or generic property-name guessing. TemplateBody validation is restricted to the closed set of CloudFormation operations that accept it; TypeName+DesiredState wrapping is restricted to exact Cloud Control CreateResource. service_name is the authoritative mapping identity and must be the exact canonical botocore service name, normalized only for ASCII case; the optional signing service_prefix is context only and cannot override it. There are no signing, endpoint, punctuation, or substring aliases. Any caller, including a future AWS SDK adapter in any language, must translate its native service identity to the canonical botocore service_name before invoking this API; the core intentionally does not guess aliases.

Constructing an Engine

Both engines take a single EngineConfig and return anyhow::Result:

use validation_engine::{EngineConfig, ExternalRuleSource};
use rego_engine::RegoEngine;
use cel_engine::CelEngine;

// No custom rules - built-in rules only
let engine = RegoEngine::new(EngineConfig::default())?;

// With custom rules
let engine = CelEngine::new(EngineConfig {
    custom_rules: vec![ExternalRuleSource { name: "my_rules.cel".into(), content: cel_source }],
    guard_rules:  vec![ExternalRuleSource { name: "policy.guard".into(), content: guard_source }],
    ..Default::default()
})?;

EngineConfig gains fields as the engine gains options - a field is added whenever one is needed for correctness or ease of use. The constructor and its with_* methods let you name only the options you set:

let config = EngineConfig::new()
    .with_custom_rules([ExternalRuleSource { name: "my_rules.cel".into(), content: cel_source }])
    .with_guard_rules([ExternalRuleSource { name: "policy.guard".into(), content: guard_source }]);
Engine custom_rules format guard_rules handling
RegoEngine Native Rego source Parsed and translated to Rego internally
CelEngine JSON with CEL expressions Parsed and translated to CEL internally

Both engines parse and translate guard_rules from raw Guard DSL source text - no pre-parsing needed.

Composite Engine

CompositeEngine (in the composite-engine crate) is an additive engine that composes two inner engines: CEL evaluates every built-in rule, and a separate external-only Rego engine evaluates the caller-supplied custom Rego and translated Guard rules. The external engine is constructed only when the configuration supplies such rules, and it still runs when built-in rules are disabled. Findings from both are concatenated; this pipeline performs the single finalize pass.

It is constructed from a CompositeEngineConfig, a type distinct from EngineConfig. The composite fixes which engine owns the built-ins, so the config has no field for engine-native built-in custom rules; it does accept caller-supplied custom rules in all three formats, layered on the built-ins. Custom CEL rules are evaluated by the engine that owns the built-ins (CEL), while custom Rego and translated Guard rules are evaluated by the external engine. RegoEngine, CelEngine, and EngineConfig are unchanged. EngineType now selects Rego, Cel, or Composite, with Composite as its default. EngineType is only a selector, so construct CompositeEngine directly when embedding, or select it in the CLI with --engine composite (the default).

Field (Rust / serialized) Type Description
rego_rules / regoRules Vec<ExternalRuleSource> Custom Rego rules layered on the built-ins, run by the external engine.
cel_rules / celRules Vec<ExternalRuleSource> Custom CEL rules layered on the built-ins, run by the built-in engine.
guard_rules / guardRules Vec<ExternalRuleSource> Guard DSL rules, translated and run by the external engine.
schema_validator_config / schemaValidatorConfig Option<SchemaValidatorConfig> Additional schemas observed by both inner engines.

Only the documented subset of the Guard language is translated; unsupported constructs are rejected at load time rather than silently ignored. CompositeEngineConfig::new() and its with_* methods set only the options you name:

use composite_engine::CompositeEngine;
use schema_validator::SchemaValidator;
use validation_engine::{
    CompositeEngineConfig, ExternalRuleSource, ValidateConfig, validate_bytes_with_path,
};

// CEL owns the built-ins and any custom CEL rules; the external-only Rego engine
// is built only because external Rego or Guard rules are supplied here.
let engine = CompositeEngine::new(
    CompositeEngineConfig::new()
        .with_cel_rules([ExternalRuleSource { name: "extra.json".into(), content: cel_source }])
        .with_rego_rules([ExternalRuleSource { name: "extra.rego".into(), content: rego_source }])
        .with_guard_rules([ExternalRuleSource { name: "policy.guard".into(), content: guard_source }]),
)?;

let schema_validator = SchemaValidator::default();
let bytes = std::fs::read("template.yaml")?;
let report = validate_bytes_with_path(
    &engine,
    &schema_validator,
    &bytes,
    ValidateConfig::default(),
    "template.yaml".to_string(),
)?;

Additional Resource Provider Schemas

Additional schemas extend the bundled CloudFormation resource schemas, so templates using properties or types CloudFormation has not published yet validate without false findings.

Supply overlay schemas through SchemaValidatorConfig::additional_schemas. The optional EngineConfig::schema_validator_config field holds this same config type: when present, a standalone engine built via new(EngineConfig) derives overlay-aware metadata (type names, GetAtt attributes, primary identifiers) from it. Language bindings and the CLI construct a SchemaValidator from the config once at their layer, then pass it to the engine so the already-built metadata is shared without redundant work.

use data_source::AdditionalSchemaSource;
use rego_engine::RegoEngine;
use schema_validator::{SchemaValidator, SchemaValidatorConfig};
use validation_engine::EngineConfig;

let overlay = AdditionalSchemaSource {
    type_name: None, // take typeName from the schema itself
    schema: std::fs::read_to_string("aws-lambda-function.json")?,
};

let schema_config = SchemaValidatorConfig {
    additional_schemas: vec![overlay.clone()],
};

// Standalone engine: nested schema config derives metadata automatically.
let engine = RegoEngine::new(
    EngineConfig::new().with_schema_validator_config(schema_config.clone()),
)?;

// Or: build a SchemaValidator separately for standalone schema validation.
let schema_validator = SchemaValidator::new(schema_config)?;

Construction fails, rather than degrading quietly, when a schema is malformed, names contradictory or non-canonical types, nests too deeply, defines an unsafe $ref graph, states nothing enforceable, contains conflicting enum representations, uses an invalid regular expression, or states a keyword/composition constraint the compiled model cannot enforce. Annotations beside a $ref are accepted; constraining siblings are rejected because draft-07 would ignore them. Apply a separate overlay to the property or referenced definition instead.

Merge model. An overlay may add entries to a collection and restate a single-valued constraint or a logical group; a bundled constraint never disappears silently - the one merge that can remove entries (required replacement) logs every removal. Adding to required or to a dependency list states a constraint, so it can legitimately produce a finding on a template that violates it.

Field kind Rule
properties, definitions, patternProperties deep-merged by key
required replaced when the overlay states the keyword (even as [], which clears it - removals are logged); unioned when the keyword is omitted
/properties/... lifecycle metadata lists, each dependentRequired/dependentExcluded key unioned
single-valued constraints (type, pattern, bounds, lengths, uniqueItems, format, additionalProperties, …) replaced when supplied
requiredOr, requiredXor, primaryIdentifier replaced as a whole group when supplied
allOf/anyOf/oneOf/if-then-else replaced when supplied
items (the schema every array element must satisfy) deep-merged, like one keyed entry
replacementStrategy, documentationUrl, sourceUrl replaced when supplied; these enrich reporting and constrain nothing
enum / enumCaseInsensitive one mutually exclusive field; supplying either replaces the other. A plain enum over a bundled case-insensitive list keeps case-insensitive comparison, so casings that validate today keep validating; supplying enumCaseInsensitive switches comparison to case-insensitive, which only ever accepts more

A $ref is never folded into the property pointing at it: overlay fields are merged beside the reference and combined with the whole chain at validation time. The table above decides each field within the chain too, so a hop that restates a single-valued constraint overrides the one further along while collections accumulate across every hop. A constraint-only overlay therefore applies, chains are followed to their end, and a definition changed by a later overlay still reaches every property referencing it. A chain longer than the resolver can follow is rejected rather than cut short. Overlays for one type apply in order.

Scope limits. An overlay cannot remove a lifecycle metadata entry, and cannot switch a case-insensitive enum to case-sensitive comparison. An overlay that states required replaces the prior list at that schema level (removals are logged); omitting the keyword preserves the base. Composition branches are full property schemas and are evaluated in full - branch required, dependency maps, value constraints, and nested if/then/else all participate in matching, and a selected conditional branch is enforced. Unrepresentable constructs and validation keywords are rejected. Conditional constraints from the separate build-time extension artifact remain independently enforced. Overlay-derived type, GetAtt, Ref, primary identifier, and schema metadata catalogs are propagated to both engines; regional availability and enum snapshots remain bundled.

Configuring Validation

ValidateConfig controls per-call behavior:

use validation_engine::ValidateConfig;
use diagnostics::DetailLevel;
use rules::{FilterConfig, Severity};

let config = ValidateConfig {
    filters: FilterConfig::default(),                    // include/exclude rules
    detail_level: DetailLevel::Detailed,                 // Standard | Detailed (default: Detailed)
    severity_level: Severity::Info,                      // minimum severity to report (default: Info)
    parameter_overrides: HashMap::from([                 // template parameter values
        ("Env".into(), "prod".into()),
    ]),
    pseudo_parameter_overrides: PseudoParameterOverrides {
        region: Some("us-west-2".into()),                // AWS::Region, etc.
        ..Default::default()
    },
    strict: false,                                       // true: upgrade Warning to Error (default: false)
    disable_builtin_rules: false,                        // true: skip all built-in rules, only evaluate custom/guard rules.
};

Performance metrics are always collected unconditionally.

Reading the Report

validate_bytes_with_path returns a ValidationReport:

report.file_path            // path to the validated template
report.status               // Ok, AnalysisIncomplete (findings may be omitted), or Error (pipeline failure)
report.diagnostics          // Vec<Diagnostic> - all findings
report.metadata.counts      // Summary { fatal, errors, warnings, informational, debug }
report.metadata.suppressed  // diagnostics removed by filters/severity gating
report.metadata.resources_scanned
report.metadata.rules_evaluated
report.metadata.strict      // whether strict mode was enabled
report.metadata.severity_level // minimum severity threshold used
report.metadata.budget_exhaustions // optional records with stable kind, description, limit, and per-budget impact
report.performance          // PerformanceMetrics with per-phase timings

Each record's analysis_incomplete value reports whether that specific exhausted budget can cause findings to be omitted. RequiredPropertyCombinations is context-only, so its value is false and the report can remain Ok; the report becomes AnalysisIncomplete when any exhausted budget has a true value.

Convert to output format:

// One serialized report model; the detail level controls whether the
// per-diagnostic enrichment fields are populated.
let standard = report.to_report(DetailLevel::Standard);  // diagnostics::output::ValidationReport, enrichment fields omitted
let detailed = report.to_report(DetailLevel::Detailed);  // diagnostics::output::ValidationReport, enrichment fields populated

Each Diagnostic contains:

d.rule_id           // e.g. "E3012", "F3002", "W3045"
d.severity          // Fatal | Error | Warn | Info | Debug
d.message           // human-readable description
d.source            // RuleOrigin: Schema | CfnLint | Engine | Custom | Guard
d.entity            // Option<Entity> { logical_id, entity_type, resource_type }
d.property_path     // e.g. "Properties.BucketName"
d.location          // Option<SourceSpan> { start_line, start_column, end_line, end_column }
d.suggested_fix     // Option<String>
d.documentation_url // Option<String>
d.category          // Option<String> - e.g. "Schema", "Best Practice", "Structure"
d.phase             // Option<Phase> - Parse, Schema, or Lint
d.rule_description  // Option<String> - human-readable rule description
d.related_resources // Option<Vec<RelatedResource>> - cross-resource references
d.condition_scenario // Option<HashMap<String, bool>> - condition values that trigger this
d.context           // Option<ViolationContext> - resolved values (Detailed level only)

Diagnostic Helpers

For engines that produce JSON diagnostics:

Function Description
extract_diagnostics(json_str, model, out, source_override) Parse a JSON array string into diagnostics, appending to out. source_override: Option<&RuleOrigin> allows overriding the origin for custom/guard rules.
make_resource_diagnostic(rule_id, message, model, resource_id, prop_path, suggested_fix) Build a Diagnostic for a known rule ID with auto-resolved span and severity. Panics if rule_id is not in the registry.

Guard Rule Loading

Function Description
guard::resolve_guard_config(rule_source_paths) -> Result<Vec<ExternalRuleSource>, String> Reads Guard DSL files from filesystem paths (files or directories, recursive). Returns pre-read rule sources.

Types

Type Description
ValidationEngine Trait that engines implement - provides evaluate_rules and rule metadata
EngineType Composite (default), Rego, or Cel - selects which validation engine evaluates rules
EngineConfig Engine construction config: custom_rules and guard_rules as ExternalRuleSource
CompositeEngineConfig Composite engine construction config: rego_rules, cel_rules, and guard_rules as ExternalRuleSource, plus optional schema_validator_config
ValidateConfig Per-call config: filters, detail level, severity level, parameter overrides, strict, disable_builtin_rules
ExternalRuleSource { name: String, content: String } - a pre-read rule file's identifier and raw content
ValidationError Parse(ParseError) or Engine(String)