✨ Add <{...}> variant data syntax for field values

[chrisjsewell]

This PR adds a new <{ ... }> syntax for injecting values from needs_variant_data directly into need field values.

What it does

Within a field value, <{ var.<path> }> is replaced by the corresponding value looked up from the configured needs_variant_data, e.g.:

# conf.py
needs_variant_data = {
    "platform": "arm",
    "build": {"opt_level": 2},
}

needs_fields = {
    "arch": {"schema": {"type": "string"}, "parse_variants": True},
    "opt": {"schema": {"type": "integer"}, "parse_variants": True},
}

.. req:: Example
   :id: VD_001
   :arch: <{ var.platform }>
   :opt: <{ var.build.opt_level }>

Here arch resolves to the string "arm" and opt to the integer 2.

Semantics

• Enablement follows the existing variant mechanism: the field must set parse_variants: True in needs_fields / needs_links. When parse_variants is False, the syntax is left untouched as a literal.
• The reference is a constrained dotted var.* path lookup — not arbitrary eval(). Operators, function calls, item access (var["cpu"]), need fields, build_tags, and Python builtins are not available. This keeps the semantics simple, safe, and replicable by other tooling (it mirrors the ubcode implementation).
• The resolved value is type-validated against the field schema (or array item schema); a warning is emitted on mismatch.
• Works for scalar, string, array, and link fields. For string fields a reference can be embedded in surrounding text (parts joined with spaces); for non-string scalars the reference must constitute the whole value so the native type is preserved.
• Clear warnings are emitted for invalid paths, missing keys (top-level and nested), and values that don't resolve to a leaf.

Changes

• New lookup_variant_data() in variant_data.py implementing the eval-free dotted-path resolution.
• resolve_functions() resolves <{...}> references against needs_config.variant_data (the VariantDataProxy remains for variant-function <<...>> and needif filter expressions).
• Documentation in dynamic_functions.rst (new "Variant data references" section) and cross-references in configuration.rst.
• Tests: field resolution (snapshot), error cases (invalid path, missing/nested-missing keys, type mismatches), and a parse_variants: False case asserting the syntax stays literal.

[codecov]

Codecov Report

❌ Patch coverage is 93.25843% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.41%. Comparing base (4e10030) to head (acb1bb7).
⚠️ Report is 292 commits behind head on master.

Files with missing lines	Patch %	Lines
sphinx_needs/needs_schema.py	89.79%	5 Missing ⚠️
sphinx_needs/variant_data.py	96.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1721      +/-   ##
==========================================
+ Coverage   86.87%   89.41%   +2.53%     
==========================================
  Files          56       74      +18     
  Lines        6532    10675    +4143     
==========================================
+ Hits         5675     9545    +3870     
- Misses        857     1130     +273     

Flag	Coverage Δ
pytests	89.41% <93.25%> (+2.53%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ubmarco]

Verdict

Looks good to me, just 3 findings.

---

1. Bug — string interpolation inserts an extra space

Severity: medium (functional, user-visible) · Where: sphinx_needs/functions/functions.py:377

When a <{...}> (or [[...]], or <<...>>) reference is embedded in surrounding text in a
string field, the resolved value gets an extra space. Example from the PR's own fixture:

.. req:: Embedded string from variant data
   :id: REQ_002
   :mystring: platform is <{ var.platform }>!

Produces (snapshot test_variant_data_fields_html):

"mystring": "platform is  arm !"

Note the double space before arm and the stray space before !. The documented
example claims a single space (docs/dynamic_functions.rst:370 → "platform is arm"), so
today the docs and the code disagree — and the code is the one that's wrong.

Root cause

Two behaviors stack:

1. _split_list flushes the literal segment without trimming whitespace adjacent to an
   interior token, so the part before <{ is "platform is " (trailing space kept) and the
   part after }> is "!".
2. resolve_functions then joins all parts for string fields with " ".join(...)
   (functions.py:377), inserting another space between every part.

So ["platform is ", "arm", "!"] → "platform is " + " " + "arm" + " " + "!" →
"platform is arm !".

Suggested direction

The literal segments already carry the author's intended spacing, so the join should
concatenate without injecting spaces — i.e. "".join(...) rather than " ".join(...).
With that, platform is <{ var.platform }>! → "platform is arm!" and the doc example becomes
correct as written.

sphinx-needs/sphinx_needs/functions/functions.py

Lines 373 to 386 in 675f33c

	else:
	resolved.append(item)
	if field_schema.type == "string":
	need[field] = " ".join(str(el) for el in resolved)
	elif field_schema.type in {"integer", "number", "boolean"}:
	# TODO(mh) unboxing the list for non-joinable types
	if len(resolved) > 1:
	raise ValueError(
	f"Field {field!r} of type {field_schema.type!r} cannot have multiple values"
	)
	need[field] = resolved[0]
	else:
	need[field] = resolved

Blast radius (for the author)

• Line 377 is shared with [[...]] dynamic functions and <<...>> variant functions
  (it predates this PR — last touched by ♻️ Allow for typed needs_extra_options fields #1516, Sep 2025). Fixing it changes their string-field
  interpolation too.
• I did not find an existing test that pins the doubled-space outcome for string field
  values (the [[copy("id")]] cases are body content via a different path; the rest are
  PlantUML node labels). This PR's REQ_002 snapshot is the first to encode it, so the fix
  should mostly require updating this PR's own snapshot — but the author should re-run the
  suite and confirm no DF/VF snapshot relied on the extra space.

---

2. Implement <{...}> as a variant-function value (don't ship it as a limitation)

Severity: feature gap · Where: functions.py (resolve_functions / _get_variant), variants.py

Today, a variant-data reference used as the value of a variant function is not resolved —
the value the variant function selects is returned verbatim. Rather than documenting this as a
limitation, the feature should support it (author's call whether in this PR or a follow-up).

Current behavior (verified empirically)

For <<[var.cpu == "arm"]:<{ var.opt }>, default>> with var = {cpu: "arm", opt: 2}:

Step	Result
_split_list / _split_string	captures the whole thing as one variant-function item; <{ var.opt }> is literal text inside it
match_variants / _get_variant (condition true)	returns the string '<{ var.opt }>' — not 2
string field outcome	field becomes the literal "<{ var.opt }>", silently, no warning
typed scalar field (e.g. integer)	worse — value coercion runs at parse time, so int("<{ var.opt }>") raises a parse warning
standalone <{ var.opt }> (for comparison)	resolves correctly to 2

Note the asymmetry: var.* already works in the variant-function condition
(<<[var.cpu == "arm"]:fast, slow>> → "fast"), because _get_variant evaluates the condition
with the var proxy in context. Only the value side is missing.

The silent literal passthrough for string fields is the worst part — no warning, wrong output.

sphinx-needs/sphinx_needs/functions/functions.py

Lines 398 to 405 in 675f33c

	def _get_variant(
	variant: VariantFunctionParsed, variants: dict[str, str], context: dict[str, Any]
	) -> None | str | int | float | bool:
	for expr, _, value in variant.expressions:
	expr = variants.get(expr, expr)
	if bool(eval(expr, context.copy())):
	return value
	return variant.final_value

Why it's a moderate, additive change (~3 files)

The two consumers of VariantFunctionParsed are cleanly separable: field resolution
(_get_variant) vs. filters/needflow colors (match_variants, which parses its own string
fresh). The constraint is that typed fields coerce the value at parse time, so a <{...}> value
must be parsed as a deferred reference rather than coerced. A clean shape:

1. variants.py — add VariantDataParsed to the value/final_value unions and a keyword-only
   parse_variant_data: bool = False to from_string; when set, a <{...}> value is stored as
   VariantDataParsed instead of being coerced. (No circular import: variant_data.py imports
   nothing from variants.py.)
2. needs_schema.py — pass parse_variant_data=True at the field/link from_string call sites
   (already guarded by parse_variants). match_variants keeps the default, so filters/colors
   are unchanged.
3. functions.py — in resolve_functions, if _get_variant returns a VariantDataParsed,
   resolve it via the existing _get_variant_data(...) before the type-check, reusing the
   validation/warning machinery this PR already added.

Result: var.* works symmetrically in both the condition and the value, for string and typed
fields, with proper type validation.

---

3. Test gaps (medium)

• lookup_variant_data error branches uncovered. "is not a mapping"
  (variant_data.py:74-78, e.g. var.platform.x where platform is a scalar) and "resolves to
  a mapping" (:86-90, e.g. var.build as a leaf) are not exercised. Codecov flags both. One
  fixture line each.
• Link fields are documented but untested. docs/dynamic_functions.rst:320 lists link
  fields as supported, and LinkSchema is wired for <{...}>, but no test exercises a link
  field with parse_variants: True and a <{...}> value.

sphinx-needs/sphinx_needs/variant_data.py

Lines 64 to 91 in 675f33c

	"""
	segments = [segment.strip() for segment in expression.split(".")]
	if len(segments) < 2 or segments[0] != "var" or any(not s for s in segments):
	raise VariantDataError(
	f"variant data reference {expression!r} is invalid: "
	"expected a dotted 'var.*' path"
	)
	current: Any = data
	traversed: list[str] = []
	for segment in segments[1:]:
	if not isinstance(current, dict):
	raise VariantDataError(
	f"variant data reference {expression!r} is invalid: "
	f"'var.{'.'.join(traversed)}' is not a mapping"
	)
	if segment not in current:
	traversed.append(segment)
	raise VariantDataError(
	f"Unknown variant data key: 'var.{'.'.join(traversed)}'"
	)
	traversed.append(segment)
	current = current[segment]
	if isinstance(current, dict):
	raise VariantDataError(
	f"variant data reference {expression!r} resolves to a mapping "
	f"('var.{'.'.join(traversed)}'); access a leaf value instead"
	)
	return current

[ubmarco]

See #1721 (comment)

[chrisjsewell]

1. Bug — string interpolation inserts an extra space

This is an existing bug that effects existing functionality, and so out of scope of this PR

2. Implement <{...}> as a variant-function value

This is out of the scope if this PR, and requires more thought because I am not convinced that such nested syntaxes are ideal

3. test gaps

legitmiate, will add (acb1bb7)