Skip to content

test: restructure the local suite into unit/cli/contract layers - #129

Merged
scarmuega merged 2 commits into
mainfrom
test/restructure-local-suite
Aug 7, 2026
Merged

test: restructure the local suite into unit/cli/contract layers#129
scarmuega merged 2 commits into
mainfrom
test/restructure-local-suite

Conversation

@scarmuega

Copy link
Copy Markdown
Contributor

Why

The e2e suite required a real tx3c (and for one test, dolos + cshell) on the runner, so CI could never run half of it — see the failures on #127. Beyond the missing binaries it had three structural problems:

  • Vacuous passes: trix_toml_rejects_alias_only_ref / trix_toml_rejects_latest_ref only asserted a non-zero exit — on a runner with no tx3c, every command fails, so they were green for the wrong reason.
  • A shared-state flake: parallel tests raced on the real ~/.tx3/trix/config.toml (check-then-write in global::ensure_global_config), producing the TOML parse error at line 1 failure on ci: make CI green #127 — and the suite wrote into the developer's real ~/.tx3.
  • Environment-dependent results: locally the suite picked up whatever toolchain was installed, so a dev run and a CI run tested different things.

What

Restructure around the process boundary: each assertion moves to the innermost layer that can make it, and no test in this repo requires a real helper binary. Real cross-binary interop is the umbrella's DX e2e journeys' job (solution/e2e/), which gate toolchain releases with real released binaries.

Layer Where Spawns Asks
Unit src/** #[cfg(test)] nothing are the rules right?
CLI tests/cli/ trix only is the binary wired to the rules?
Contract tests/contract/ trix + a fake tx3c is trix's side of the tx3c contract right?
  • Unit: extracted interfaces::verify_cache_at (cache integrity rules, now testable against any directory) and added unit tests for it, interfaces::validate, and the resolver. The formerly vacuous lockfile tests are real assertions here now.
  • CLI: init scaffolding/preservation plus one wiring probe per fail-before-spawn error path (clap ref rejection, unknown alias, [interfaces] validation, digest tamper — which provably fires before tool resolution).
  • Contract: a std-only fake tx3c compiled at test time with plain rustc (deliberately not a cargo target — it cannot leak into dist artifacts). Tests assert what trix sends (argv, project-source-lowered vs cached-TII-decoded routing, per-protocol codegen layout) and how it interprets results (diagnostics envelope, garbage stdout, tool death, and the version gate against arbitrary reported versions — coverage real binaries can't provide).
  • Hermeticity: new TX3_HOME override for the ~/.tx3 root (cross-platform, unlike faking $HOME). Every spawned trix gets a throwaway root, an empty PATH, and scrubbed TX3_* vars: no more global-config race, no writes to the real ~/.tx3, identical behavior on a loaded dev machine and a bare runner.
  • Deleted: devnet_starts_and_cshell_connects (journey 04 covers the real round-trip better; its pkill -f dolos cleanup also killed unrelated processes) and the unused predicates/insta/libc dev-deps.

tests/README.md documents the methodology and where a new test belongs.

Verification

All of CI's gates pass locally with no toolchain reachable (TX3_HOME isolation + empty PATH reproduce a bare runner):

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo test --workspace --all-targets --all-features ✓ — 103 tests (78 unit / 10 cli / 15 contract), ~1.5s, zero ignored

Follow-ups (umbrella, not this repo)

Per the release-gate methodology, two journey gaps exist regardless of this PR: no journey exercises trix codegen or trix use/[interfaces] with real binaries. Those belong in solution/e2e/ as new journeys.

🤖 Generated with Claude Code

scarmuega and others added 2 commits August 7, 2026 09:13
The old e2e suite needed a real tx3c (and for one test, dolos + cshell)
on the machine, so CI could never run half of it, two tests passed
vacuously whenever every command failed, and parallel tests raced on the
real ~/.tx3/trix/config.toml. Restructure around the process boundary:
each assertion moves to the innermost layer that can make it, and no
test in this repo requires a real helper binary. Real cross-binary
interop belongs to the umbrella's DX e2e journeys, which gate releases.

- Unit layer: extract `interfaces::verify_cache_at` so the cache
  integrity rules are testable against any directory; add unit tests
  for it, `interfaces::validate`, and the reference resolver. The
  formerly vacuous alias-only/latest lockfile tests become real
  assertions here.
- CLI layer (tests/cli): spawns only the trix binary. init scaffolding
  and preservation, plus one wiring probe per fail-before-spawn error
  path (clap ref rejection, unknown alias, [interfaces] validation,
  digest tamper).
- Contract layer (tests/contract): trix's side of the tx3c process
  contract against a fake tx3c compiled at test time with plain rustc
  (never a cargo target, so it can't leak into dist artifacts). Asserts
  argv, artifact routing (project source lowered vs cached interface
  TII decoded), output interpretation, failure surfacing, and the
  version gate against arbitrary reported versions.
- Hermeticity: new TX3_HOME override for the ~/.tx3 root (works on
  every OS, unlike faking $HOME); the harness gives each test a
  throwaway root, an empty PATH, and scrubbed TX3_* vars, eliminating
  the global-config race and any dependence on installed toolchains.
- Delete devnet_starts_and_cshell_connects (journey 04 covers the real
  round-trip; its pkill -f dolos also killed unrelated processes) and
  the now-unused predicates/insta/libc dev-deps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Backslashes in the interpolated repo path are escape sequences inside a
TOML basic string, so trix.toml failed to parse on the Windows runner.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@scarmuega
scarmuega merged commit d37a809 into main Aug 7, 2026
10 checks passed
@scarmuega
scarmuega deleted the test/restructure-local-suite branch August 7, 2026 12:34
scarmuega added a commit to tx3-lang/e2e that referenced this pull request Aug 7, 2026
…work

Fill the journey gaps that became the sole real-binary coverage after
tx3-lang/trix#129 hermeticized trix's local suite:

- 07-codegen-consume: real `tx3c codegen` with the published
  codegen-v1beta0 templates, compile the generated ts-client in a host
  package, then resolve a transfer through it against a local devnet.
- 08-interface-consume: build a real TII, prime the interface cache
  byte-for-byte as `trix use` writes it, then real `tx3c decode` via
  `inspect tir` (alias + full ref) and codegen-for-interface.
- 09-protocol-fixture: the acme/hydra-heads production-scale protocol
  through check/build/inspect. No `trix test` slice yet: every upstream
  protocols/ fixture ships a stale scaffold tests/basic.toml referencing
  a `transfer` tx none of them define (tracked in the umbrella plans).
- 10-live-network: secrets-gated preprod resolve via a Demeter TRP
  endpoint, mirroring the sdks live-e2e env contract. Skips green
  without the env; new dx-e2e-live.yml runs it weekly + on dispatch.

Also: graduate 04-devnet-roundtrip to the stable job (the trix#123
expect fix ships in trix 0.26.2 on both channels — verified locally),
add 07/08 to the beta cherry-pick, and add pull_request triggers so
journey changes are validated pre-merge now that this is its own repo.

All journeys verified green in isolation and in the full suite against
trix 0.26.2 / tx3c 0.23.0 (the current stable channel), no leaked
daemons.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant