refactor(e2e-report): group the expectation grid by feature and fix scenario indexing

tests/e2e-cucumber/README.md

@@ -93,11 +93,29 @@ There is no tag-filter tiering. Each CI job runs the **whole** suite
 **pass / xfail / skip** at runtime from its capability tags plus the known-bug
 matrix, then reconciles the actual result against that expectation.
 
+### Naming
+
+Each feature file has a short **key** that prefixes both its scenario names and
+its ids. The key is usually the file's stem, but not always — `install_lifecycle`
+uses `lifecycle` and `model_serving` uses `serve` — so `FEATURE_KEYS` in
+`tests/feature_naming.rs` is the list, not this page:
+
+- **Scenario name** — `Scenario: <key>-<NN> - <description>`, numbered
+  sequentially in declaration order. The report sorts the grid's rows by this
+  index, so it must match the order in the file. Without the key the index names
+  nothing: every file used to number from 1, so "1" meant eight different
+  scenarios.
+- **`@id:`** — `<key>-<slug>`, so an id alone says which feature it belongs to.
+
+`tests/feature_naming.rs` enforces all of this (sequential, unique suite-wide,
+feature-qualified ids) in the ordinary `cargo test` run. Adding a feature file
+means adding its key to `FEATURE_KEYS` there.
+
 Scenarios carry stable-id and capability tags:
 
 | Tag | Meaning |
 |---|---|
-| `@id:<slug>` | Stable scenario id. Keys the expectation matrix and the report grid; every scenario has one. |
+| `@id:<key>-<slug>` | Stable scenario id, prefixed with its feature's key. Keys the expectation matrix and the report grid; every scenario has one. |
 | `@requires-gpu` | Needs a real AMD GPU. Resolves to **skip** (n/a) on a host with none (e.g. the mock job). |
 | `@requires-engine:<vllm\|lemonade>` | Pins the serve engine. Resolves to skip where that engine can't start (e.g. vLLM on a lemonade-only Strix host). |
 | `@requires-os:<linux\|windows>` | Premise is OS-specific; skip on other OSes. |

tests/e2e-cucumber/expectations.toml

@@ -158,7 +158,7 @@ flaky = true
 
 # --- EAI-7383: `rocm help` lists subcommands in declaration order, not
 # alphabetically. Platform-independent (pure CLI help output). ---
-[["help-lists-subcommands-alphabetically"]]
+[["examine-help-lists-subcommands-alphabetically"]]
 when = {}
 bug = "EAI-7383"
 reason = "rocm help lists subcommands in declaration order, not alphabetically."

tests/e2e-cucumber/features/chat.feature

@@ -1,7 +1,7 @@
 Feature: Chat and endpoint detection
 
   @id:chat-served-model-discoverable
-  Scenario: 1 - A served model is discoverable through the services list
+  Scenario: chat-01 - A served model is discoverable through the services list
     Given a model is being served
     And the model is registered with the CLI
     When the user checks for running services
@@ -13,7 +13,7 @@ Feature: Chat and endpoint detection
   # offers it — the notice must precede any request. (Previously an
   # untestable-black-box gap.)
   @id:chat-privacy-notice-accurate @requires-os:linux
-  Scenario: 2 - The privacy notice is shown before using a local endpoint
+  Scenario: chat-02 - The privacy notice is shown before using a local endpoint
     Given a model is being served locally
     And the model is registered with the CLI
     When the user opens interactive chat
@@ -23,7 +23,7 @@ Feature: Chat and endpoint detection
     Then interactive chat exits successfully
 
   @id:chat-managed-model-interactive @requires-os:linux
-  Scenario: 3 - Interactive chat uses a running managed model
+  Scenario: chat-03 - Interactive chat uses a running managed model
     Given a running managed model is available locally
     When the user opens interactive chat
     Then the local endpoint is shown for confirmation
@@ -35,7 +35,7 @@ Feature: Chat and endpoint detection
     Then interactive chat exits successfully
 
   @id:chat-endpoint-shown-in-services
-  Scenario: 4 - A served model's endpoint is shown in the services list
+  Scenario: chat-04 - A served model's endpoint is shown in the services list
     Given a model is being served
     And the model is registered with the CLI
     When the user lists running services
@@ -46,7 +46,7 @@ Feature: Chat and endpoint detection
   # assertion (a tools-bearing request is accepted) is engine-agnostic, so no GPU
   # is required — dropping @requires-gpu gives this per-PR mock-lane coverage.
   @id:chat-tool-definitions-accepted
-  Scenario: 5 - Chat requests that include tool definitions are accepted
+  Scenario: chat-05 - Chat requests that include tool definitions are accepted
     Given a managed runtime is active
     And a model is served in the background
     When a chat request with tool definitions is sent
@@ -57,7 +57,7 @@ Feature: Chat and endpoint detection
   # reply, which is engine-agnostic — real generation is covered by the
   # @requires-gpu serve-*-inference scenarios.
   @id:chat-end-to-end-local-model
-  Scenario: 6 - End-to-end chat through a locally served model
+  Scenario: chat-06 - End-to-end chat through a locally served model
     Given a managed runtime is active
     And a model is served in the background
     And the served model has been detected
@@ -70,7 +70,7 @@ Feature: Chat and endpoint detection
   # reports `rocm chat` as covered. Runs on mock (no GPU): the local provider
   # resolves the planted managed-service record and talks to the mock server.
   @id:chat-cli-oneshot-prompt
-  Scenario: 7 - The chat CLI answers a one-shot prompt against a local server
+  Scenario: chat-07 - The chat CLI answers a one-shot prompt against a local server
     Given a model is being served
     And the model is registered with the CLI
     When the user sends a one-shot chat prompt through the CLI

tests/e2e-cucumber/features/dash.feature

@@ -6,7 +6,7 @@ Feature: Interactive dashboard
   # not yet promoted to a blocking contract (tracked as a follow-up).
 
   @id:dash-opens-and-navigates @requires-os:linux
-  Scenario: 1 - A user opens the dashboard and navigates to ROCm setup
+  Scenario: dash-01 - A user opens the dashboard and navigates to ROCm setup
     When the user opens the dashboard with demo data
     Then the dashboard home view is displayed
     When the user opens the ROCm view
@@ -15,7 +15,7 @@ Feature: Interactive dashboard
     Then the dashboard exits successfully
 
   @id:dash-chat-offline-reply @requires-os:linux
-  Scenario: 2 - A user receives a response in interactive chat
+  Scenario: dash-02 - A user receives a response in interactive chat
     Given interactive chat uses an offline assistant
     When the user opens interactive chat
     And the user sends a message about GPU health
@@ -24,7 +24,7 @@ Feature: Interactive dashboard
     Then interactive chat exits successfully
 
   @id:dash-loading-service-status @requires-os:linux
-  Scenario: 3 - The dashboard reports a model that is still loading as loading
+  Scenario: dash-03 - The dashboard reports a model that is still loading as loading
     Given a managed model is still loading
     When the user opens the dashboard
     And the user opens the Observe view
@@ -33,7 +33,7 @@ Feature: Interactive dashboard
     Then the dashboard exits successfully
 
   @id:dash-managed-service-metrics @requires-os:linux
-  Scenario: 4 - Observe displays metrics from a managed model
+  Scenario: dash-04 - Observe displays metrics from a managed model
     Given a managed model exposes serving metrics
     When the user opens the dashboard
     And the user opens the Observe view
@@ -42,7 +42,7 @@ Feature: Interactive dashboard
     Then the dashboard exits successfully
 
   @id:dash-help-guidance @requires-os:linux
-  Scenario: 5 - A user can discover dashboard help and next-step guidance
+  Scenario: dash-05 - A user can discover dashboard help and next-step guidance
     When the user opens the dashboard with demo data
     And the user opens dashboard help
     Then navigation and next-step guidance are displayed
@@ -51,7 +51,7 @@ Feature: Interactive dashboard
     Then the dashboard exits successfully
 
   @id:dash-command-palette-navigation @requires-os:linux
-  Scenario: 6 - A user navigates to Serving through the command palette
+  Scenario: dash-06 - A user navigates to Serving through the command palette
     When the user opens the dashboard with demo data
     And the user opens the command palette
     Then dashboard destinations are displayed
@@ -61,7 +61,7 @@ Feature: Interactive dashboard
     Then the dashboard exits successfully
 
   @id:dash-managed-service-visible @requires-os:linux
-  Scenario: 7 - A managed model is visible in the dashboard
+  Scenario: dash-07 - A managed model is visible in the dashboard
     Given a running managed model is available locally
     When the user opens the dashboard
     And the user opens the Observe view

tests/e2e-cucumber/features/diagnose.feature

@@ -11,38 +11,38 @@ Feature: Diagnosing failures and listing fixes
   # and a plan) and the query/refusal contracts.
 
   @id:diagnose-matches-known-symptom
-  Scenario: 1 - Diagnosing a recognised failure reports a likely cause and a fix
+  Scenario: diagnose-01 - Diagnosing a recognised failure reports a likely cause and a fix
     Given a user who hit a known ROCm failure
     When the user asks the CLI to diagnose that symptom
     Then the CLI reports a likely cause with a suggested fix
 
   @id:diagnose-always-offers-a-way-forward
-  Scenario: 2 - Diagnosing any failure always gives the user a way to escalate
+  Scenario: diagnose-02 - Diagnosing any failure always gives the user a way to escalate
     Given a user who hit a failure the CLI does not recognise
     When the user asks the CLI to diagnose that symptom in machine-readable form
     Then the CLI always points to somewhere the problem can be reported
 
   @id:diagnose-json-has-match-flag
-  Scenario: 3 - A diagnosis is available in machine-readable form for tooling
+  Scenario: diagnose-03 - A diagnosis is available in machine-readable form for tooling
     Given a user who hit a known ROCm failure
     When the user asks the CLI to diagnose that symptom in machine-readable form
     Then the result is machine-readable and identifies the matched cause
 
-  @id:fix-lists-known-recipes
-  Scenario: 4 - The user can see every fix the CLI knows how to apply
+  @id:diagnose-fix-lists-known-recipes
+  Scenario: diagnose-04 - The user can see every fix the CLI knows how to apply
     When the user asks the CLI which fixes it offers
     Then the CLI lists the fixes it can apply
     And each fix indicates whether the CLI can apply it automatically
 
-  @id:fix-dry-run-changes-nothing
-  Scenario: 5 - Previewing a fix explains the change without making it
+  @id:diagnose-fix-dry-run-changes-nothing
+  Scenario: diagnose-05 - Previewing a fix explains the change without making it
     Given a user who has chosen a known fix
     When the user previews that fix without applying it
     Then the CLI describes what the fix would change
     And nothing on the machine is changed
 
-  @id:fix-unknown-id-rejected
-  Scenario: 6 - Asking for a fix the CLI does not know is refused clearly
+  @id:diagnose-fix-unknown-id-rejected
+  Scenario: diagnose-06 - Asking for a fix the CLI does not know is refused clearly
     Given a user who names a fix the CLI does not offer
     When the user asks the CLI to apply that fix
     Then the CLI refuses and explains that the fix is not recognised

tests/e2e-cucumber/features/examine.feature

@@ -1,32 +1,32 @@
 Feature: GPU detection and system inspection
 
   @id:examine-version
-  Scenario: 1 - The CLI reports its version
+  Scenario: examine-01 - The CLI reports its version
     When the user asks for the version
     Then a version string is returned
 
   @id:examine-engines-list
-  Scenario: 2 - The CLI lists all supported engines
+  Scenario: examine-02 - The CLI lists all supported engines
     When the user lists available engines
     Then all supported engines are listed
 
   # Dogfooding #24: the `rocm help` subcommand list is in declaration order, not
   # alphabetical, which makes it harder to scan. Expected to FAIL until fixed —
   # surfaces the bug so it can be ticketed.
-  @id:help-lists-subcommands-alphabetically
-  Scenario: 5 - The help output lists subcommands in alphabetical order
+  @id:examine-help-lists-subcommands-alphabetically
+  Scenario: examine-03 - The help output lists subcommands in alphabetical order
     When the user asks for help
     Then the subcommands are listed in alphabetical order
 
   @id:examine-detects-gpu-and-driver @requires-gpu
-  Scenario: 3 - System inspection detects the GPU and driver
+  Scenario: examine-04 - System inspection detects the GPU and driver
     Given a machine with an AMD GPU
     When the user inspects the system
     Then the inspection reports which GPU is installed
     And the inspection reports that the driver is available
 
   @id:examine-distinguishes-unmanaged-rocm @requires-gpu
-  Scenario: 4 - System inspection distinguishes CLI-managed from pre-existing ROCm
+  Scenario: examine-05 - System inspection distinguishes CLI-managed from pre-existing ROCm
     Given a machine with a ROCm install that was not set up by the CLI
     When the user inspects the system
     Then the inspection reports the install as pre-existing