feat: nav evaluator

[aclauer]

Contribution path

• Small, safe change that does not need a tracking issue
• Linked issue or discussion: DIM-XXX / #XXX / URL

Problem

Automated navigation tests

Solution

dimos nav-eval

run runs on the datasets
ingest automatically adds test cases for a new dataset
pick-case lets you manually add test cases. you can specify tags, negative cases, etc

How to Test

dimos nav-eval run

AI assistance

Checklist

• I have read and approved the CLA.

[greptile-apps]

Greptile Summary

The PR introduces the dimos nav-eval workflow for ingesting recordings, curating cases, replaying navigation pipelines, and reporting planner metrics. The latest validation changes cover the previously omitted positive-only fields, but non-finite and collision-band overrides remain insufficiently validated.

• Adds dataset manifests, case generation and curation, pipeline replay, metrics, reporting, visualization, and parallel suite execution.
• Shares packed voxel-key constants and helpers across mapping and evaluator code.
• Registers the evaluator CLI and removes the prior scenario-based evaluator blueprint.

Confidence Score: 4/5

The PR is not yet safe to merge because invalid numeric overrides can silently corrupt evaluation scores and invalid worker counts remain accepted.

Numeric validation still permits NaN values to reach goal and kinematic checks, while the previously reported worker-count issue remains in the current CLI and silently selects serial execution for zero or negative counts.

Files Needing Attention: dimos/navigation/nav_3d/evaluator/config.py, dimos/navigation/nav_3d/evaluator/cli.py, dimos/navigation/nav_3d/evaluator/runner.py

Important Files Changed

Filename	Overview
dimos/navigation/nav_3d/evaluator/config.py	Expands positivity validation, but non-finite values and collision-band bounds remain accepted and can corrupt evaluation results.
dimos/navigation/nav_3d/evaluator/cli.py	Adds the nav-eval command suite; worker-count validation remains outstanding and numeric overrides can pass non-finite values into evaluation.
dimos/navigation/nav_3d/evaluator/runner.py	Coordinates suite execution and scoring; invalid worker counts reach its serial fallback.
dimos/navigation/nav_3d/evaluator/metrics.py	Implements collision, reachability, clearance, and kinematic metrics whose results depend on receiving valid finite configuration values.
dimos/navigation/nav_3d/evaluator/test_nav_eval.py	Covers the evaluator pipeline extensively but does not establish rejection of non-finite overrides or invalid worker counts.

_{Reviews (6): Last reviewed commit: "Fix" | Re-trigger Greptile}

[greptile-apps]

Incomplete numeric override validation

When --set supplies a non-positive value such as kinematic_window_m=0, validate() accepts it because several numeric fields are omitted from the positivity checks. The invalid value then reaches operations such as np.arange(..., spacing), causing a runtime exception instead of a readable CLI validation error.

Knowledge Base Used: Navigation flow: from goal to motor commands

[greptile-apps]

Unbounded worker count option

When --workers is zero or negative, Typer accepts the value even though evaluation requires at least one worker. Zero and negative values silently bypass the requested multiprocessing path, while invalid counts reaching pool construction abort evaluation before any dataset is scored.

[greptile-apps]

Numeric validation remains incomplete

When --set supplies a non-positive value for an omitted field such as support_radius_m, support_depth_m, align_tol, max_slope, or max_step_m, validation accepts it. These values then invalidate support, alignment, or kinematic calculations, causing incorrect scores or an empty replay that terminates with ValueError("no cases to evaluate").

Suggested change

	for name in (
	"voxel_size",
	"max_range",
	"robot_height",
	"robot_length",
	"robot_width",
	"goal_tolerance",
	"visit_radius_m",
	"kinematic_window_m",
	):
	for name in (
	"voxel_size",
	"max_range",
	"robot_height",
	"robot_length",
	"robot_width",
	"support_radius_m",
	"support_depth_m",
	"goal_tolerance",
	"align_tol",
	"visit_radius_m",
	"max_slope",
	"max_step_m",
	"kinematic_window_m",
	):

[greptile-apps]

Numeric validation still accepts invalid bounds

When --set supplies nan for fields such as goal_tolerance or max_slope, the <= 0 checks accept it, causing valid paths to fail goal checks or invalid paths to pass kinematic checks. ground_margin and body_clearance are also omitted from the positivity loop, so non-positive collision-band bounds can produce incorrect collision results and silently corrupt evaluation scores.

Knowledge Base Used: Navigation flow: from goal to motor commands