feat(sim): add xArm GraspGenX simulation - #3366
Draft
TomCC7 wants to merge 17 commits into
Draft
Conversation
1 task
❌ 7 Tests Failed:
View the top 3 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
A PD-held arm rests 1e-4..1e-3 rad off target, so microradian equality gates rejected every preview-then-execute pick and every plan requested while the arm merely held position (INVALID_START / 'no longer matches the stored plan start'). 0.01 rad absorbs hold jitter and still catches a genuinely moved arm.
Compose GraspGenXModule into xarm-grasp-sim so pick's provider path runs learned proposals (autoconnect fills the GraspGenSpec ref); CC's xArm gripper config from grasp_config. xarm-perception-sim is unchanged. Adds static_box_obstacles to ManipulationModuleConfig next to floor_z: startup box obstacles for scene furniture the planner must always respect. The grasp-sim table matches data/xarm_grasp_sim/scene.xml.
Convert MuJoCo's internal closing-joint feedback to aperture values and keep direct gripper RPCs in the coordinator's hold-last cache. This prevents arm trajectory commands from closing a gripper that pick just opened. Increase the xArm simulation pre-grasp clearance from 5 cm to 10 cm and cover both regressions with focused tests.
pointcloud_to_convex_hull_obj centers the hull on the mean of the points, but the obstacle was placed at obj.pose, which carries the bounding-box centre. The two differ by ~1.6cm on tall asymmetric objects, so mesh obstacles sat offset from the geometry they represent.
Plumb the existing WorldObstacleMonitor use_mesh_obstacles option through start_obstacle_monitor and enable it for the grasp sim. Bounding boxes envelop an object at every height, so a side approach collides with empty space; the hull follows the observed geometry. Other blueprints keep boxes.
pointcloud_to_convex_hull_obj named its output hull_{id(points)}.obj. id() is
a memory address, and CPython reuses a freed address for the next same-sized
array, so sequential callers overwrote each other: five of six scene objects
collapsed onto one hull file and every obstacle took the shape of whichever
was written last. Intermittent, because it depends on allocator reuse.
The default name is now a uuid. The obstacle monitor passes an explicit path
per object instead, so rescanning overwrites in place rather than growing the
cache without bound.
Pick suppressed every object obstacle for the whole transaction, so the approach was planned against an empty scene and swept through whatever stood between the arm and the target. Suppression is now staged. Candidate selection and the approach plan against the full scene; only the legs that must reach into and carry the target - grasp, close, retreat, and their feasibility check - hide the target, and nothing else is ever hidden. A failed restore is recorded on the transaction so it still surfaces, and pick_suppress_all_object_obstacles remains as a diagnostic escape hatch.
TomCC7
force-pushed
the
feat/grasp-05-xarm-sim
branch
from
August 5, 2026 05:45
8c7a8ae to
a5a10cf
Compare
TomCC7
force-pushed
the
feat/grasp-04-visualization
branch
from
August 5, 2026 05:45
74e7634 to
4cfb213
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
The full GraspGenX pick path needs a reproducible xArm simulation environment with object ground truth, planning obstacles, calibrated frames, gripper state, and live visualization.
Solution
xarm-grasp-simscene, blueprints, and test guide.The legacy gravity-sag tolerance patch is intentionally omitted: it modified a stored-plan freshness gate that no longer exists in the current
PlanExecutionManagerarchitecture.This is stack 5 of 5. Previous: #3365. Review the diff against
feat/grasp-04-visualization.How to Test
uv run dimos run xarm-grasp-sim-agentSetup and operator steps are in
XARM_GRASP_SIM_README.md. Verified: 235 consolidated tests passed with 7 deselected; blueprint registry checks passed with 3 optional-dependency skips and 42 deselected; Ruff andgit diff --checkpassed across the complete stack.AI assistance
OpenAI Codex with GPT-5 was used extensively for branch extraction, implementation cleanup, verification, and this description.
Checklist