RFC: Choose the canonical DimOS simulation path: PimSim or Sim2

[Nabla7]

Related: #2894, #2176, #3310, #3311, #3346

Decision requested

We need to choose the simulation path that the team will support going forward:

1. Continue building on the current PimSim integration to preserve delivery speed and
   working robot/evaluation verticals.
2. Reintroduce the stronger generic runtime contracts from feat/sim2-runtime, accepting
   more architectural work before returning to product features.

This is not a choice between PimSim and DimOS.

PimSim already runs as ordinary DimOS modules, inside DimOS workers, using DimOS
blueprints, typed streams, transports, RPC, hardware adapters, and
ControlCoordinator.

The choice is between:

• the current working, robot-oriented PimSim implementation; and
• a more generic Sim2 runtime that absorbs the working PimSim implementation.

What PimSim actually is

The current documentation does not explain the integration boundary adequately. This
has created the incorrect impression that PimSim bypasses the DimOS module system or
introduces a parallel application runtime.

It does neither.

Discovery and blueprint integration

DimOS defines the provider-neutral interface:

class SimulationProvider(Protocol):
    def build(self, request: SimulationRequest) -> SimulationBinding: ...

PimSim registers PimSimProvider
through the dimos.simulation.providers package entry point.

PimSimProvider.build() returns a normal SimulationBinding. Its backend field is a
DimOS Blueprint.

The regular DimOS ModuleCoordinator then:

• deploys the modules into DimOS worker processes;
• connects typed ports;
• selects LCM or Zenoh transports;
• injects module RPC references;
• starts and stops module lifecycles;
• composes the simulation with mapping, navigation, manipulation, perception, memory,
  skills, and agents.

PimSim has no separate blueprint system, worker manager, transport system, agent
runtime, navigation stack, or application lifecycle.

Existing PimSim DimOS modules

Component	DimOS interface	Responsibility
PimSimG1	Module; lidar, odometry and TF outputs; lifecycle, reset, respawn and sensor-health RPCs	Owns G1 physics. Joint commands and state cross shared memory between the simulation and the DimOS ControlCoordinator.
PimSimGo2	Module; cmd_vel input; point-cloud, lidar, RGB, camera-info, odometry, joint-state and TF outputs; scene RPCs	Owns Go2 physics and sensors. It currently also owns the Go2 locomotion policy, which is a known exception that should be moved into the normal DimOS control path.
PimSimXArm7	Module; TF and scene-entity outputs; lifecycle and reset RPCs	Owns xArm mechanics. Joint and gripper commands/state use the existing DimOS shared-memory manipulator adapter.
PimSimXArmCamera	Module; TF input; typed RGB, depth and camera-info outputs	Renders from scene assets and synchronized physics transforms without coupling rendering to the physics loop.
PimSimXArmSceneBridge	Module; scene-state input; DetObject output; injected manipulation and episode-control specs	Converts simulation state into ordinary DimOS perception and manipulation interfaces.
PimSimClient	Provider-neutral SceneControl implementation	Lets the existing DimSim-style E2E tests reset, respawn, add obstacles, publish goals and query evaluator truth.

Runtime and transport boundary

DimOS Blueprint / ModuleCoordinator
                   |
                   v
         PimSim DimOS Module(s)
          |         |          |
          |         |          +-- RPC: reset and evaluator intervention
          |         +------------- typed streams: camera, lidar, odom, TF
          +----------------------- SHM: joint commands and robot state
                   |
                   | direct in-process calls
                   v
              SimSession
                   |
                   v
             MuJoCo backend

The ownership split is:

DimOS owns	PimSim owns
Blueprints and module lifecycle	Physics state and contact
Worker deployment	Deterministic session stepping
Typed messages and transports	MuJoCo model binding and composition
ControlCoordinator and policies	Simulated sensor generation
Mapping and navigation	Scene-package execution
Manipulation and perception	Renderer implementations
Memory, skills and agents	Evaluator-only physical truth
E2E test runner	Reset and scenario mechanics

The intended data paths are:

• High-rate joint commands and state: shared memory.
• Camera, lidar, odometry, TF, detections and commands: typed DimOS streams, normally
  over Zenoh.
• Reset, respawn and evaluator intervention: low-frequency RPC.
• Policies: DimOS ControlTask and ControlCoordinator.
• Cameras: independent DimOS modules consuming synchronized simulation state.

The PimSim core is a passive library. SimSession provides load, reset, step,
snapshot, and close. It contains no DimOS dependency.

A DimOS module owning a synchronous internal library is normal DimOS architecture.
Planning modules own planning libraries and perception modules own inference
runtimes. Turning every MuJoCo operation into another module or RPC call would put
distributed communication inside the physics loop.

Therefore, the current PimSim integration is already deeply integrated with DimOS and
uses the module system at the correct subsystem boundary.

The main known boundary violation is the Go2 locomotion policy living inside
PimSimGo2. That should be corrected under either option.

Shared requirements

The following requirements apply regardless of which implementation is selected:

1. DimOS blueprints, skills and agents must not access simulator internals.
2. Policies belong in ControlCoordinator, not inside the simulation runtime.
3. One process owns each authoritative MuJoCo model and physics state.
4. The physics loop must not depend on per-tick Zenoh or RPC calls.
5. High-rate control uses shared memory.
6. Observable robot and sensor state uses typed DimOS streams.
7. Scenario control and evaluator intervention use RPC.
8. Camera rendering can use mesh or Gaussian-splat implementations independently of
   MuJoCo while preserving calibration and scene alignment.
9. Evaluator truth must remain separate from ordinary robot observations.
10. Authored, procedural and captured environments compile into the same validated
    scene.meta.json package contract.
11. The existing DimSim-style E2E tests remain the provider-neutral evaluation
    contract.
12. The overlapping contents of dimos/simulation must be audited and removed as
    their replacements reach parity.

Option A: Continue with the current PimSim integration

Treat the current pimsim-dimos provider and its DimOS modules as the supported
simulation path.

Continue extending the already working G1, Go2 and xArm verticals while correcting
specific architectural debt incrementally.

Work implied by this option

• Document the module, shared-memory, typed-stream and RPC boundaries in both
  repositories.
• Move the Go2 policy into ControlCoordinator.
• Keep G1 and xArm on the existing DimOS hardware-adapter path.
• Continue the scene compiler, procedural generation, camera and evaluation roadmap.
• Replace robot-specific duplication when it causes concrete maintenance problems.
• Retire old MuJoCo paths in dimos/simulation once PimSim covers their consumers.
• Continue validating through the existing DimSim-compatible E2E test bodies.

Advantages

• Preserves the currently working robot, scene, sensor and manipulation workflows.
• Keeps implementation work close to user-visible evaluations.
• Avoids stopping feature work for another foundational rewrite.
• Allows developers to use the system now through ordinary DimOS blueprints.
• Keeps the simulation core independently testable and free of DimOS imports.

Architectural limitations

• G1, Go2 and xArm currently have separate physics-owning module implementations.
• The current shared-memory implementation has fixed-size, independently sequenced
  buffers rather than one coherent versioned action/observation frame.
• There is no generic SimModule or canonical robot-profile abstraction.
• Live execution exists, but the full Sim2 lockstep clock contract does not.
• External renderers and sensors do not yet share a canonical
  WorldManifest / WorldStateFrame synchronization protocol.
• Repository separation continues to create some review and installation friction.

Choosing this option means accepting these limitations temporarily in exchange for
continuing product and evaluation work immediately.

Option B: Reintroduce Sim2 as the canonical runtime

Use feat/sim2-runtime
as the architectural reference, but do not merge the old WIP branch wholesale.

Create a clean dimos/sim2 migration namespace and port the working PimSim
implementation behind its contracts.

Sim2 contracts worth preserving

• One generic SimModule owns the simulation runtime.
• SimBackend hides MuJoCo, kinematic or future backend details.
• SimRobotSpec describes robot identity, control family, assets, joints, sensors and
  capabilities.
• Robot-specific simulation knowledge lives beside each robot as a profile.
• ControlCoordinator owns policies and task arbitration.
• A versioned, dynamically sized RobotChannel carries coherent action and
  observation frames over shared memory.
• Live and lockstep modes use the same policy and hardware-adapter path.
• WorldManifest provides stable entity discovery.
• WorldStateFrame provides episode- and tick-stamped dynamic state.
• SensorReady provides an exact synchronization barrier for external cameras and
  sensors.
• Scenes use the existing ScenePackage contract.
• Unsupported backend capabilities fail explicitly.

Intended structure

dimos/sim2/
├── spec.py
├── module.py
├── session.py
├── backend/
│   ├── base.py
│   ├── mujoco/
│   └── kinematic.py
├── control/
│   ├── channel.py
│   ├── adapters/
│   └── tick_source.py
├── sensors/
├── scene/
└── testing/

The working PimSim implementation would supply the MuJoCo backend, scene composition,
robot bindings, sensors, renderers and validated robot behavior. This is a port and
consolidation, not a clean-room rewrite.

Advantages

• Produces one generic simulation module rather than one physics module per robot.
• Makes robot addition profile-driven.
• Gives ControlCoordinator a coherent, versioned and deterministic control ABI.
• Supports exact lockstep evaluation and sensor synchronization.
• Establishes a canonical world-state contract for cameras, splats, lidar and dynamic
  actors.
• Creates a clean destination from which the current dimos/simulation package can
  be retired.

Costs and risks

• Feature work slows while runtime, control and world-state contracts are reviewed
  and landed.
• ControlCoordinator, shared-memory adapters and robot blueprints must change.
• The previous Sim2 attempt touched too many systems at once and did not reach user
  adoption.
• A poorly scoped revival could repeat that failure despite the stronger design.
• Existing PimSim behavior must be preserved through incremental ports and explicit
  parity tests.

Choosing this option means prioritizing the generic runtime architecture before
continuing broad feature development.

Direct comparison

Question	Continue PimSim	Reintroduce Sim2
Is it already usable through DimOS modules?	Yes	Prototype only
Immediate robot coverage	G1, Go2 and xArm	Must be ported
Near-term evaluation work	Continues immediately	Slows during foundation work
Physics ownership	Robot-oriented DimOS modules	One generic SimModule
Robot integration	Current runtime classes	Profiles by control family
Control ABI	Existing SHM adapters	Versioned coherent RobotChannel
Policy ownership	Correct for G1/xArm; Go2 must move	Enforced through generic adapters
Execution modes	Live	Live and lockstep
External sensor contract	Typed sensor streams	Manifest, world state and sensor barriers
Scene packages	Working	Contract exists; implementation must be ported
Review surface	Incremental fixes and features	Cross-cutting runtime and control work
Long-term cleanup	Incremental retirement	Explicit replacement of old simulation tree

Required validation under either option

The selected path must demonstrate:

• G1 policy execution through ControlCoordinator and shared memory.
• Go2 policy execution through ControlCoordinator, with no policy inside the
  simulator module.
• xArm plan, execute, grasp, retained lift, place and deterministic reset.
• Camera, depth, lidar, odometry, TF and joint-state cadence and timestamp checks.
• The same DimSim-style test bodies against supported providers:
  • walk forward;
  • dynamic path replanning;
  • semantic navigation to bed;
  • semantic navigation to couch;
  • semantic navigation to kitchen;
  • semantic navigation to television.
• Source-checkout and installed-package scene resolution.
• Deterministic reset, seed, episode and tick identity.
• Separation of ordinary observations from evaluator-only truth.
• An explicit list of old simulator paths deleted after parity.

Documentation required

Whichever option is selected, the main DimOS simulation documentation must immediately
explain:

• which DimOS modules own simulation;
• where the physics loop runs;
• how ControlCoordinator exchanges commands and state through shared memory;
• which data uses typed streams over Zenoh;
• which operations use RPC;
• how cameras and external sensors consume simulation state;
• how scenes are compiled and loaded;
• how developers add robots, policies, sensors, environments and evaluations;
• which existing G1, Go2 and xArm workflows currently work.

The current absence of this explanation is itself an integration problem.

Review requested

Please choose one:

Decision: Continue PimSim / Reintroduce Sim2

Reason:
Blocking concern:
Minimum architectural correction required:
Required validation before adoption:
Old path(s) that should be retired:

If rejecting both paths, please provide an alternative with explicit answers for:

• DimOS module ownership;
• physics process ownership;
• ControlCoordinator integration;
• shared-memory layout;
• typed-stream boundaries;
• scene and world-state contracts;
• deterministic reset and lockstep execution;
• external camera and sensor synchronization.

[Nabla7]

Update: the integration branch has now been split into reviewable pieces.

PimSim does not replace the DimOS runtime. It provides simulated hardware as normal DimOS modules.

Plain-English Runtime Trace

dimos --simulation mujoco --simulation-provider pimsim \
  run unitree-g1-groot-wbc
        |
        v
The normal DimOS G1 blueprint asks for simulated G1 hardware
        |
        v
The installed PimSimProvider returns:
- a normal DimOS Blueprint containing PimSimG1
- the shared-memory adapter address
- visualization configuration
        |
        v
DimOS combines:
PimSimG1 + ControlCoordinator + navigation + visualization
        |
        v
The normal ModuleCoordinator deploys every module into
the normal DimOS forkserver worker pool

There is no second blueprint system, worker manager, transport system, control stack, or application runtime inside PimSim.

Follow The Code

1. DimOS defines SimulationRequest, SimulationBinding, and SimulationProvider:

   dimos/simulation/providers.py

2. The G1 blueprint loads the requested provider and receives its binding:

   unitree_g1_groot_wbc.py

3. The binding configures the existing ControlCoordinator hardware adapter:

   unitree_g1_groot_wbc.py

4. The simulated backend is composed with the ordinary DimOS control, navigation, and visualization modules:

   unitree_g1_groot_wbc.py

5. The external package builds the PimSimG1 blueprint:

   pimsim_dimos/provider.py

6. PimSim registers that provider through standard Python package entry points:

   integrations/dimos/pyproject.toml

After installation, PimSimG1 is constructed and run exactly like an in-repository DimOS module.

Who Owns Communication?

High-rate control:
DimOS policy
  -> ControlCoordinator
  -> simulated hardware adapter
  -> shared memory
  -> PimSim physics

Observable sensor state:
PimSim camera/lidar/odom/TF
  -> typed DimOS Out[T]
  -> blueprint-selected transport, normally Zenoh
  -> mapping/perception/Rerun

Evaluator controls:
pytest SceneControl
  -> DimOS RPC
  -> reset/respawn/add obstacle/query evaluator truth

PimSim does not independently choose or run the application transport. Its modules declare normal typed DimOS ports. The DimOS blueprint and ModuleCoordinator materialize and connect those transports.

The normal deployment path remains:

• ModuleCoordinator.build()
• WorkerManager.deploy_parallel()
• Forkserver worker creation
• Worker constructs the module

Reviewable PR Split

The original integration branch had roughly 90 changed files. It has now been reduced and split as follows:

• refactor(sim): move shared-memory control ABI to hardware #3372: shared-memory simulated-hardware ABI
• fix(config): preserve runtime objects in blueprint configuration #3373: blueprint runtime-value handling
• fix(zenoh): route local sessions through the coordinator router #3374: local Zenoh router behavior
• feat(sim): add external simulation provider contract #3375: external simulation-provider contract and G1 blueprint integration

PRs #3372, #3373, and #3374 are independent and can be reviewed in parallel.

After those three merge, main will be merged normally into #3375. Its diff will then reduce to the intended four provider-facing files:

• dimos/core/global_config.py
• dimos/simulation/providers.py
• dimos/simulation/test_providers.py
• dimos/robot/unitree/g1/blueprints/basic/unitree_g1_groot_wbc.py

The architectural decision requested by this RFC remains unchanged: continue extending this working PimSim integration, or use the Sim2 contracts as the destination for a broader runtime consolidation.

Detailed integration documentation: docs/dimos-integration.md

[Nabla7]

Option C: Move PimSim into the DimOS monorepo as the working Sim2 implementation

Discussion has identified a third option that combines the delivery advantages of PimSim with monorepo ownership.

Move the existing working PimSim implementation into dimos/sim2, preserving its behavior and internal dependency boundary. Do not merge or rewrite around the old feat/sim2-runtime implementation first.

What moves

pimsim/src/pimsim/*
    -> dimos/sim2/*

pimsim/integrations/dimos/src/pimsim_dimos/*
    -> dimos/sim2/modules/* and dimos/sim2/provider.py

The tracked source, tests, documentation, and required assets move. Standalone repository infrastructure such as the nested lockfile, build configuration, caches, and distributions does not.

Boundary retained inside the monorepo

dimos.sim2 core/backends/scenes/sensors/rendering
    must not depend on DimOS modules, messages, workers, or transports

dimos.sim2 modules/provider/scene_control
    may depend on DimOS runtime APIs

This preserves the current passive simulation-core design while giving DimOS one repository, one dependency graph, one CI system, and direct ownership of the complete simulation stack.

What happens initially

• Existing G1, Go2, and xArm behavior is preserved.
• Existing robot-specific DimOS modules remain during the move.
• ControlCoordinator, shared memory, typed streams, Zenoh, RPC, and DimSim-style E2E tests remain unchanged.
• The external Python entry-point installation boundary disappears.
• The current PimSim repository becomes read-only after parity is established.

What is adopted from the old Sim2 experiment

The old branch becomes a design reference rather than a code dependency.

Selected concepts are introduced incrementally when their requirement becomes concrete:

• a minimal generic physics SimModule to remove duplicated lifecycle code;
• canonical WorldManifest and WorldStateFrame streams;
• profile-driven robot bindings;
• coherent versioned shared-memory frames;
• lockstep execution through ControlCoordinator;
• sensor synchronization barriers for external renderers and deterministic evaluation.

The old runtime, backend, adapters, blueprints, registry, control socket, and duplicate model types are not merged wholesale.

Why this differs from the existing options

	Option A	Option B	Option C
Repository	Separate PimSim repository	DimOS monorepo	DimOS monorepo
Starting implementation	Working PimSim	Old Sim2 contracts	Working PimSim
Migration strategy	Incremental external integration	Architecture-first port	Mechanical move, then incremental consolidation
Near-term feature work	Continues	Slows during runtime work	Continues after parity move
External provider packaging	Required	No	No
Old Sim2 code	Reference only	Starting architecture	Requirements catalogue only

Effect on the current PR sequence

• refactor(sim): move shared-memory control ABI to hardware #3372, fix(config): preserve runtime objects in blueprint configuration #3373, and fix(zenoh): route local sessions through the coordinator router #3374 remain valid independent prerequisites.
• feat(sim): add external simulation provider contract #3375 should be held or revised because its external package-entry-point mechanism would no longer be required for the built-in Sim2 implementation.
• The provider request/binding abstraction may remain as an internal blueprint boundary and for future third-party simulators.

Required migration gate

Before retiring the external repository, the monorepo implementation must demonstrate unchanged behavior for:

• G1 policy execution through ControlCoordinator;
• Go2 navigation, mapping, lidar, and camera streams;
• xArm planning, grasp, retained lift, place, and camera streams;
• scene-package loading;
• deterministic reset and evaluator RPC;
• the existing DimSim-style E2E tests.

Updated decision request

Decision:
- A: Continue external PimSim
- B: Reintroduce Sim2 architecture first
- C: Move working PimSim into dimos/sim2, then consolidate incrementally

Reason:
Blocking concern:
Required correction:
Required validation:

[spomichter]

You mention "What PimSim actually is" but dont lay out anywhere the actual features of this sim versus current sim and how its relevant and needed for roadmap. Pls share in your own words why worth allocating engineering resources for this. One thing youve mentioned is prgammatic scene generation, explain why we cant just use existing mujoco