Skip to content

feat: message-input point-lio rust module — WIP scaffold + raw sensor replay (#2821) - #3332

Draft
kumarganduri wants to merge 2 commits into
dimensionalOS:mainfrom
kumarganduri:kumar/pointlio-rust-module
Draft

feat: message-input point-lio rust module — WIP scaffold + raw sensor replay (#2821)#3332
kumarganduri wants to merge 2 commits into
dimensionalOS:mainfrom
kumarganduri:kumar/pointlio-rust-module

Conversation

@kumarganduri

@kumarganduri kumarganduri commented Aug 2, 2026

Copy link
Copy Markdown

Contribution path

Draft / work in progress. This PR implements option 1 from the issue
(message inputs), as proposed in
#2821 (comment)
happy to change course if maintainers prefer another direction. The
estimator itself is intentionally not implemented yet (see "Remaining" section).

Problem

Point-LIO is welded to the Livox SDK: pointlio_native consumes raw network
packets, so it only works with one sensor family, and testing it without
hardware requires impersonating a lidar on the network (VirtualMid360:
sudo, loopback IP aliases, multicast routes). The issue asks for point cloud

  • IMU message inputs instead.

Solution (so far)

Everything up to the estimator, wired end to end and testable from a file:

  • pcap_to_raw_db.py (livox/scripts): offline converter, Mid-360 pcap ->
    memory2 db with lidar (PointCloud2, 10 Hz frames) + imu (Imu, 200 Hz)
    streams. Parses the Livox UDP payloads directly (layout from the public
    Livox-SDK2 header; conventions byte-for-byte with the live driver: mm->m,
    reflectivity/255, frame ts = first packet ts, accel g->m/s^2 by 9.80665).
    No SDK, no sudo, no network. Validated on the mid360_shake_stairs
    capture: 9.98 Hz frames at ~20k pts (spec: 200k pts/s), 200.0 Hz IMU, zero
    per-stream timestamp regressions, geometry verified visually.
  • RawSensorReplay (livox/replay.py): publishes such a db as
    raw_lidar/imu topics on the shared replay clock — the message-boundary
    counterpart of VirtualMid360.
  • PointLioRust (pointlio/rust): dimos-module crate, raw_lidar
    (PointCloud2) + imu (Imu) in -> lidar + odometry out. Output ports
    keep the existing PointLio names so downstream consumers wire identically.
    The estimator is an explicit slot (currently: passthrough clouds + identity
    odometry) so the wiring is provable before the math exists.
  • pointlio-rust-replay blueprint: the whole chain from a file.

Verified live over zenoh: raw_lidar 10 Hz + imu 200 Hz in, lidar/
odometry out 1:1 with input frames, zero drops over 30 s of replay.

Remaining (in scope for this PR or follow-ups, per maintainer preference)

  • The estimator. Blocked on direction (see the issue comment): the
    existing core (dimos-module-pointlio) is GPL-2.0, so a faithful Rust port
    can't live in this repo — options are an FFI wrap of the C++ core behind
    this module's message interface, or a port in a separate GPL repo consumed
    as a binary, or a clean-room implementation.
  • Undistortion/TF details, tuning-parameter plumbing (the current config is
    the minimal frame-id pair), nix flake for the crate, trajectory-level
    validation harness against the C++ implementation.

Base-branch note

This branch is based on 29f35555 (mid-July) rather than current main:
current main cannot start any blueprint on macOS (first coordinator->worker
set_transport RPC times out — details and bisect in #3331). Will rebase
onto main once that is resolved; the diff here touches nothing near the RPC
layer. The conflict GitHub reports is the auto-generated
all_blueprints.py registry; it regenerates cleanly on rebase
(pytest dimos/robot/test_all_blueprints_generation.py).

How to Test

# one-time: fetch pcap + build the replay db (~3 min)
uv run python -m dimos.hardware.sensors.lidar.livox.scripts.pcap_to_raw_db \
    --pcap "$(python -c "from dimos.utils.data import get_data; print(get_data('mid360_shake_stairs/mid360_shake_stairs.pcap'))")"

# build + run the chain
(cd dimos/hardware/sensors/lidar/pointlio/rust && cargo build --release)
uv run dimos run pointlio-rust-replay

Expect lidar + odometry publishing at 10 Hz (identity poses; the rerun
view shows the replayed clouds). Replay-tested only; no hardware involved.

AI assistance

Done with the help of Claude Code (Fable 5)

Checklist

  • This PR is scoped to one issue or clearly stated problem.
  • I ran the relevant checks (pre-commit, cargo clippy/fmt, blueprint
    registry generation test, live replay verification) for the files I
    changed.
  • I have reviewed and understood every line in this PR.
  • I disclosed AI assistance above.
  • I have read and approved the CLA.

…alOS#2821)

Decodes Livox point/IMU UDP payloads straight out of a pcap and writes the
messages the live mid360_native driver would publish into a replayable
memory2 db (lidar @ 10 Hz frames, imu @ 200 Hz) — no virtual NIC, no sudo,
no SDK, deterministic output.

Conventions mirror the driver exactly (mm/1000, reflectivity/255, frame ts =
first packet ts, accel g -> m/s^2 by 9.80665, identity orientation with -1
covariance flag); the packet layout is from the public Livox-SDK2 header.

Validated on mid360_shake_stairs.pcap (120 s): 9.98 Hz frames at ~20k
points/frame (spec: 200k pts/s), 200.0 Hz IMU, zero per-stream timestamp
regressions, indoor geometry confirmed visually.

First step for the point-lio rust module: its lidar+imu message inputs can
now be fed from replay instead of a network sensor.
…dimensionalOS#2821)

Option 1 architecture from the issue, wired end to end:

- PointLioRust (rust, dimos-module): raw_lidar (PointCloud2) + imu (Imu) in,
  lidar + odometry out. Output ports keep the existing PointLio names so
  downstream consumers wire identically. The estimator is an explicit slot
  publishing identity odometry until the implementation direction (FFI wrap
  of the GPL core vs a port in a separate repo) is settled on the issue.
- RawSensorReplay: publishes a recorded raw db (pcap_to_raw_db output) as
  raw_lidar/imu topics on the shared replay clock — the message-boundary
  counterpart of VirtualMid360, with no virtual NIC, sudo, or SDK.
- pointlio-rust-replay blueprint: the whole sensor->SLAM chain from a file.

Verified live on the mid360_shake_stairs recording: raw_lidar 10 Hz, imu
200 Hz, lidar/odometry out 1:1 with input frames, zero drops over 30 s.
@github-actions github-actions Bot added the first-time-contributor PR opened by an author who had not previously committed to this repository label Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

first-time-contributor PR opened by an author who had not previously committed to this repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant