feat: message-input point-lio rust module — WIP scaffold + raw sensor replay (#2821) - #3332
Draft
kumarganduri wants to merge 2 commits into
Draft
feat: message-input point-lio rust module — WIP scaffold + raw sensor replay (#2821)#3332kumarganduri wants to merge 2 commits into
kumarganduri wants to merge 2 commits into
Conversation
…alOS#2821) Decodes Livox point/IMU UDP payloads straight out of a pcap and writes the messages the live mid360_native driver would publish into a replayable memory2 db (lidar @ 10 Hz frames, imu @ 200 Hz) — no virtual NIC, no sudo, no SDK, deterministic output. Conventions mirror the driver exactly (mm/1000, reflectivity/255, frame ts = first packet ts, accel g -> m/s^2 by 9.80665, identity orientation with -1 covariance flag); the packet layout is from the public Livox-SDK2 header. Validated on mid360_shake_stairs.pcap (120 s): 9.98 Hz frames at ~20k points/frame (spec: 200k pts/s), 200.0 Hz IMU, zero per-stream timestamp regressions, indoor geometry confirmed visually. First step for the point-lio rust module: its lidar+imu message inputs can now be fed from replay instead of a network sensor.
…dimensionalOS#2821) Option 1 architecture from the issue, wired end to end: - PointLioRust (rust, dimos-module): raw_lidar (PointCloud2) + imu (Imu) in, lidar + odometry out. Output ports keep the existing PointLio names so downstream consumers wire identically. The estimator is an explicit slot publishing identity odometry until the implementation direction (FFI wrap of the GPL core vs a port in a separate repo) is settled on the issue. - RawSensorReplay: publishes a recorded raw db (pcap_to_raw_db output) as raw_lidar/imu topics on the shared replay clock — the message-boundary counterpart of VirtualMid360, with no virtual NIC, sudo, or SDK. - pointlio-rust-replay blueprint: the whole sensor->SLAM chain from a file. Verified live on the mid360_shake_stairs recording: raw_lidar 10 Hz, imu 200 Hz, lidar/odometry out 1:1 with input frames, zero drops over 30 s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
Point-LIO is welded to the Livox SDK:
pointlio_nativeconsumes raw networkpackets, so it only works with one sensor family, and testing it without
hardware requires impersonating a lidar on the network (
VirtualMid360:sudo, loopback IP aliases, multicast routes). The issue asks for point cloud
Solution (so far)
Everything up to the estimator, wired end to end and testable from a file:
pcap_to_raw_db.py(livox/scripts): offline converter, Mid-360 pcap ->memory2 db with
lidar(PointCloud2, 10 Hz frames) +imu(Imu, 200 Hz)streams. Parses the Livox UDP payloads directly (layout from the public
Livox-SDK2 header; conventions byte-for-byte with the live driver: mm->m,
reflectivity/255, frame ts = first packet ts, accel g->m/s^2 by 9.80665).
No SDK, no sudo, no network. Validated on the
mid360_shake_stairscapture: 9.98 Hz frames at ~20k pts (spec: 200k pts/s), 200.0 Hz IMU, zero
per-stream timestamp regressions, geometry verified visually.
RawSensorReplay(livox/replay.py): publishes such a db asraw_lidar/imutopics on the shared replay clock — the message-boundarycounterpart of
VirtualMid360.PointLioRust(pointlio/rust): dimos-module crate,raw_lidar(PointCloud2) +
imu(Imu) in ->lidar+odometryout. Output portskeep the existing PointLio names so downstream consumers wire identically.
The estimator is an explicit slot (currently: passthrough clouds + identity
odometry) so the wiring is provable before the math exists.
pointlio-rust-replayblueprint: the whole chain from a file.Verified live over zenoh:
raw_lidar10 Hz +imu200 Hz in,lidar/odometryout 1:1 with input frames, zero drops over 30 s of replay.Remaining (in scope for this PR or follow-ups, per maintainer preference)
existing core (
dimos-module-pointlio) is GPL-2.0, so a faithful Rust portcan't live in this repo — options are an FFI wrap of the C++ core behind
this module's message interface, or a port in a separate GPL repo consumed
as a binary, or a clean-room implementation.
the minimal frame-id pair), nix flake for the crate, trajectory-level
validation harness against the C++ implementation.
Base-branch note
This branch is based on
29f35555(mid-July) rather than current main:current main cannot start any blueprint on macOS (first coordinator->worker
set_transportRPC times out — details and bisect in #3331). Will rebaseonto main once that is resolved; the diff here touches nothing near the RPC
layer. The conflict GitHub reports is the auto-generated
all_blueprints.pyregistry; it regenerates cleanly on rebase(
pytest dimos/robot/test_all_blueprints_generation.py).How to Test
Expect
lidar+odometrypublishing at 10 Hz (identity poses; the rerunview shows the replayed clouds). Replay-tested only; no hardware involved.
AI assistance
Done with the help of Claude Code (Fable 5)
Checklist
registry generation test, live replay verification) for the files I
changed.