Flexiv Trainer

Flexiv Trainer is an all-in-one, local-first software platform for training physical AI skills on Flexiv robots. It takes you through the full imitation-learning workflow — collecting teleoperation demonstrations, saving them as LeRobot episodes, merging episodes into training datasets, and launching policy training runs — from a single web UI on your own machine, with no cloud services required.

The steps to set up and use this software can be summarized as:

1. Install the software.
2. Start the server.
3. Set up robots and cameras in the UI.
4. Teleoperate and record demonstration episodes.
5. Review and merge episodes into one training dataset.
6. Choose a policy and start training.

Supported Platforms

OS	CPU
Ubuntu 22.04+	x86-64, aarch64

Software Requirements

1. Python 3.12 or 3.14. Orbbec cameras require 3.12; on 3.14 only RealSense cameras are supported.
2. GPU acceleration (optional but recommended).
3. System config for realtime scheduling (see below).

Grant realtime privileges to non-root users (Linux only)

This is required to run the teleoperation module:

echo "${USER}    -   rtprio    99" | sudo tee -a /etc/security/limits.conf
echo "${USER}    -   nice     -20" | sudo tee -a /etc/security/limits.conf
echo "${USER} soft memlock unlimited" | sudo tee -a /etc/security/limits.conf
echo "${USER} hard memlock unlimited" | sudo tee -a /etc/security/limits.conf

Important

Restart your device for the changes to take effect.

Hardware Requirements

1. A single- or dual-arm teleoperation setup eligible for high-transparency teleoperation.
2. Supported cameras connected to your computer: 1 egocentric and 2 in-hand (left and right wrist), with the two in-hand cameras mounted on the follower robots.

Supported robots

• Flexiv Rizon series

Supported cameras

• Intel RealSense cameras
• Orbbec cameras (Gemini 305/330 series and other OrbbecSDK v2 devices)

Both vendors can be used at the same time; each camera location is assigned a device serial independently, so a setup may mix RealSense and Orbbec cameras.

Orbbec support needs the pyorbbecsdk2 SDK, installed separately with --no-deps:

pip install --no-deps 'pyorbbecsdk2>=2.1'

The --no-deps is required, not optional. pyorbbecsdk2 declares hard pins for its bundled example scripts (av==12.3.0, open3d, pygame, …) that conflict with LeRobot's av>=15,<16; a plain install fails to resolve. The binding itself does not import those packages and works against LeRobot's own av.

The wheel is published for CPython 3.8–3.13 on Linux (x64/ARM64), Windows x64 and Apple Silicon — there is no 3.14 wheel and no source distribution, so Orbbec cameras need a Python 3.12 environment. When the SDK is absent the app still runs, reporting Orbbec as unavailable while RealSense cameras keep working.

On Linux, install the Orbbec udev rules once so cameras are reachable without root. Skipping this is the usual cause of a camera that appears in lsusb but never connects in the app:

sudo ./scripts/install-orbbec-udev-rules.sh

Replug the camera afterwards so the new permissions apply.

The rules cover both the raw USB node and the V4L2 /dev/videoN nodes. Without the V4L2 rule the camera is still discovered but fails to open with uvc_open failed ... Return Code: -6, because those nodes default to root:video and most accounts are not in the video group.

Use a USB 3.0 port; the Gemini 305 falls back to reduced frame rates on USB 2.0.

Install

1. Create a Python 3.12+ virtual environment:

   python3 -m venv .venv

2. Activate the virtual environment:

   source .venv/bin/activate

3. Upgrade pip and install the package:

   pip install --upgrade pip
   pip install -e .

To also install development dependencies (for running tests and linting):

pip install -e ".[dev]"

PyTorch on NVIDIA Jetson

The standard PyPI PyTorch package for aarch64 does not support CUDA (GPU acceleration) and will fall back to CPU-only execution. To enable GPU acceleration on NVIDIA Jetson devices, you must install the Jetson-compatible PyTorch wheels and their corresponding CUDA dependencies.

For JetPack 7 / CUDA 13, you can replace the default CPU PyTorch after installation:

pip install --no-deps --force-reinstall \
  --index-url https://pypi.jetson-ai-lab.io/sbsa/cu130 \
  torch==2.10.0 torchvision==0.25.0

Refer to NVIDIA's official Jetson documentation for more details.

Start Flexiv Trainer

Start the backend server from the same environment where you installed the package:

source .venv/bin/activate
flexiv-trainer-server

After startup, the terminal prints the URL for the web UI. By default this is:

http://127.0.0.1:8000/

Open that URL in your browser to use the UI.

If the port is still occupied after closing a previous server session, clear leftover processes and start again:

source .venv/bin/activate
scripts/cleanup-server.sh
flexiv-trainer-server

First-Time Setup In The UI

When the UI opens, you will start on the home page.

1. Enter the serial numbers for the two leader robots.
2. Enter the serial numbers for the two follower robots.
3. In the System Status tile, connect:
   • Teleop Service
   • Robot Data Service
   • Cameras
4. Confirm the system shows healthy status before moving on.

The home page also shows the storage locations used by the app for episodes, merged datasets, and training outputs.

Collect Demonstrations In Data Collection

Open Data Collection from the navigation bar.

On this page you can verify robot and camera health, view the camera feeds, monitor telemetry, run teleoperation, and record episodes.

Recommended workflow

1. Check System Status and make sure teleoperation, robot data, and cameras are all connected.
2. Confirm that the egocentric and wrist camera feeds are updating at around 30 FPS.
3. If needed, use Home All Robots before starting teleoperation.
4. In Teleoperation Control, click Start to begin teleoperation.
5. In Episode Recording, choose which entries you want to record.
   • The default selection includes all available camera observations, robot states, and robot actions.
   • Use Select All or Deselect All if needed.
6. Click the recording Start button when you are ready to capture a demonstration.
7. Perform the task through teleoperation.
8. Click the recording Stop button when one episode is complete.
9. Click Save Episode to keep the recording, or Discard Episode to throw it away.

Saved episodes are written to the episode storage directory shown on the home page. By default, episodes are stored under:

.local/episodes/

Load And Merge Episodes In Data Processing

Open Data Processing from the navigation bar.

Step 1: Load episodes

1. Click the add button in Load Episodes.
2. In the episode browser, select one or more saved episode datasets.
3. Use Select All or Deselect All if you want to quickly toggle the full list.
4. Click Load.

Step 2: Review selected episodes

1. Click an episode in the list to review it.
2. Use the checkboxes to choose which episodes should be merged.
3. Use Select All or Deselect All to toggle the current list.
4. Click Merge Selected Episodes.

Step 3: Review the merged dataset

The merging will take some time depending on the actual dataset size. When done:

1. Review the merged dataset.
2. Click Next if the merged dataset looks correct.

Merged datasets are stored under:

.local/datasets/

Convert TCP Actions To B-Spline Training Targets

The experimental B-spline converter creates a new LeRobot dataset while leaving the source dataset unchanged. Observations and videos are copied without re-encoding. The copied dataset's discrete TCP action is replaced by one flattened B-spline parameter segment per frame.

When B-Spline Diffusion is selected in Policy Training, the Start button runs this conversion automatically in the background and launches training only after the converted dataset validates successfully. Other policy types continue to train directly from the selected dataset. The command below remains available when a reusable converted dataset is wanted ahead of time.

With the default cubic configuration, one arm is represented by XYZ plus rotation-6D. The parameter matrix has shape:

(chunk_size + 2 * degree, 1 knot + 9 Cartesian controls)
= (16, 10)

Run the converter on a merged dataset:

flexivtrainer-convert-bspline \
  .local/datasets/my_task \
  .local/datasets/my_task_bspline

TCP pose axes are detected from the action names. To select arms explicitly:

flexivtrainer-convert-bspline \
  .local/datasets/my_task \
  .local/datasets/my_task_bspline \
  --side left_arm \
  --side right_arm

Useful fitting options include:

--degree 3
--chunk-size 10
--stride 1
--max-error 0.002
--smoothing 1e-12

The converted dataset stores its representation and per-episode reconstruction report in meta/bspline.json. With the default settings, every episode must contain at least 12 frames.

Important

This command prepares trainable spline-parameter targets. A B-spline-aware policy/rollout adapter is still required to reshape, validate, align, and evaluate predicted parameters. Do not send a checkpoint trained on this converted action feature through the existing Cartesian waypoint executor.

Start Training

Open Policy Training from the navigation bar, then work through the steps.

Step 1: Load the training dataset

1. Click the add button to browse merged datasets.
2. Select the dataset you want to train on.
3. Click Next.

Step 2: Review the dataset

1. Review the dataset preview (whole dataset or a single episode).
2. Click Next.

Step 3: Choose a policy

1. Select a training policy.
2. The training output directory is generated automatically from the dataset name, policy, and a timestamp, and is shown under Training Output Directory.
3. Click Next.

Step 4: Run training

1. Under Computation Device, leave the device on auto or pick a specific device. Use the reload button to re-evaluate available devices.
2. Click the start button in Training Control.
3. Watch progress in the progress bar and the terminal log pane. Use Pause/Resume while a run is live.

Training outputs are written to the output directory shown in Step 3. By default these live under:

.local/training/

Typical End-To-End Session

For a normal session, the order is:

1. Start flexiv-trainer-server.
2. Open the web UI.
3. Configure robot serial numbers on Home.
4. Connect teleop service, robot data service, and cameras.
5. Open Data Collection.
6. Start teleoperation and record one or more demonstration episodes.
7. Save the episodes you want to keep.
8. Open Data Processing.
9. Load and merge the saved episodes into one training dataset.
10. Open Policy Training.
11. Select the merged dataset and choose a policy.
12. Pick a computation device and start training.

Open UI From Another Device

If you want to use the UI from another device on the same LAN, set a public base URL before starting the server:

export FLEXIV_TRAINER_PUBLIC_BASE_URL=http://<backend-host-ip>:8000
flexiv-trainer-server

Then open that public URL in a browser on the other device.

API Documentation

The backend API doc can be accessed at:

http://127.0.0.1:8000/docs