Introduction

This book teaches you to drive Ubicoders virtual robots from code, and then serves as the reference you come back to.

What the SDK is

vrobots_sdk is the Rust SDK for controlling Ubicoders virtual robots running in a Unity simulator. It talks to the simulator over two transports, zenoh for state, commands and services and iceoryx2 for camera frames, with FlatBuffers on the wire in both directions.

The Rust crate is the single implementation. The C++ and Python SDKs are thin bindings over it, so the three surfaces cannot drift: the same lifecycle, the same snapshots, the same timestamps, and the same stable error codes in Appendix C.

What you need

Installing the SDK and the simulator covers them in order.

The one idea to internalise first

This SDK is STM32-shaped, not Arduino-shaped. main() does setup and then owns a plain loop. There is no base class, no runner, no setup() and update() callbacks, and the SDK never calls your code. If you have used the older Python client, this is the single largest difference, and every page in the book assumes it.

The whole of main in the first example shows the shape.

From examples/rust/src/bin/ex01_hello_states.rs:

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states(); // immutable latest snapshot, never torn
        let [x, y, z] = s.kin.lin_pos;
        println!("State t={:.3} pos=({x:.3},{y:.2},{z:.2})", s.elapsed);
        robot.rate(HZ); // drift-compensated pacing, Hz
    }
}

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();  // header and library must be the same release
        vrsdk::set_log_callback(on_log);

        vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
        robot.connect();  // blocks until the first state snapshot arrives
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();  // latest snapshot, never torn
            const double* p = s.kin().lin_pos;
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)\n", s.elapsed, p[0], p[1], p[2]);
            robot.rate(HZ);  // drift-compensated pacing, Hz
        }
    } catch (const vrsdk::Error& e) {
        // `code()` is the SDK's stable number -- the same one Python's
        // VrError.code and the CLI's `error [N]` report.
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
    mr.connect()

    # ===== loop =====
    while True:
        s = mr.states  # immutable latest snapshot, never torn
        x, y, z = s.kin.lin_pos
        print(f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f})")
        mr.rate(HZ)  # drift-compensated pacing, Hz

The program prints one line per iteration at the rate robot.rate paces it to, until you stop it with Ctrl+C. The shape of a program explains why the loop belongs to you rather than to the SDK.

How the book is organised

Chapters 3 to 6 are independent of each other. Read chapter 2 before any of them, because they all lean on the five rules it sets out.

Reading paths

Examples

Thirty-three complete programs live under examples/rust/src/bin/. Each is a real fn main rather than a snippet, takes no command-line arguments (settings are constants at the top of the file), and is mirrored one for one in Python and C++ under examples/python/ and examples/cpp/, with the same numbers and the same behaviour.

Run one by its bin name, or by the equivalent name in the language you are using:

cargo run -p vrobots-examples --bin ex01_hello_states
./target/cpp-build/ex01_hello_states
python examples/python/ex01_hello_states.py

The C++ line assumes the build in Installing the SDK and the simulator; on Windows the binary is target\cpp-build\Release\ex01_hello_states.exe.

Every code block in this book is copied from one of those files or from a signature in the SDK source, so anything you read here compiles as written.

Versions

This book documents SDK 0.1.4 against simulator v3.0.0. The IPC pins that build speaks are flatbuffers 25.12.19, iceoryx2 0.9.3 and zenoh 1.9.0, and vrobots --version prints the set your build actually carries. The pins are exact on purpose: Versions and pins explains what a caret pin one patch off does, and why it looks like the simulator has stopped publishing.

Next: Getting started

See also: Five rules that explain everything, Appendix D: Glossary

Getting started

What the simulator is, what it is not, and what you will have working by the end of this chapter.

What the simulator is

The Ubicoders virtual robots simulator is a Unity application that runs a rigid-body physics world containing one or more robots. Each robot publishes a full state snapshot at 25 Hz, accepts commands, answers a small set of request/response services, and can stream camera frames. It is a target for control code: you write the controller, the simulator provides the plant.

vrobots_sdk is the client side of that conversation. One program behaves like one embedded system bound to one robot: construct, connect, then run a plain control loop.

What it is not

• It is not a robotics framework. There is no scheduler, no node graph, no message passing between your components. The SDK gives you a handle and gets out of the way.
• It does not stabilise anything for you. Sending pulse widths to a multirotor drives the thrust curves directly. There is no attitude hold and no rate damping between your numbers and the rotors.
• It is not a physical robot. Truth blocks in the state snapshot are simulator-exact values no real vehicle could know. They exist so you can measure your estimator against them, not so you can fly on them.
• It does not manage your robot's lifetime. A robot you create outlives the process that created it. See Hello service.

One core, three languages

The Rust crate is the single implementation. The C++ SDK is a header-only RAII wrapper over a C ABI, and the Python SDK is a PyO3 binding, both over that same Rust core. The bindings add sugar, not behaviour, so lifecycle, snapshots and timestamps behave identically by construction and the three surfaces cannot drift.

This chapter is written in Rust. Every example under examples/rust/src/bin/ is mirrored one for one in examples/python/ and examples/cpp/, printing the same numbers.

How your program reaches the robot

Two transports carry the traffic, and they are not interchangeable.

flowchart LR
  P[Your program]
  Z([zenoh])
  I([iceoryx2])
  S[Unity simulator]

  P -->|commands, service requests| Z
  Z -->|commands, service requests| S
  S -->|state, service replies| Z
  Z -->|state, service replies| P
  S -->|camera frames| I
  I -->|camera frames| P

zenoh carries states, commands, setpoints and services, and it crosses a network: the simulator can run on another machine if you point the SDK at a router. iceoryx2 carries camera frames through shared memory, which makes it fast and makes it same-host only. A remote simulator will therefore answer states() and refuse to deliver a single frame.

The wire format on both is FlatBuffers, and the version pins are exact on purpose. A mismatch does not raise an error, it delivers nothing, which reads as "the simulator is not publishing". Installing the SDK and the simulator shows how to print the pins this build speaks.

Note. The transports and the topic names they carry are the subject of Two transports, one simulator and The topic namespace. This chapter uses them without explaining them further.

What you will have by the end

Working through the eight pages of this chapter, in order, leaves you with:

• The SDK installed and the simulator running, on Windows, Ubuntu or WSL.
• Proof, from the vrobots command line tool, that the simulator is publishing and under which system id.
• A program that reads a multirotor's position at a rate you choose.
• A multirotor climbing under pulse widths you sent.
• A truck driving a gentle left arc.
• A camera mounted, frames read, and the camera unmounted again.
• A robot created from code and deleted from code.

Each of those is one of the first five example programs, run unmodified. Nothing in this chapter asks you to write a program from scratch.

Next: Installing the SDK and the simulator

See also: The shape of a program, Five rules that explain everything, When nothing happens

Installing the SDK and the simulator

Install the SDK with one pip command, then get a simulator running on Windows, Ubuntu or WSL.

pip install ubicoders-vrsdk

What pip gives you

That command is the whole SDK install: the wheel carries the compiled Rust core, so no Rust toolchain, no flatc, no protoc and no repository clone is involved. It puts two things on your machine: vrsdk, the package every Python example in this book imports, and vrobots, the command line tool of The vrobots command, which is the same program as the Rust build's rather than a second implementation.

numpy arrives with the wheel, because frame.image hands back an ndarray. opencv-python does not, and it is what the camera examples want in order to open a window:

pip install "ubicoders-vrsdk[examples]"

Without it, Hello image prints metadata instead of opening a window and Saving a frame writes a PPM instead of a PNG. The one page that needs OpenCV outright is Showing frames in a window, in every language.

Gotcha. No source distribution is published, deliberately: a source build needs the Rust toolchain and a private submodule. On a platform with no wheel, pip therefore stops with No matching distribution found for ubicoders-vrsdk rather than starting a compile that cannot finish.

Getting the example programs

The wheel ships the library, not the example programs the pages of this book run. Those live in the repository, and the Python ones need nothing from it but themselves:

git clone https://github.com/ubicoders0/vrobots_sdk
python vrobots_sdk/examples/python/ex01_hello_states.py

Plain git clone, with no --recurse-submodules: the submodule only matters if you build the Rust or C++ surfaces. Every Python example imports vrsdk and nothing else from the tree, so one file copied out of it runs just as well on its own.

Building the Rust and C++ surfaces

Skip this section unless you are working in Rust or C++. Neither surface is published as a package, so both start from a clone that includes the vrobots_msgs submodule. That submodule ships the generated FlatBuffers code, which is why no flatc is needed, and it is a private repository, so the clone below needs access to it.

git clone --recurse-submodules https://github.com/ubicoders0/vrobots_sdk
cd vrobots_sdk
cargo build --workspace

Rust 1.89 or newer is required, since the crate is edition 2024; install it from https://rustup.rs/. If you already cloned without the submodule, the build fails and says so. Repair it with:

git submodule update --init --recursive

The C++ examples are a CMake project over the header-only wrapper. Building the C ABI crate generates the C header, so that is the only further prerequisite:

cargo build -p vrobots-sdk-capi --release
cmake -S examples/cpp -B target/cpp-build -DCMAKE_BUILD_TYPE=Release
cmake --build target/cpp-build --config Release

That puts one binary per example under target/cpp-build/, which is the path the sh block on each page names. On Windows the binaries land in target\cpp-build\Release\ and carry an .exe suffix, and the DLL is copied beside each one; on Linux the build rpath points at the cargo target directory, so no LD_LIBRARY_PATH is needed.

Getting the simulator

Prebuilt simulator packages are at https://www.ubicoders.com/virtualrobots. Windows 11 and Ubuntu 22.04 or newer are supported; macOS is not supported yet.

Windows

Run virtual_robots.exe.

Ubuntu

The build needs xdg-utils present and its own executable bit set.

sudo apt install xdg-utils -y
sudo chmod +x ./virtual_robots.x86_64
./virtual_robots.x86_64

Double-clicking virtual_robots.x86_64 in a file manager works as well.

WSL

WSL needs a graphics bridge before Unity will render. Install the Mesa and Vulkan packages and force the D3D12 gallium driver, which routes rendering to the Windows GPU rather than to the CPU rasteriser.

Save this as install_wsl_graphics.bash:

#!/bin/bash
# 1. Install necessary drivers and diagnostic tools
sudo apt-get update
sudo apt install xdg-utils -y
sudo apt install mesa-utils mesa-vulkan-drivers vulkan-tools -y

# 2. Add GPU bridge variables to .bashrc for persistence
# We use GALLIUM_DRIVER to force the D3D12 bridge (Windows GPU)
# and VK_ICD_FILENAMES to ensure Vulkan doesn't default to the CPU (llvmpipe)
if ! grep -q "GALLIUM_DRIVER=d3d12" ~/.bashrc; then
  echo 'export GALLIUM_DRIVER=d3d12' >> ~/.bashrc
  echo 'export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA' >> ~/.bashrc
fi

# 3. Reload environment
source ~/.bashrc

# 4. Verify the setup
echo "--- Checking OpenGL  ---"
glxinfo -B | grep -E "Device|Accelerated"

echo "--- Checking Vulkan  ---"
vulkaninfo | grep "Vulkan Instance Version"
vkcube

Run it, then reload your shell so the exported variables apply:

bash install_wsl_graphics.bash && source ~/.bashrc

Check that Vulkan came up on the GPU. vkcube should open a spinning cube window; if it does not, the simulator will not render either.

vulkaninfo | grep "Vulkan Instance Version"
vkcube

Launch the simulator with Vulkan forced. The LD_LIBRARY_PATH edit removes /opt/zenoh-c/lib from the loader path, so the simulator loads its own vendored zenoh rather than a system copy.

#!/bin/bash
sudo chmod +x ./virtual_robots.x86_64
export LD_LIBRARY_PATH=$(echo "$LD_LIBRARY_PATH" | sed 's|:/opt/zenoh-c/lib||; s|/opt/zenoh-c/lib:||; s|/opt/zenoh-c/lib||')
nohup ./virtual_robots.x86_64 -force-vulkan > output.log 2>&1 &

Gotcha. Camera frames ride iceoryx2 shared memory, so they are same-host only. A simulator running under WSL and a client running on Windows are two hosts as far as iceoryx2 is concerned: states arrive over zenoh, frames never do. Run both sides in the same place when you want images.

Verifying the install

This prints what the SDK you just installed actually speaks, and it needs no simulator.

vrobots --version

vrobots-sdk 0.1.4
  vrobots_msgs  v2.0.2-31-gac335c0 (schema_version 3)
  flatbuffers   25.12.19
  zenoh         1.9.0
  iceoryx2      0.9.3
  src_id        122

The first line is the SDK release. The second is a git describe of the vrobots_msgs submodule the FlatBuffers code was generated from, so it moves when the schema does. The three pins are exact rather than caret ranges: iceoryx2 compares major, minor and patch on every shared-memory open, and a version one patch off does not error, it silently delivers nothing. That is the first thing to compare against the simulator build when fields look like garbage or a camera stream never appears.

In a source checkout, cargo run -p vrobots-sdk --bin vrobots -- --version prints the same block.

Next: First contact: is anything publishing?

See also: Versions and pins, When nothing happens

First contact: is anything publishing?

Use the vrobots command to prove the simulator is talking before you write any code.

cargo run -p vrobots-sdk --bin vrobots -- topic list

What a healthy scene looks like

topic list opens a zenoh session, listens for a window, and reads the iceoryx2 registry. Anything under vrobots/** that speaks during that window appears.

wire       Hz     bytes  topic
[z]      25.0     33800  vrobots/1/z/state
[i]         -         -  vrobots/1/i/cam/front_left/720p_rgba8

2 topic(s); zenoh observed over 1.5s
[z] zenoh, measured by listening.  [i] iceoryx2, read from the registry:
    it exists, but Hz/bytes were not measured -- same host only.

The four columns:

The dashes are the load-bearing detail. zenoh has no registry, so a [z] row means the topic actually published while you were listening and its numbers are real measurements. iceoryx2 does have a registry, so an [i] row means the service is defined; it may be streaming or it may be a leftover, and a row marked (stale: no process attached) is one whose owning process is gone.

Note. The system id is the second path segment: vrobots/1/z/state is robot 1. This is how you find out which ids the scene really has, which matters because ids are allocated at scene load and keep incrementing. See System ids, and the two kinds of robot.

Two flags are worth knowing now: -k filters keys by case-insensitive substring, and -t sets the zenoh observation window in seconds (default 1.5).

What an empty list means

With nothing running, the tool tells you what it ruled out:

(no topics)

Nothing published on `vrobots/**` in 1.5s, and no iceoryx2 camera
stream is registered. Usually one of:
  - the simulator is not in Play mode
  - it is on another machine (pass --router tcp/<host>:7447)
  - the window was too short for zenoh discovery (try -t 5)

An empty list is a legitimate result, not an error, and the command still exits 0. Work the three causes in order.

flowchart TD
  A[topic list is empty] --> B{Simulator in Play mode?}
  B -->|No| B1[Press Play, run it again]
  B -->|Yes| C{Simulator on this machine?}
  C -->|No| C1[Add --router tcp/host:7447]
  C -->|Yes| D{Does -t 5 list topics?}
  D -->|Yes| D1[zenoh discovery needed a longer window]
  D -->|No| E{Did you pass -k?}
  E -->|Yes| E1[The filter excluded everything]
  E -->|No| F[Compare pins: vrobots --version]

Loading a project into the Unity editor is not enough: the robots publish only while the scene is playing. A remote simulator needs --router tcp/<host>:7447, and even then only the [z] rows will appear, because iceoryx2 camera streams cannot cross hosts at all.

What rate am I actually getting?

Once a topic is listed, watch that one key. The argument is an exact key expression; there are no wildcards.

cargo run -p vrobots-sdk --bin vrobots -- topic hz vrobots/1/z/state

vrobots/1/z/state  [z]  watched 5.0 s

  rate          25.00 Hz      126 samples over a 5.000 s span
  interval   mean 40.00 ms    min 39.10   max 41.20   jitter 0.42 (sd)
  latency    mean 1.20 ms    min 0.80   max 2.10   publish stamp -> here
  seq        1041 -> 1166      0 gap(s), 0 missed
  payload        1352 B avg   33.8 kB/s over the window

Every line reports a spread as well as a summary, because a mean interval on its own hides the one long stall that broke a control loop. The two numbers to read first are the max interval and the gap count: together they explain a loop that stutters even though the average rate looks correct. -w changes the window, which defaults to 5 seconds.

Gotcha. Silence here is a failure, unlike topic list. If nothing arrives, topic hz returns a timeout naming the three things it could be: an idle topic, a typo in the key (a zenoh subscribe on a wrong key is silent, never an error), or a simulator on another host.

Next: Hello states

See also: The vrobots command, Measuring rates, Discovery from code

Hello states

Your first program: connect to a robot and print its position at your own rate.

cargo run -p vrobots-examples --bin ex01_hello_states
./target/cpp-build/ex01_hello_states
python examples/python/ex01_hello_states.py

The whole program

There is no framework here. main does setup, then owns a plain infinite loop.

From examples/rust/src/bin/ex01_hello_states.rs:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const SYS_ID: u32 = 1; // the multirotor in the test scene
const HZ: f64 = 50.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states(); // immutable latest snapshot, never torn
        let [x, y, z] = s.kin.lin_pos;
        println!("State t={:.3} pos=({x:.3},{y:.2},{z:.2})", s.elapsed);
        robot.rate(HZ); // drift-compensated pacing, Hz
    }
}

#include <cstdio>
#include <vrobots_sdk.hpp>

constexpr std::uint32_t SYS_ID = 1;  // the multirotor in the test scene
constexpr double HZ = 50.0;

/// Everything the SDK waits on, retries or drops shows up here. Registering it
/// before connect() is the point: connect is the noisiest moment, and a hang
/// with no log is the hardest thing to debug.
static void on_log(vrsdk::LogLevel level, const char* target, const char* message) {
    std::printf("[%-5s %s] %s\n", vrsdk::to_string(level), target, message);
}

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();  // header and library must be the same release
        vrsdk::set_log_callback(on_log);

        vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
        robot.connect();  // blocks until the first state snapshot arrives
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();  // latest snapshot, never torn
            const double* p = s.kin().lin_pos;
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)\n", s.elapsed, p[0], p[1], p[2]);
            robot.rate(HZ);  // drift-compensated pacing, Hz
        }
    } catch (const vrsdk::Error& e) {
        // `code()` is the SDK's stable number -- the same one Python's
        // VrError.code and the CLI's `error [N]` report.
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

import vrsdk
from vrsdk import RobotType, VirtualRobot

SYS_ID = 1  # the multirotor in the test scene
HZ = 50


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
    mr.connect()

    # ===== loop =====
    while True:
        s = mr.states  # immutable latest snapshot, never torn
        x, y, z = s.kin.lin_pos
        print(f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f})")
        mr.rate(HZ)  # drift-compensated pacing, Hz


if __name__ == "__main__":
    try:
        main()
    except KeyboardInterrupt:
        print("\nstopped.")
    except vrsdk.VrError as e:
        # Every SDK failure is one exception type carrying a stable code.
        raise SystemExit(f"error [{e.code} {e.kind}] {e.detail}")

Three surfaces, one shape: setup, then a loop you own. The C++ version registers a log callback where Rust and Python call init_logging, and states is a method in Rust and C++ but a property in Python. Fifty lines a second, until you press Ctrl-C:

State t=0.000 pos=(0.000,1.05,0.00)
State t=0.020 pos=(0.000,1.05,0.00)
State t=0.040 pos=(0.000,1.05,0.00)
State t=0.060 pos=(0.000,1.05,0.00)

init_logging("info") also puts the SDK's own connect progress on stderr. Your position numbers will differ; a robot sitting still on the ground is the expected first sight.

The STM32 shape, line by line

Read the program as four moves.

1. init_logging("info") installs a tracing subscriber. RUST_LOG overrides the filter you pass, and the call is a no-op if a subscriber is already installed.
2. VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID)) attaches to a robot the scene already contains. Some(id) never touches the manager's create service. It subscribes the state topic and blocks until the first snapshot arrives, so states() is valid the instant connect returns.
3. robot.states() hands back the latest snapshot. This is the whole read API.
4. robot.rate(HZ) sleeps until the next tick, compensating for drift, so the loop runs at the rate you asked for rather than that rate minus your own work.

There is no base class, no runner, no setup() and no update() callback. The SDK never calls your code. This is the single biggest difference from the older Python client, and The shape of a program is the page that argues it.

What states() guarantees

Three guarantees, and one absence that surprises people.

• It never blocks. An SDK-owned subscriber thread keeps a snapshot fresh; you read it at whatever rate suits your controller. A 50 Hz loop against a 25 Hz stream is legal and sees each sample twice.
• It is never torn. You get a whole snapshot or the previous whole snapshot, never a position from one sample and a velocity from the next.
• It never fails. There is no error return. If the simulator stops, states() keeps handing back the last snapshot forever, so a frozen robot and a stopped simulator look identical from inside the loop.
• There is no "is it new" flag. Nothing on the snapshot says whether you have already seen it.

Gotcha. Because a stall presents as unchanging numbers rather than an error, polling states() cannot detect one. wait_new_state(timeout) blocks for a genuinely newer sample and returns VrError::Timeout when none arrives, which is a status rather than a failure. That is the detector, and ex09_state_paced_loop is the example.

elapsed is not a wall clock

s.elapsed is seconds since this robot's first state sample, as an f64, monotonic, and shared by every one of that robot's streams including its cameras. It is the field to print and to plot against.

It is not the time of day and it is not your process's uptime. When you need to compare across streams, use s.t_ns, which is nanoseconds since the unix epoch on the simulator's clock and is directly subtractable from a camera frame's t_ns. s.seq is the per-topic sequence number: a jump larger than one means a sample was dropped.

Which id is the multirotor

SYS_ID is 1 because in the test scene, booted straight into the Flatworld scene, sys_id 1 is the multirotor and sys_id 0 is the truck.

That is a convenience, not a contract. Ids are allocated at scene load and keep incrementing across loads, so the same scene reloaded gives you different numbers. A const at the top of an example is there so the example has something to run against, and vrobots topic list is what tells you the truth.

Next: Hello control

See also: Truth, measured and believed, Pacing your loop, Timestamps and sequence numbers

Hello control

Close the loop: send pulse widths to a multirotor and read them back from the state stream.

cargo run -p vrobots-examples --bin ex02_hello_control
./target/cpp-build/ex02_hello_control
python examples/python/ex02_hello_control.py

The whole program

The same loop as Hello states, with two lines added: one that computes a command and one that sends it.

From examples/rust/src/bin/ex02_hello_control.rs:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const SYS_ID: u32 = 1; // the multirotor in the test scene
const PWM_US: f64 = 1501.0; // microseconds per rotor, on the 1100-2000 band
const HZ: f64 = 100.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states();
        let [x, y, z] = s.kin.lin_pos;
        println!(
            "State t={:.3} pos=({x:.3},{y:.2},{z:.2}) echo={:?}",
            s.elapsed, s.actuator.pwm
        );

        // Do some COOL control here and publish -- PID/EKF is user code, NOT the SDK.
        let cool_control_result = [PWM_US; 4];
        robot.set_mr_pwm(cool_control_result)?;

        robot.rate(HZ);
    }
}

constexpr std::uint32_t SYS_ID = 1;  // the multirotor in the test scene
constexpr double PWM_US = 1501.0;    // microseconds per rotor, 1100-2000 band
constexpr double HZ = 100.0;

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();
        vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
        robot.connect();
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();
            const double* p = s.kin().lin_pos;

            // Do some COOL control here and publish -- PID/EKF is user code,
            // NOT the SDK.
            const std::vector<double> cool_control_result = {PWM_US, PWM_US, PWM_US, PWM_US};
            robot.set_mr_pwm(cool_control_result);

            // The echo: what the robot actually latched, from the state stream.
            const std::vector<std::uint32_t> echo = s.pwm();
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)  pwm_echo=[", s.elapsed, p[0], p[1],
                        p[2]);
            for (std::size_t i = 0; i < echo.size(); ++i) {
                std::printf("%s%u", i ? "," : "", echo[i]);
            }
            std::printf("]  rotor0=%.1f rad/s\n",
                        s.actuator().measured_count > 0 ? s.actuator().measured[0] : 0.0);

            robot.rate(HZ);
        }
    } catch (const vrsdk::Error& e) {
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

SYS_ID = 1  # the multirotor in the test scene
PWM_US = 1501.0  # microseconds per rotor, on the 1100-2000 band
HZ = 100


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
    mr.connect()

    # ===== loop =====
    while True:
        s = mr.states
        x, y, z = s.kin.lin_pos
        print(
            f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f}) "
            f"echo={s.actuator.pwm}"
        )

        # Do some COOL control here and publish -- PID/EKF is user code, NOT the
        # SDK. `set_mr_pwm(a, b, c, d)` and `set_mr_pwm([a, b, c, d])` are the
        # same call.
        cool_control_result = [PWM_US] * 4
        mr.set_mr_pwm(cool_control_result)

        mr.rate(HZ)

Python accepts the four values loose or as one sequence; C++ takes a std::vector<double> and reads the echo through s.pwm(), which is the same actuator.pwm array the other two print directly.

State t=0.000 pos=(0.000,1.05,0.00) echo=[1501, 1501, 1501, 1501]
State t=0.010 pos=(0.000,1.05,0.00) echo=[1501, 1501, 1501, 1501]
State t=0.020 pos=(0.000,1.05,0.00) echo=[1501, 1501, 1501, 1501]

The loop prints the snapshot before it sends, so the first line or two echo whatever was latched on that robot before you started. Your own value appears a physics step after your first send.

The 1100 to 2000 microsecond band

set_mr_pwm takes four pulse widths in microseconds, one per rotor, and every one must be finite and inside 1100 to 2000. The SDK checks that client-side and returns VrError::InvalidArgument before anything reaches the wire, so a bad value is one of the few command mistakes you find out about immediately.

1100 is idle. A flying drone commanded [1100; 4] falls. There is no fixed hover value: hover is wherever total thrust crosses weight for the robot's current mass and thrust curves, both of which are configurable.

Gotcha. Length is not validated the same way. set_mr_pwm_n accepts a slice of any non-empty length, and a wrong rotor count is dropped by the simulator with a log line no client can see, leaving the previous command latched. See Driving a multirotor.

You are the flight controller

SET_MR_PWM is the lowest actuation level the simulator offers. Nothing sits between these four numbers and the thrust curves: no attitude stabilisation, no rate damping, no mixer. Four equal pulse widths produce four equal thrusts, and any imbalance in mass or inertia tips the vehicle over with nothing to catch it.

That is deliberate. The point of the simulator is that the stabilisation is your code. A PID loop, an EKF, an LQR: all of it lives in your main, and the SDK contributes nothing to it.

The actuator echo is the only receipt

Commands are published to vrobots/{sys_id}/z/cmd and the robot acknowledges nothing. It drains its command queue at the start of the next physics step and moves on. A wrong command id, a wrong sys_id and a wrong array length all present identically from outside: the state stream does not change.

So the receipt is s.actuator.pwm, the commanded pulse widths echoed back inside the next snapshot. Watching it settle on [1501, 1501, 1501, 1501] is the proof that your command landed on the robot you meant.

Commands also latch. Each one is a setpoint, not an impulse, and the last one received stays in effect until the next arrives. There is no watchdog, so a 5 Hz sender and a 100 Hz sender are both fine, and a controller that stops sending leaves the robot flying its final command. This example runs at 100 Hz while physics runs at 50 Hz, which is harmless.

1501 will not lift it, 1700 will

PWM_US is 1501, barely off idle, so the printed position does not change: the example is about seeing the echo, not about flying. Edit the constant to 1700 and run it again to watch the vertical component of pos move.

Note. Which component that is depends on the robot's frame. The multirotor publishes frd, where the third component points down, so climbing makes it more negative. Read s.coord_frame_id rather than assuming, as Frames, axes and units explains.

Next: Hello car

See also: Commands latch, Driving a multirotor, Actuators

Hello car

Drive the truck, whose three channels use a different band from the multirotor's.

cargo run -p vrobots-examples --bin ex05_hello_car
./target/cpp-build/ex05_hello_car
python examples/python/ex05_hello_car.py

The whole program

The same loop shape as Hello control, a different actuator, and a different robot: SYS_ID is 0 because the truck is the other vehicle in the test scene.

From examples/rust/src/bin/ex05_hello_car.rs:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const SYS_ID: u32 = 0; // the truck in the test scene
const STEER_US: f64 = 1400.0; // left of centre
const THROTTLE_US: f64 = 1650.0; // light forward
const BRAKE_US: f64 = 1100.0; // released
const HZ: f64 = 50.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Truck, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states();
        let [x, y, z] = s.kin.lin_pos;
        // lin_vel is a BODY-frame vector, so no single component is "the speed";
        // its magnitude is.
        let [vx, vy, vz] = s.kin.lin_vel;
        let speed = (vx * vx + vy * vy + vz * vz).sqrt();
        println!(
            "State t={:.3} pos=({x:.3},{y:.2},{z:.2}) speed={speed:.2} m/s echo={:?}",
            s.elapsed, s.actuator.pwm
        );

        // A gentle left arc: steering left of centre, light forward throttle.
        robot.set_car(STEER_US, THROTTLE_US, Some(BRAKE_US))?;

        robot.rate(HZ);
    }
}

constexpr std::uint32_t SYS_ID = 0;      // the truck in the test scene
constexpr double STEER_US = 1400.0;      // left of centre
constexpr double THROTTLE_US = 1650.0;   // light forward
constexpr double BRAKE_US = 1100.0;      // released
constexpr double HZ = 50.0;

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();
        vrsdk::VirtualRobot robot(vrsdk::RobotType::Truck, SYS_ID);
        robot.connect();
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();
            const double* p = s.kin().lin_pos;

            // A gentle left arc: steering left of centre, light forward
            // throttle, brake released.
            robot.set_car(STEER_US, THROTTLE_US, BRAKE_US);

            // Speed from the body-frame twist, so it is visible that the truck
            // really is moving rather than that the command was merely accepted.
            const double* v = s.kin().lin_vel;
            const double speed = std::sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]);

            const std::vector<std::uint32_t> echo = s.pwm();
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f) speed=%.2f m/s  pwm_echo=[", s.elapsed,
                        p[0], p[1], p[2], speed);
            for (std::size_t i = 0; i < echo.size(); ++i) {
                std::printf("%s%u", i ? "," : "", echo[i]);
            }
            std::printf("]\n");

            robot.rate(HZ);
        }
    } catch (const vrsdk::Error& e) {
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

SYS_ID = 0  # the truck in the test scene
STEER_US = 1400.0  # left of centre
THROTTLE_US = 1650.0  # light forward
BRAKE_US = 1100.0  # released
HZ = 50


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    car = VirtualRobot(RobotType.TRUCK, sys_id=SYS_ID)
    car.connect()

    # ===== loop =====
    while True:
        s = car.states
        x, y, z = s.kin.lin_pos
        # lin_vel is a BODY-frame vector, so no single component is "the speed";
        # its magnitude is.
        speed = math.dist(s.kin.lin_vel, (0.0, 0.0, 0.0))
        print(
            f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f}) "
            f"speed={speed:.2f} m/s echo={s.actuator.pwm}"
        )

        # A gentle left arc: steering left of centre, light forward throttle.
        car.set_car(STEER_US, THROTTLE_US, BRAKE_US)

        car.rate(HZ)

The brake is the third argument in all three, and it is optional in all three: Rust wraps it in Some, C++ in a std::optional, Python defaults it to None. Omitting it sends the two-channel form, which brakes nothing.

The truck pulls away and curves left, so pos walks and speed climbs to a steady value:

State t=0.000 pos=(0.000,0.35,0.00) speed=0.00 m/s echo=[1400, 1650, 1100]
State t=0.020 pos=(0.021,0.35,0.00) speed=1.06 m/s echo=[1400, 1650, 1100]
State t=0.040 pos=(0.043,0.35,0.01) speed=1.09 m/s echo=[1400, 1650, 1100]

The three channels

set_car(steer, throttle, brake) takes pulse widths in microseconds, one per channel. The truck's factory band is 1100 to 1900, which is not the multirotor's 1100 to 2000.

The SDK's client-side check is the wider 1100 to 2000 band it applies to every pulse width, so a value of 1950 is accepted by the SDK and handled by the truck's own limits. Stay inside 1100 to 1900 and the two agree.

Brake is bottom-anchored

Steer and throttle are centre-anchored: 1500 is neutral for both, and the interesting values live on either side of it. Brake is not. 1100 is released and 1900 is full, so the neutral value for the brake channel is the bottom of the band, not the middle.

Sending 1500 on the brake channel therefore applies braking rather than none, which is a common reason a truck that should be accelerating crawls instead.

Gotcha. brake is an Option<f64>. Passing None sends the two-channel form of SET_CAR, which brakes nothing. That is a different statement from "the brake is released to 1100": the channel is absent from the message. Pass Some(1100.0) when you want to say released explicitly.

Speed is a magnitude, not a component

kin.lin_vel is a body-frame vector, so no single component of it is "the speed". The example takes the magnitude, which is why the arithmetic is spelled out rather than reading vy and calling it done.

kin.lin_pos, by contrast, is a world-frame position. That split (pose in world, twist and acceleration in body) is physics rather than configuration, and it holds for every robot and for both the truth and the estimate blocks.

The truck also disagrees with the multirotor about which way is up: it publishes "fru" where the multirotor publishes "frd", so the third component of a vector means opposite things on the two vehicles. s.coord_frame_id is the authoritative answer, and Frames, axes and units is the page that settles it.

Next: Hello image

See also: Driving the truck, The truck drivetrain, Kinematics

Hello image

Open the camera the robot already has, and read frames as they arrive.

cargo run -p vrobots-examples --bin ex03_hello_image
./target/cpp-build/ex03_hello_image
python examples/python/ex03_hello_image.py

Opening a camera

Every vrobot ships with front_left and front_right mounted, at 720p rgba8. Reading images does not start with creating a camera: it starts with attaching to one of those. open_camera opens the iceoryx2 subscriber and touches the simulator not at all. The name, resolution and format are constants, as they are in every example.

From examples/rust/src/bin/ex03_hello_image.rs:

const SYS_ID: u32 = 1; // the multirotor in the test scene
const CAMERA: &str = "front_left"; // every vrobot ships front_left and front_right
const RESOLUTION: &str = "720p";
const FORMAT: &str = "rgba8"; // Unity's native readback -- four channels, NOT rgb8
const FRAMES: u64 = 120; // then exit
const HZ: f64 = 100.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

    // open_camera SUBSCRIBES to a camera the robot already has, without mutating
    // the sim. The name, resolution and format must match the publisher exactly
    // -- on iceoryx2 those three strings are the stream identity -- so a mismatch
    // surfaces as VrError::Timeout (ex13 shows that path).
    let cam = robot.open_camera(CAMERA, RESOLUTION, FORMAT)?;
    println!("camera stream: {}", cam.service_name());

    let mut seen = 0u64;

constexpr std::uint32_t SYS_ID = 1;  // the multirotor in the test scene
constexpr const char* CAMERA = "front_left";  // every vrobot ships front_left and front_right
constexpr const char* RESOLUTION = "720p";
constexpr const char* FORMAT = "rgba8";  // Unity's native readback -- four channels, NOT rgb8
constexpr std::uint64_t FRAMES = 120;    // then exit
constexpr double HZ = 100.0;

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();
        vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
        robot.connect();

        // open_camera SUBSCRIBES to a camera the robot already has, without
        // mutating the sim. The name, resolution and format must match the
        // publisher exactly -- on iceoryx2 those three strings are the stream
        // identity -- so a mismatch surfaces as a timeout (ex13 shows that path).
        vrsdk::CameraStream cam = robot.open_camera(CAMERA, RESOLUTION, FORMAT);
        std::printf("camera stream: %s\n", cam.service_name().c_str());

        std::uint64_t seen = 0;

SYS_ID = 1  # the multirotor in the test scene
CAMERA = "front_left"  # every vrobot ships front_left and front_right
RESOLUTION = "720p"
FORMAT = "rgba8"  # Unity's native readback -- four channels, NOT rgb8
FRAMES = 300  # then exit
HZ = 100


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
    mr.connect()

    # open_camera SUBSCRIBES to a camera the robot already has, without
    # mutating the sim. The name, resolution and format must match the
    # publisher exactly -- on iceoryx2 those three strings are the stream
    # identity -- so a mismatch surfaces as a TIMEOUT (ex13 shows that path).
    cam = mr.open_camera(CAMERA, RESOLUTION, FORMAT)
    print(f"camera stream: {cam.service_name}")

    seen = 0

The printed name is the iceoryx2 service, and it matches a row of vrobots topic list character for character:

camera stream: vrobots/1/i/cam/front_left/720p_rgba8

Two processes can open the same stream and neither disturbs the other, because neither owns the camera. mount_camera is the other verb: it creates a camera of your choosing, and it is what page Lens and mount pose covers. You need it only when the pair the robot ships cannot serve you.

The loop, and what fresh() means

Images and states are two independent streams. The image half of the loop runs once per frame while the state half runs every iteration, and the code says which by branching on fresh().

#![allow(unused)]
fn main() {
    // ===== loop =====
    while seen < FRAMES {
        let s = robot.states();

        // Images are a separate stream with their own timestamps -- never assume
        // they match the state's. Compare t_ns explicitly when fusing.
        if let Some(frame) = cam.fresh() {
            // Some only if new since the last read
            seen += 1;
            println!(
                "Image {} t={:.3} size=({}x{}) seq={} lag_vs_state={:.1} ms",
                frame.camera_name,
                frame.elapsed,
                frame.width,
                frame.height,
                frame.seq,
                (s.t_ns - frame.t_ns) as f64 / 1e6
            );
}

        // ===== loop =====
        while (seen < FRAMES) {
            const vrsdk::State s = robot.states();

            // A value only if new since the last read.
            if (auto frame = cam.fresh()) {
                ++seen;
                std::printf("Image %s t=%.3f size=(%ux%u) seq=%llu lag_vs_state=%.1f ms\n",
                            frame->camera_name.c_str(), frame->elapsed(), frame->width(),
                            frame->height(),
                            static_cast<unsigned long long>(frame->seq()),
                            static_cast<double>(s.t_ns - frame->t_ns()) / 1e6);

    # ===== loop =====
    while seen < FRAMES:
        s = mr.states

        # Images are a separate stream with their own timestamps -- never assume
        # they match the state's. Compare t_ns explicitly when fusing.
        if cam.fresh:
            frame = cam.frame  # metadata for the image we are about to read
            img = cam.image  # numpy (h, w, c) uint8, top-down, RGB(A)
            seen += 1

            print(
                f"Image {frame.camera_name} t={frame.elapsed:.3f} "
                f"size=({frame.width}x{frame.height}) seq={frame.seq} "
                f"lag_vs_state={(s.t_ns - frame.t_ns) / 1e6:.1f} ms"
            )

Rust and C++ ask and receive in one move, so the frame arrives inside an Option that the if unwraps. Python splits it: cam.fresh is a boolean property and cam.image is the numpy array, and reading the image is what consumes the freshness.

State t=0.410 (no new frame)
Image front_left t=0.412 size=(1280x720) seq=17 lag_vs_state=12.4 ms
      sky-ness (B-R) top=+98 bottom=-25 (top-down: sky above ground), fov_y=61.9 deg
State t=0.420 (no new frame)

fresh() returns Some only when a frame has arrived since the last call, and it hands each frame out exactly once. That makes it the right read for work that must not run twice on one image. Its counterpart latest() returns the current frame regardless and does not consume freshness.

The lag_vs_state figure is why the two streams are never paired by the SDK. Frames arrive at the render rate and states at 25 Hz, so no frame belongs to any state. Both t_ns values are on the same clock, so subtracting them is meaningful, and doing that subtraction explicitly is what fusion looks like here.

Gotcha. Camera frames ride iceoryx2 shared memory, so they are same-host only. zenoh will happily reach a simulator on another machine and deliver states, and not one frame will follow. There is no error: open_camera times out after ConnectOptions::camera_timeout, five seconds by default.

Rows in frame.data are row-major and top-down: row 0 is the top of the picture. The wire is bottom-up, following Unity's render order, and the SDK flips while copying. The sky-ness figures are the check on that, measuring blue minus red on the first and last rows: outdoors the top row is sky and the bottom is ground. Channel order is the renderer's own RGBA, never swapped, so OpenCV users convert to BGR at the call site.

Nothing to clean up

There is no unmount at the end, and that is the point of opening rather than mounting. This handle never created a camera, so it has nothing to remove; letting the stream go ends this subscription and front_left keeps rendering and publishing for everyone else.

#![allow(unused)]
fn main() {
    // Nothing to unmount: this handle never created a camera. Dropping the stream
    // ends this subscription only -- front_left keeps rendering and publishing for
    // everyone else.
    let stats = cam.stats();
    println!(
        "{seen} frame(s), received={} decode_errors={} seq_gaps={}",
        stats.received, stats.decode_errors, stats.seq_gaps
    );
    Ok(())
}
}

        // Nothing to unmount: this handle never created a camera. Letting the
        // stream go ends this subscription only -- front_left keeps rendering and
        // publishing for everyone else.
        const vrsdk_camera_stats_t st = cam.stats();
        std::printf("%llu frame(s), received=%llu decode_errors=%llu seq_gaps=%llu\n",
                    static_cast<unsigned long long>(seen),
                    static_cast<unsigned long long>(st.received),
                    static_cast<unsigned long long>(st.decode_errors),
                    static_cast<unsigned long long>(st.seq_gaps));
        return 0;

    # Nothing to unmount: this handle never created a camera. Letting the stream
    # be collected ends this subscription only -- front_left keeps rendering and
    # publishing for everyone else.
    st = cam.stats
    print(
        f"{seen} frame(s), received={st.received} "
        f"decode_errors={st.decode_errors} seq_gaps={st.seq_gaps}"
    )

120 frame(s), received=120 decode_errors=0 seq_gaps=0

That count is FRAMES, so the Python run prints 300 rather than 120: it opens a window and wants more of them. Because nothing was mutated, the loop bound is a convenience rather than a cleanup deadline, and Ctrl-C is as safe an exit as running to the end. The one example that does have a cleanup step is ex17_camera_pose, which mounts a camera of its own.

Next: Hello service

See also: Mount, open and unmount, Freshness, Inside a frame, Showing frames in a window

Hello service

Create a robot from code and delete it again, which is what explicit lifecycle means.

cargo run -p vrobots-examples --bin ex04_hello_service
./target/cpp-build/ex04_hello_service
python examples/python/ex04_hello_service.py

The whole program

Lifecycle and configuration are one-shot request and response, so this is the first example with no loop in it.

From examples/rust/src/bin/ex04_hello_service.rs:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const ROBOT_TYPE: RobotType = RobotType::Multirotor;

fn main() -> Result<(), VrError> {
    vrobots_sdk::init_logging("info");

    // Create a NEW robot in the sim (no sys_id -> manager create; reply carries the id).
    let robot = VirtualRobot::connect(ROBOT_TYPE, None)?;
    let sys_id = robot.sys_id();
    println!("created sys_id = {sys_id}");

    // The create reply is a receipt; the robot *exists* once its state topic
    // publishes, which connect() already waited for -- so this is real data.
    let s = robot.states();
    println!(
        "first state: t={:.3} seq={} name={:?}",
        s.elapsed, s.seq, s.name
    );
    println!("its state topic: {}", vrobots_sdk::topics::state(sys_id));

    // Deletion is explicit and never implicit. delete() waits for the state topic
    // to fall silent: the manager's ack is only a receipt, absence is the proof.
    robot.delete()?;
    println!(
        "deleted sys_id = {sys_id} (is_deleted={})",
        robot.is_deleted()
    );

    // The handle is spent. Commands do not silently do nothing -- they say why.
    match robot.set_mr_pwm([1500.0; 4]) {
        Ok(()) => println!("unexpected: a deleted robot accepted a command"),
        Err(e) => println!("the handle is spent, as expected: [{}] {e}", e.code()),
    }
    Ok(())
}

int main() {
    try {
        vrsdk::check_version();

        // Create a NEW robot in the sim: `create` means "no sys_id", so the
        // manager assigns one and the reply carries it. (A constructor would be
        // ambiguous with the attach form -- see the header.)
        vrsdk::VirtualRobot robot = vrsdk::VirtualRobot::create(vrsdk::RobotType::Multirotor);
        robot.connect();
        const std::uint32_t sys_id = robot.sys_id();
        std::printf("created sys_id = %u\n", sys_id);

        // The create reply is a receipt; the robot *exists* once its state topic
        // publishes, which connect() already waited for -- so this is real data.
        const vrsdk::State s = robot.states();
        std::printf("first state: t=%.3f seq=%llu name=\"%s\"\n", s.elapsed,
                    static_cast<unsigned long long>(s.seq), s.name.c_str());
        // The C++ surface has no topic-name builder (Rust has
        // `vrobots_sdk::topics`, Python has `vrsdk.topics`), and the shape is
        // fixed by the wire, so compose it here.
        std::printf("its state topic: vrobots/%u/z/state\n", sys_id);

        // Deletion is explicit and never implicit. The manager's ack is only a
        // receipt, so remove() also waits for the robot's state topic to fall
        // silent -- that is the real confirmation.
        robot.remove();
        std::printf("deleted sys_id = %u (removed=%s)\n", sys_id,
                    robot.removed() ? "true" : "false");

        // The handle is spent. Commands do not silently do nothing -- they say
        // why.
        try {
            robot.set_mr_pwm({1500.0, 1500.0, 1500.0, 1500.0});
            std::printf("unexpected: a deleted robot accepted a command\n");
        } catch (const vrsdk::Error& e) {
            std::printf("the handle is spent, as expected: [%d] %s\n", e.code(), e.what());
        }
        return 0;
    } catch (const vrsdk::Error& e) {
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

ROBOT_TYPE = RobotType.MULTIROTOR  # or RobotType.TRUCK / RobotType.from_key("truck")


def main() -> None:
    vrsdk.init_logging("info")

    # Create a NEW robot in the sim (no sys_id -> manager create; the reply
    # carries the assigned id).
    robot = VirtualRobot(ROBOT_TYPE)
    robot.connect()
    sys_id = robot.sys_id
    print(f"created sys_id = {sys_id}")

    # The create reply is a receipt; the robot *exists* once its state topic
    # publishes, which connect() already waited for -- so this is real data.
    s = robot.states
    print(f"first state: t={s.elapsed:.3f} seq={s.seq} name={s.name!r}")
    print(f"its state topic: {vrsdk.topics(sys_id)['state']}")

    # Deletion is explicit and never implicit. delete() waits for the state topic
    # to fall silent: the manager's ack is only a receipt, absence is the proof.
    robot.delete()
    print(f"deleted sys_id = {sys_id} (is_deleted={robot.is_deleted})")

    # The handle is spent. Commands do not silently do nothing -- they say why.
    try:
        robot.set_mr_pwm(1500, 1500, 1500, 1500)
        print("unexpected: a deleted robot accepted a command")
    except vrsdk.VrError as e:
        print(f"the handle is spent, as expected: [{e.code} {e.kind}] {e.detail}")

Three differences worth naming. C++ spells creation VirtualRobot::create rather than a one-argument constructor, because a literal 0 would be ambiguous between "attach to sys_id 0" and a null options pointer. C++ spells deletion remove(), because delete is a keyword. And C++ has no topic-name builder, so it composes vrobots/<id>/z/state inline where Rust calls topics::state and Python calls vrsdk.topics.

The run takes a second or two, most of it spent waiting for the new robot's state topic to start and then to stop:

created sys_id = 7
first state: t=0.000 seq=0 name="multirotor"
its state topic: vrobots/7/z/state
deleted sys_id = 7 (is_deleted=true)
the handle is spent, as expected: [8] deleted: sys_id 7 was deleted from the sim; this handle is spent

Create versus attach

The second argument to connect decides which of two quite different things happens.

Every example so far passed Some(SYS_ID). This one passes None, so the manager allocates the id and robot.sys_id() is the only way to learn it.

Creating is limited by the scene's catalog, not the SDK's. The sandbox scene registers multirotor, truck and msd; any other key is refused with a message naming the ones it does know, which is also the only live way to enumerate the catalog. Robot types that are scene-authored only, such as the cart pole and the Global Hawk, can be attached to but never created.

Note. Create is the one non-idempotent service in the system. The SDK sends it exactly once and never retries, because every retry that reaches the manager reserves another id and spawns another robot. What connect actually does walks the four steps.

Robots outlive the process

Dropping a VirtualRobot closes its zenoh session. It does not delete the robot. The robot keeps flying its last latched command, keeps publishing state, and is still there after your program exits, after you rebuild, and after you run something else.

This is why the example calls delete(): without it, every run would leave another multirotor in the scene. Comment the delete() line out and run it twice to see exactly that, then attach ex01_hello_states to one of the ids it printed. Create in one process, attach from another, is the normal shape of a multi-program session.

The way back from a littered scene is to reload it. Ids are allocated at scene load and keep incrementing, so the reloaded scene numbers its robots differently.

Absence is the proof, both ways

connect(type, None) returns only once the new robot's state topic has published, and delete() returns only once that topic has fallen silent. Neither waits on the service acknowledgement, because an acknowledgement is packed the instant the request arrives and says nothing about whether the work happened.

That principle runs through the whole services chapter: an ack is a receipt, not a result, and the state stream is the confirmation. It is rule two of Five rules that explain everything.

After delete() the handle is spent, and it says so rather than failing quietly. Every command on it returns VrError::Deleted, error code 8, with a message naming the id.

Next: When nothing happens

See also: Robot lifecycle, System ids, and the two kinds of robot, Appendix C: Error reference

When nothing happens

A symptom-to-cause table for the failures that stop people on their first afternoon.

Almost every one of these is a correct behaviour presenting as a broken one. The SDK reports what it can, but a great deal of the system communicates by silence: a command that landed on the wrong robot, a topic name with a typo, and a simulator that is not playing all look the same from inside your loop.

Two commands answer most of it before you read any further:

cargo run -p vrobots-sdk --bin vrobots -- topic list
cargo run -p vrobots-sdk --bin vrobots -- --version

Symptoms and causes

Sim bug. In simulator v3.0.0 a client-created multirotor does not move. Its rigidbody never integrates, so it hangs where it spawned and ignores every pulse width and even a direct body force, while its actuator echo and rotor-speed model answer perfectly normally. Created trucks and mass-spring-dampers have live physics, and the scene's own multirotor flies. See issues/created-multirotor-frozen-dynamics.md. The examples that need a multirotor to move take an optional sys_id so you can attach to the scene's one instead.

When the table does not have it

Turn the logs up. RUST_LOG overrides whatever filter the program passed to init_logging, and the SDK's own tracing events name each connect step as it happens.

RUST_LOG=vrobots_sdk=debug cargo run -p vrobots-examples --bin ex01_hello_states

RUST_LOG reaches the Rust core, so the other two surfaces turn the volume up in their own idiom instead. Python routes the same events into the standard logging module, so vrsdk.init_logging("debug") (or raising the vrobots_sdk logger's level yourself) is the equivalent. C++ registers a handler with vrsdk::set_log_callback and then calls vrsdk::set_log_level(vrsdk::LogLevel::Debug). Logging covers all three.

Add zenoh's own view with RUST_LOG=vrobots_sdk=debug,zenoh=info. iceoryx2 logs to stderr outside tracing entirely and is controlled by IOX2_LOG_LEVEL, which the SDK defaults to errors only.

Two counters are worth printing from inside a loop that misbehaves without failing: stats() carries received, decode error, sequence gap and missed sample counts, and last_error() holds the most recent decode error. Neither tears down the session, which is the point: a malformed payload is counted and the loop keeps running.

Next: Concepts

See also: Logging, Appendix C: Error reference, Known simulator issues

Concepts

The model you need before writing a real control loop, and why five correct behaviours still surprise everyone.

Chapter 1 got a program talking to the simulator. This chapter explains the system it is talking to, so that the four API chapters after it can be short: once you know how a command travels, "commands latch" is one sentence rather than one chapter.

Five behaviours that are correct and still surprise everyone

Almost every confused bug report against this SDK is one of five behaviours. None of them is a defect. Each is invisible until it bites, and each has a different failure signature, which is why Five rules that explain everything is the page to read twice.

The common thread is that the simulator is a separate process with its own clock. You are not calling into it; you are publishing to it and reading what it publishes back. Nothing in this SDK hides that, because hiding it is what produces the bugs above.

What this chapter covers

Read it in order the first time. Every later page assumes the vocabulary this one defines, and terms are defined once, here or in Appendix D: Glossary.

What it does not cover

This chapter is the model, not the API. It names methods where a name makes a concept concrete, and it does not tabulate their arguments: that is what Reading state, Sending commands, Cameras and images and Services and configuration are for. Per-robot behaviour lives in Supported virtual robots.

Next: Two transports, one simulator

See also: Getting started, Appendix D: Glossary

Two transports, one simulator

Why states travel over zenoh and camera frames over iceoryx2, and what that costs you.

Publish and subscribe, not call and return

Nothing in this system is a function call. The simulator publishes what it knows and subscribes to what you send, and both ends run on their own clocks. A publisher does not know who is listening, a subscriber does not know who is sending, and neither blocks waiting for the other. Services are the one exception, and even they are request/response over the same publish machinery rather than a procedure call: the reply says the request arrived, not that anything happened.

That is the whole reason the rest of this chapter exists. Every surprise in Five rules that explain everything follows from the fact that you are talking to a process that is not waiting for you.

Two transports carry that traffic, and they are not interchangeable.

Why the split is not arbitrary

A state snapshot is around a kilobyte and arrives 25 times a second. A 720p RGBA frame is nearly four megabytes. Pushing frames through a network-capable transport would mean serialising and copying them for a subscriber that is, in practice, always the process next door. iceoryx2 hands over a pointer into shared memory instead, so the frame is never copied across a socket at all.

The price is exactly the property that makes it fast. Shared memory does not cross a machine boundary. Set ConnectOptions::router_endpoint at a simulator on another host and you get states, commands and services, and no images: the camera stream never appears and open_camera times out after camera_timeout. That is not a misconfiguration you can fix with a flag.

Gotcha. A remote connection that works perfectly for control and returns nothing but timeouts from every camera call is not broken. Check whether the simulator is on this host before debugging the camera code.

The second consequence of the split is discovery. Zenoh has no registry, so vrobots topic list can only report what actually published during its observation window: a robot that is paused is a robot that does not exist as far as that listing is concerned. iceoryx2 does have a registry, so a camera service can be listed without anything streaming through it, and a dead entry is marked stale rather than vanishing. Discovery from code calls this the observed-versus-registered distinction, and it is what the [z] and [i] tags are telling you:

wire       Hz     bytes  topic
[z]      25.0      1234  vrobots/1/z/state
[i]         -         -  vrobots/0/i/cam/front/720p_rgba8

The - columns are not missing data. They are a registry entry, which by construction has no measured rate.

FlatBuffers on the wire

Both transports carry FlatBuffers payloads, generated from the schemas in the vrobots_msgs submodule and shared byte for byte with the simulator's C# side. The submodule ships the generated Rust, so building the SDK needs no flatc.

Two properties of that choice show up in the API. Decoding verifies the buffer before reading any field, so a truncated or hostile payload produces VrError::Decode rather than an out-of-bounds read. And a nested table that is missing from the wire decodes to its Default (all zero, false, empty) rather than failing, which is what lets an older simulator talk to a newer SDK: fields it does not know about arrive as zero, not as an error.

The pins are exact, and that is not pedantry

ipc_versions.json at the repository root is the source of truth for the three IPC versions this SDK must match.

The exactness is load-bearing for one specific reason. iceoryx2 compares major.minor.patch on every shared-memory open. A caret pin that resolves one patch release away from the simulator's does not error and does not warn: it silently delivers nothing. The symptom is a camera stream that never produces a frame, which reads as "the simulator is not publishing" and sends you looking in entirely the wrong place.

Three mechanisms keep the pins honest, so this is a failure you should never actually see: build.rs fails the build on drift from ipc_versions.json, scripts/check_versions.ps1 fails CI, and the release workflow refuses to build. vrobots --version prints the versions a given binary was built against, which is the first thing to check when a simulator and an SDK disagree. Versions and pins covers the whole procedure.

Next: The topic namespace

See also: Cameras and images, Versions and pins, Discovery from code

The topic namespace

Every key the simulator publishes or subscribes, and how a robot's id fits into it.

One rule generates every name

Every key in the system has the shape vrobots/<sys_id>/<transport>/<subject...>. The segment after the id names the transport: z for zenoh, i for iceoryx2. Two reserved words, manager and scene, sit where a system id would, so a swarm-wide service can never collide with a robot's own.

flowchart LR
  R["vrobots"] --> ID["{sys_id}"]
  R --> MGR["manager"]
  R --> SCN["scene"]
  ID --> Z["z"]
  ID --> I["i"]
  Z --> ST["state"]
  Z --> FR["frames"]
  Z --> CM["cmd"]
  Z --> SRV["srv/{segment}"]
  I --> CAM["cam/{name}/{res}_{fmt}"]
  MGR --> MZ["z/srv/create<br/>z/srv/delete"]
  SCN --> SZ["z/srv/frame"]

Both ends of a wire must agree on these names byte for byte, and a mismatch is silent: a subscriber on a slightly wrong key never fires at all. That is why no call site in the SDK builds a name inline. Every one of them comes from the vrobots_sdk::topics module, which mirrors the simulator's own VRobotsTopics.cs, and it is public so your code can print the same string the simulator uses.

The full pattern table

Two further per-robot zenoh keys exist and are covered where they are used. vrobots/{sys_id}/z/frames publishes the coordinate-frame definitions that result from a robot's frame configuration, which is a different thing from the srv/frames service that sets them: Coordinate frames covers both. vrobots/{sys_id}/z/estimate carries an external state estimate that the fixed wing can be told to fly on instead of the truth, described in Fixed wing control.

A concrete camera key, with every placeholder filled in:

vrobots/1/i/cam/front_left/720p_rgba8

The {res}_{fmt} segment is built from the resolution and pixel format you asked for, which is why changing a camera's format renames its stream. The full list of service segments is in Appendix A: Topic reference; the ones a given robot answers depend on its type, and asking for one it does not serve is how a capability probe works.

The id slot is the routing

There is no addressing anywhere else in a message. A robot subscribes to its own z/cmd and nothing else, so the id in the key is the delivery decision. Send a command with the wrong id and it is not misrouted to another robot; it is received by whichever robot owns that key, and the robot you meant hears nothing.

Gotcha. A wrong sys_id and a command a robot does not implement look identical from outside: the state stream does not change. Confirm the id against vrobots topic list before suspecting the command.

Wildcards

Two wildcard keys are worth knowing, both zenoh only.

ALL_STATES is the shape a swarm program wants: fifty VirtualRobot handles is fifty sessions and fifty subscriber threads, where one subscription on the wildcard is one of each. That is a different program from the one this book teaches, and the key is here so it is buildable.

Wildcards work for subscribing, not for measuring. measure_rate and vrobots topic hz need an exact key and reject a wildcard with VrError::InvalidArgument.

Reading the namespace back

vrobots topic list is the ground truth for what exists right now, and it is the first command to run when something is not responding. It reports the transport tag, the measured rate for zenoh topics, and the registry entry for iceoryx2 ones. The vrobots command documents the flags; Discovery from code does the same listing from inside a program via list_topics.

topics::sys_id_of(key) parses the id back out of a key, returning None for manager and scene. That is the right answer rather than a parse failure: those two scopes belong to no robot.

Next: System ids, and the two kinds of robot

See also: Appendix A: Topic reference, The vrobots command, More than one robot

System ids, and the two kinds of robot

Scene-authored robots you attach to, created robots you spawn, and why an id is never a constant.

Two kinds of robot

A sys_id is a u32 that names one robot for as long as the scene is loaded. Every topic carries it, every command is routed by it, and there are exactly two ways to end up holding one.

Scene-authored robots are the ones the scene placed. They exist the moment the simulator enters Play mode, before any client connects, and they are still there after every client exits. You reach them by naming their id: VirtualRobot::connect(RobotType::Multirotor, Some(1)).

Created robots are the ones the SDK spawned. connect(type, None) asks the manager's srv/create and the reply carries a fresh id. They are yours in the sense that you know their id, and not in any stronger sense: nothing stops another client attaching to them, and they survive your process exiting.

The attach path is the more capable of the two. It never queries the catalog, so it works for any robot the scene contains regardless of what can be created, and it is the only way to reach three of the six robot types.

Ids are allocated at load time and keep incrementing

On a fresh boot straight into the Flatworld scene, sys_id 0 is the truck and sys_id 1 is the multirotor. Every example up to ex20 names its id in a const for exactly that reason.

Those constants are a convenience and never a contract. Ids are allocated as the scene loads and keep incrementing across scene loads, so reloading the scene, or creating and deleting a robot, moves the numbers. A const SYS_ID: u32 = 1 that worked this morning can address a robot that no longer exists this afternoon, and the failure is a connect that times out waiting for a first state sample rather than anything that names the id as the problem.

Gotcha. connect reporting no state on vrobots/<id>/z/state within probe_timeout usually means that id is wrong, not that the simulator is down. Run vrobots topic list and read the ids that are actually publishing.

The habit that makes this a non-issue: take the id as a command-line argument rather than compiling it in. That is what ex29 through ex33 do, because a cart-pole, a half-drone and a Global Hawk cannot be created and their ids are not knowable when the example is written.

The spawn catalog belongs to the scene

The set of types srv/create will spawn is registered by the scene's own spawner, not by the SDK. The sandbox scene registers three:

Keys are matched exactly and case-sensitively on the wire. RobotType::catalog_key() produces the wire form, and RobotType::from_catalog_key(&str) parses it back case-insensitively, accepting the synonyms the examples and the simulator's UI use: car for a truck, cart_pole and invpen for a cart-pole, mass_spring_damper for an MSD, half_drone, global_hawk and rq4b. Only catalog_key() is ever put on the wire. RobotType::is_in_sandbox_catalog() reports the third column above.

The refusal message is the catalog

A key the running scene does not register is refused with ok = false, and the refusal names every key that scene does know:

unknown type 'globalhawk' (known: multirotor, truck, msd)

connect surfaces that string verbatim as VrError::Service. That makes a failed create the only live way to enumerate the catalog: there is no list service, and the table above is a measurement of the sandbox scene rather than a property of the SDK. Another scene may register more, and the refusal from that scene will say so.

The three you can only attach to

cartpole, halfdrone and globalhawk exist as scene-authored robots and must be reached by id.

The Global Hawk is a further step removed: it lives in the IMU scene, not the sandbox. Launching the sandbox and looking for one finds nothing, and no amount of correcting the id helps. Load the right scene first, then read the ids off vrobots topic list.

Note. Tier gating changes what is reachable. Under Pro every robot in the scene serves simultaneously; under Guest only the robot selected in the simulator's SYS-ID dropdown has open ports, so a correct id can still time out. Known simulator issues has the detail.

Deletion is explicit, always

Nothing in the SDK removes a robot implicitly. Dropping a VirtualRobot closes the session and leaves the robot exactly as it was, still running, still publishing, still latched on its last command. Only delete() removes one, and a handle whose robot has been deleted is spent: further commands return an error rather than silently doing nothing.

That is the fourth of the five rules, and Robot lifecycle covers the full sequence.

Next: The shape of a program

See also: Robot lifecycle, Supported virtual robots, What connect actually does

The shape of a program

The SDK never calls your code: main does setup and owns the loop.

Two shapes, and this SDK is the second one

Robotics clients come in two shapes. In the first, a framework owns the program: you subclass something, fill in setup() and update(), hand the class to a runner, and the runner calls you. In the second, you own the program: main() does its setup and then runs a plain loop, calling the library when it wants something.

The first is the Arduino shape. This SDK is deliberately the second, the STM32 shape.

flowchart TB
  subgraph AR["Arduino shape: the framework calls you"]
    direction TB
    A1["runner starts"] --> A2["your setup"]
    A2 --> A3["your update callback"]
    A3 --> A4["runner decides when"]
    A4 --> A3
  end
  subgraph ST["STM32 shape: you call the SDK"]
    direction TB
    B1["main starts"] --> B2["connect"]
    B2 --> B3["read states"]
    B3 --> B4["your control law"]
    B4 --> B5["send command"]
    B5 --> B6["rate or wait_new_state"]
    B6 --> B3
  end

There is no base class, no runner, no setup()/update() pair, and no registration call. The SDK never calls your code. Every arrow in the right-hand diagram is a call your main makes.

The canonical program

This is the whole shape, and every example in the book is a variation on it. From book/.research/00-shared.md, which quotes the crate's own documentation in crates/vrobots-sdk/src/lib.rs:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

fn main() -> Result<(), VrError> {
    // ===== setup =====
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(1))?;

    // ===== loop =====
    loop {
        let s = robot.states();               // latest snapshot, never torn, never blocks
        let [x, y, z] = s.kin.lin_pos;
        println!("State t={:.3} pos=({x:.3},{y:.2},{z:.2})", s.elapsed);

        robot.set_mr_pwm([1501.0; 4])?;       // your controller's output
        robot.rate(100.0);                    // drift-compensated pacing, Hz
    }
}

#include <vrobots_sdk.hpp>

int main() {
    vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, /*sys_id=*/1);
    robot.connect();
    while (true) {
        vrsdk::State s = robot.states();
        std::printf("t=%.3f\n", s.elapsed);
        robot.set_mr_pwm({1501, 1501, 1501, 1501});
        robot.rate(100.0);
    }
}

from vrsdk import VirtualRobot, RobotType

def main():
    # ===== setup =====
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=1)
    mr.connect()

    # ===== loop =====
    while True:
        s = mr.states
        x, y, z = s.kin.lin_pos
        print(f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f})")
        mr.set_mr_pwm(1501, 1501, 1501, 1501)
        mr.rate(100)

if __name__ == "__main__":
    main()

Each surface documents this same shape as its own opening example, which is the clearest evidence that the shape is the API rather than a Rust convention. The one structural difference is that C++ and Python split construction from connection into two statements, where Rust's connect is a constructor that does both.

There is no expected output to quote: the loop never terminates, printing one line per iteration until you stop it with Ctrl-C. examples/rust/src/bin/ex01_hello_states.rs is this program with the printing worked out, and it is the subject of Hello states.

Read the four calls in the loop as four separate decisions you are making:

Why this shape

Three reasons, in the order they matter.

A control loop has a rate, and it is yours. A callback framework decides when your code runs. A cascaded controller with an inner loop at 200 Hz and an outer at 20 Hz does not fit that, and neither does a program that wants to run as fast as samples arrive. Owning the loop means the rate is a line of your code: rate for a fixed schedule, wait_new_state for one iteration per published sample.

It is the shape the target hardware has. Code written against this SDK is meant to move to a real vehicle, where main is a while (1) over a timer. Keeping the simulator client the same shape means the loop body ports; a callback body does not.

There is nothing to learn. No lifecycle, no ordering rules between setup and the first update, no question about which thread a callback runs on. The SDK does own background threads (one subscriber per state stream, one reader per camera stream), and none of them ever enters your code. They keep a snapshot fresh; you read it when you like.

Note. One process is not limited to one robot. A VirtualRobot is a handle, so construct as many as you need, each with its own session and snapshot. Call rate() on exactly one of them, or you sleep once per handle per iteration. examples/rust/src/bin/ex18_multi_robot.rs is the worked case.

Dropping the handle does not stop the robot

Because main owns the program, it is tempting to read the end of main as the end of everything. It is not. Dropping a VirtualRobot closes the zenoh session, unsubscribes, and stops the SDK's own threads. It leaves the robot running, still latched on the last command it received, until something explicitly deletes it or the scene is reloaded.

That is the intended behaviour rather than an oversight, and it is rule four. If your program should leave nothing behind, call delete() before returning, which is what the examples that create a robot do.

Next: What connect actually does

See also: Pacing your loop, More than one robot, Robot lifecycle

What connect actually does

The four steps behind a create, why only one of them is sent once, and every option you can change.

One call, two paths

There is one entry point, and a second that takes options. The two signatures, from crates/vrobots-sdk/src/robot.rs:

#![allow(unused)]
fn main() {
pub fn connect(robot_type: RobotType, sys_id: Option<u32>) -> VrResult<VirtualRobot>

pub fn connect_with(
    robot_type: RobotType,
    sys_id: Option<u32>,
    options: ConnectOptions,
) -> VrResult<VirtualRobot>
}

explicit VirtualRobot(RobotType type, std::uint32_t sys_id,
                      const vrsdk_connect_options_t* options = nullptr)

[[nodiscard]] static VirtualRobot create(RobotType type,
                                         const vrsdk_connect_options_t* options = nullptr)

void connect()

class VirtualRobot:
    def __init__(
        self,
        robot_type: RobotType,
        sys_id: Optional[int] = None,
        *,
        src_id: Optional[int] = None,
        router: Optional[str] = None,
        connect_timeout: Optional[float] = None,
        probe_timeout: Optional[float] = None,
        service_timeout: Optional[float] = None,
        camera_timeout: Optional[float] = None,
        robot_name: Optional[str] = None,
        client_name: Optional[str] = None,
        start_active: Optional[bool] = None,
        activate_after_create: Optional[bool] = None,
        coord_frame_id: Optional[str] = None,
        axis_convention: Optional[int] = None,
    ) -> None: ...
    def connect(self) -> None: ...

The three surfaces split the same work differently. Rust folds construction and connection into one call and takes options as a second entry point; C++ and Python construct first and connect() second, and neither has a connect_with because options ride on construction: C++ takes a pointer to the C options struct (null for the defaults) and Python takes the same fields as keyword arguments. C++ also needs a named create factory for the None path, because a literal 0 would be ambiguous between "attach to sys_id 0" and a null options pointer.

connect is connect_with with ConnectOptions::default(). Both open a zenoh session first, and both end by subscribing the robot's state topic and blocking until the first snapshot arrives, so states() is guaranteed to return real data the instant connect returns. What differs is the middle.

Some(id) attaches. It never touches srv/create, so it works for any robot the scene contains whatever the spawn catalog says, and the only thing that can go wrong is that nothing publishes on that id: VrError::Timeout after probe_timeout.

None creates, in four steps.

The four-step create path

sequenceDiagram
  participant P as Your program
  participant M as manager
  participant R as New robot
  P->>M: 1. payload-less GET on srv/create
  M-->>P: ack, reserves nothing
  P->>M: 2. the real create, sent exactly once
  M-->>P: ack carrying the new sys_id
  P->>R: 3. srv/activate, retried
  R-->>P: ack
  R->>P: 4. first sample on z/state
  Note over P,R: connect returns here

Step 1 is a reachability probe. A payload-less GET on the create endpoint is answered ok = false and reserves nothing, so it is free to send and free to retry. It answers one question: is the manager there at all? Without it, a manager that is absent and a manager that is slow are indistinguishable at the moment it matters.

Step 2 is the create, sent exactly once. This is the only non-idempotent service in the system. The manager allocates an id and spawns a robot for every request that reaches it, so a retry that lands leaves you with two robots and a handle to one of them. The SDK therefore sends this one query with no retry loop at all, and a timeout here is reported rather than papered over. A refusal comes back as VrError::Service carrying the simulator's own message, which lists the catalog: see System ids, and the two kinds of robot.

Step 3 activates, with retry. srv/activate releases a dormant robot's dynamics hold, and an already-active robot acks and no-ops, so it is idempotent and safe to re-send. It needs the retry: a robot created a moment ago takes about a second before zenoh has discovered its srv/* queryables, and the first attempt often finds nobody. This step is skipped when activate_after_create is false.

Step 4 waits for the robot to publish. An ok create reply means the id is allocated, not that a robot exists in the scene. The SDK waits for the state topic to produce a sample, so "spawned but never published" fails at connect rather than three lines later at the first states() call.

Note. Confirmation is by presence throughout this system, not by acknowledgement. connect returns when the state topic speaks; delete returns when it has been silent for a second. The ack in between is a receipt that the request arrived. That is rule two, and it is the same reason configuration services are confirmed by measuring the state stream.

Retry, and the one place it is forbidden

Almost every service in the SDK is queried through a retry loop, because a GET that finds no responder yet is the normal state of affairs for a second or so after a robot appears. Two are deliberately excluded.

ConnectOptions

Every timeout, the router endpoint and the identity live in one struct with documented defaults. It is #[non_exhaustive], so build it with ConnectOptions::default() and chain the with_* setters.

The three most commonly changed, chained. From the doctest on ConnectOptions in crates/vrobots-sdk/src/options.rs:

#![allow(unused)]
fn main() {
let opts = ConnectOptions::default()
    .with_router("tcp/192.168.1.10:7447")   // sim on another machine
    .with_src_id(200)                       // second client in the session
    .with_probe_timeout(Duration::from_secs(20));
}

vrsdk_connect_options_t options{};
vrsdk_options_default(&options);
options.service_timeout_s = PROBE_TIMEOUT_S;

vrsdk::VirtualRobot robot(vrsdk::RobotType::HalfDrone, sys_id, &options);
robot.connect();

mr = VirtualRobot(
    RobotType.MULTIROTOR,
    sys_id=1,
    router="tcp/192.168.1.10:7447",   # sim on another machine
    src_id=200,                       # second client in the session
    probe_timeout=20.0,
)

The table above is the reference for all three, with two spelling changes. C++ carries the plain C struct, so the fields are router_endpoint, connect_timeout_s, probe_timeout_s and so on, and it must be initialised with vrsdk_options_default: a zeroed struct is not the defaults, it is every timeout set to zero. Python takes them as keyword arguments, named without the _s suffix, and omitting one means "keep the default" rather than "set it to zero".

Rust constructs a value and produces no output; the C++ and Python forms above connect with it.

src_id is worth one more sentence. It identifies your traffic, so two clients sharing a session that both leave it at 122 are indistinguishable in a topic dump, and a program reading the command bus cannot filter out its own publishes. Give the second client its own.

Next: Five rules that explain everything

See also: Robot lifecycle, When nothing happens, Reading someone else's commands

Five rules that explain everything

The behaviours that account for almost every confused bug report against this SDK.

Every rule below is deliberate, and every one of them has a failure mode that looks like something else. Learn the detection column and the rest of the book gets easier.

Rule 1: commands latch and get no reply

What it means. Every command is a setpoint, not an impulse. The last one received stays in effect until the next arrives. There is no watchdog and no expiry, so a controller that stops sending leaves the robot flying its final command, and a 5 Hz sender and a 50 Hz sender are equally valid. Nothing is acknowledged: publishing a command returns as soon as the bytes are queued.

Why. A command topic is a bus, not a call. Making it a call would mean the simulator blocking on each client, and making commands expire would mean a control loop that stutters becomes a robot that falls out of the sky. Latching is what real actuator interfaces do.

How you detect the failure. The only evidence a command landed is the actuator echo in the state stream. actuator.pwm is your last command echoed back, so a command that never arrived shows as an echo that never changes. From examples/rust/src/bin/ex02_hello_control.rs:

#![allow(unused)]
fn main() {
let s = robot.states();
let [x, y, z] = s.kin.lin_pos;
println!(
    "State t={:.3} pos=({x:.3},{y:.2},{z:.2}) echo={:?}",
    s.elapsed, s.actuator.pwm
);
}

// The echo: what the robot actually latched, from the state stream.
const std::vector<std::uint32_t> echo = s.pwm();
std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)  pwm_echo=[", s.elapsed, p[0], p[1],
            p[2]);
for (std::size_t i = 0; i < echo.size(); ++i) {
    std::printf("%s%u", i ? "," : "", echo[i]);
}

s = mr.states
x, y, z = s.kin.lin_pos
print(
    f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f}) "
    f"echo={s.actuator.pwm}"
)

Running that prints one line per iteration in which echo holds the pulse widths the robot is currently applying. A wrong sys_id, a command the robot does not implement, and a wrong array length all present identically: the echo does not move.

Rule 2: an ack is a receipt, not a result

What it means. Every srv/* reply is packed the instant the request arrives. The change itself lands in phase 0 of the robot's next physics step, after the reply is already on its way back to you. ok = true therefore means "the request was received", and nothing more.

Why. The service handler runs on the network thread; the change has to run on the physics thread, at a point in the step where applying it is safe. Waiting for that before replying would put a physics frame of latency into every service call.

How you detect the failure. By measuring, because the ack will not tell you. Only srv/skin ever answers ok = false. A wrong rotor count, an unknown frame id, a drive_mode that is not 2 or 4: all acked ok and refused by a simulator log line no client can see. The SDK refuses what it can before sending, which is what every VrError::InvalidArgument from a configuration call is. For the rest, read the state stream back and check the robot behaves differently. That is why the configuration examples measure rather than assert.

Gotcha. A typo in a skin name on a robot that has a skin catalog is acked ok and dropped, so it looks exactly like success. Only a tier refusal produces ok = false. See Skins.

One physics step, in order

Both rules above are consequences of one sequence. This is what happens between two state samples:

sequenceDiagram
  participant C as Your program
  participant Q as Command queue
  participant S as Physics step
  C->>Q: publish on z/cmd, no reply
  Note over Q: last value wins, and waits
  S->>Q: phase 0, drain the queue
  S->>S: phase 0, apply commands and service changes
  S->>S: integrate the dynamics
  S-->>C: publish z/state, actuator echo included

Physics runs at 50 Hz and state is published at 25 Hz, so a loop faster than 25 Hz reads the same snapshot more than once, and a loop faster than 50 Hz sends commands that are overwritten in the queue before anything drains it. Neither is an error. Pacing your loop is where you decide which rate you actually want.

Rule 3: states() never fails

What it means. states() never blocks, never returns a half-written value, and has no error case. An SDK-owned subscriber keeps a snapshot fresh in the background and states() hands you an Arc of the most recent one. If the simulator stops, it keeps returning that last snapshot forever.

Why. A control loop that has to handle an error on every sensor read is a control loop full of error handling. Making the read infallible pushes the one real question, "is this data still arriving?", to the one place that should ask it.

How you detect the failure. With wait_new_state(timeout), which blocks for a sample newer than the current one and returns VrError::Timeout when none comes. A timeout is a status, not a failure: the session is fine and the next call may well succeed. From examples/rust/src/bin/ex09_state_paced_loop.rs:

#![allow(unused)]
fn main() {
Err(VrError::Timeout(detail)) => {
    // Not a broken session: no sample arrived in time. The sim is
    // paused, stopped, or the machine is very busy. states() still
    // returns the last snapshot it had.
    let s = robot.states();
    println!(
        "no new state in {:?} ({detail}); still holding seq={} at t={:.3}",
        TIMEOUT, s.seq, s.elapsed
    );
}
}

try {
    robot.wait_new_state(TIMEOUT_S);
} catch (const vrsdk::Error& e) {
    if (e.code() != VRSDK_ERR_TIMEOUT) {
        throw;  // a real failure
    }
    // Not a broken session: no sample arrived in time. The sim is
    // paused, stopped, or the machine is very busy. states() still
    // returns the last snapshot it had.
    const vrsdk::State s = robot.states();
    std::printf("no new state in %.1fs; still holding seq=%llu at t=%.3f\n", TIMEOUT_S,
                static_cast<unsigned long long>(s.seq), s.elapsed);
    continue;
}

try:
    mr.wait_new_state(TIMEOUT)
except vrsdk.VrError as e:
    if e.code != vrsdk.err.TIMEOUT:
        raise  # a real failure
    # Not a broken session: no sample arrived in time. The sim is
    # paused, stopped, or the machine is very busy. `states` still
    # returns the last snapshot it had.
    s = mr.states
    print(
        f"no new state in {TIMEOUT}s ({e.detail}); "
        f"still holding seq={s.seq} at t={s.elapsed:.3f}"
    )
    continue

Rust distinguishes the timeout by matching the VrError::Timeout variant. C++ and Python have one error type each, so they branch on the numeric code and re-raise anything else: e.code() != VRSDK_ERR_TIMEOUT and e.code != vrsdk.err.TIMEOUT are the same test, and it is the same stable number in both.

That arm prints a line each time the simulator goes quiet and keeps the loop alive. Propagating the timeout out of main with ? instead is what turns a paused simulator into a crashed program. stats() gives the same answer cumulatively: received stops climbing, and seq_gaps counts the samples that went missing while the stream was still up. Stream health covers both.

Rule 4: robots outlive the process

What it means. A VirtualRobot is a handle to something that already existed or that you asked to be created, and dropping it closes your session and nothing else. The robot keeps running, keeps publishing, and keeps its last latched command. Deletion is explicit: only delete() removes a robot.

Why. The simulator is a persistent world, not a library object. A client that crashes mid-flight should not take the scene down with it, and a program that attaches to a scene-authored robot must not be able to delete it by exiting.

How you detect the failure. vrobots topic list shows every robot that is publishing. A scene that accumulates robots across runs is a program creating and not deleting; a robot that keeps moving after your program exits is rule 1 plus rule 4, and the fix is to send a stop command before returning rather than to hope the drop does it. is_deleted() reports whether this handle's robot was deleted, and a spent handle returns an error from commands rather than silently doing nothing.

Rule 5: every vector is in the robot's frame, not yours

What it means. Vectors on the wire are expressed in the frame the publisher names in coord_frame_id, and different robots genuinely disagree. Your outgoing headers carry your own frame (ConnectOptions::coord_frame_id, "unity" by default) and the robot converts before acting, but nothing converts what you read.

Why. There is no single correct convention. Aerospace wants forward-right-down, the renderer works in Unity's left-handed frame, and computer vision wants right-down-forward. Rather than picking one and silently converting, the simulator tags every message with the frame it used.

How you detect the failure. Print coord_frame_id next to every vector you print. The signature is a sign error: an altitude that goes negative as the aircraft climbs, or a distance between two robots that is wrong in one axis. examples/rust/src/bin/ex18_multi_robot.rs holds two robots whose frames differ and labels the computed separation as wrong for exactly this reason. Frames, axes and units is the full treatment.

Two more that are almost rules

Decode errors are counted, not fatal. A payload that fails to decode does not tear down the session or interrupt the loop. It increments stats().decode_errors and is retrievable through last_error(), and the next sample is handled normally. A stream that is producing numbers and also producing decode errors is a version-skew symptom, and it is invisible unless you look at the counters.

State and camera streams are never paired. They are independent streams on independent transports with independent rates, and the SDK does not associate a frame with a snapshot. Both carry t_ns on the same clock, so fusion code compares those timestamps explicitly and decides for itself what counts as simultaneous. Freshness has the mechanics.

Next: Frames, axes and units

See also: Commands latch, Stream health, Robot lifecycle, When nothing happens

Frames, axes and units

Which vector is in which frame, what coord_frame_id decides, and the conventions that bite.

Everything is SI

Metres, seconds, kilograms, newtons, radians, pascals, degrees Celsius. There are no scaled integers anywhere in the state stream: this SDK publishes physical units directly, so a gyroscope reading is rad/s rather than a count to be divided by something.

Three places break the pattern, and each is called out where it appears: mount_euler_deg on a camera and initial_pole_angle_deg on a cart-pole are in degrees because the simulator's inspector shows degrees, and pulse widths are in microseconds because that is what a PWM channel is.

The wire carries f32. The SDK widens every value to f64 on the way in, which is lossless, and narrows to f32 on the way out, which is not. A setpoint you send and read back through the actuator echo can differ in the seventh significant figure. It is never a control problem and it is occasionally an equality-comparison problem.

Axes is an integer, deliberately

The declaration is the argument. From crates/vrobots-sdk/src/state.rs:

#![allow(unused)]
fn main() {
#[repr(transparent)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
pub struct Axes(pub i32);
}

/**
 * The axis convention a vector is expressed in -- the enum tag beside
 * `coord_frame_id`, which is the authoritative name.
 */
typedef int32_t vrsdk_axes_t;

class State:
    @property
    def axis_convention(self) -> int: ...
    @property
    def axis_convention_name(self) -> str: ...

def axes_name(value: int) -> str: ...

The openness survives every binding. C++ gets a typedef over int32_t rather than an enum class, which is the same decision for the same reason, and Python reports it as a plain int with axes_name to turn one into a label. It is the one place in the SDK where C++ does not wrap a C type in something stronger.

That is a type rather than a program, so there is no output to show. Four constants are defined on it:

Python has no constants for these: compare the integer, or read axis_convention_name and compare the string.

It is not a Rust enum, and the reason is that the set is open. A scene can register a coordinate frame at runtime, and such a frame has no constant here. A Rust enum would force every unknown value to collapse into a catch-all on decode, so a frame the SDK has never heard of would round-trip as "unspecified" and the information would be gone. A transparent i32 carries whatever the simulator sent, whether or not this build knows a name for it, which also keeps the type trivially FFI-safe for the C++ and Python bindings.

Axes::name() returns the registry id string for a known constant ("unity", "frd", "cv") and "" for anything else, including a runtime-registered frame.

coord_frame_id is the authoritative one

Every message carries both a string frame id and an Axes tag. The string wins. It is the only thing that can name a runtime-registered frame, and Axes is a convenience tag beside it for the cases where a numeric comparison is easier.

"fru" is a registered id with no Axes constant, and it is the scene default on a fresh launch, which makes this concrete rather than theoretical: a robot can publish a frame whose Axes tag is UNSPECIFIED and whose string is meaningful.

Frames are set at three levels, scene, robot and device, and the most specific wins. scene_frame() reads the scene level; srv/frames sets the robot and device levels. Coordinate frames is the whole picture.

The robots do not agree with each other. The truck publishes "fru", while the multirotor and the Global Hawk publish "frd" (the Global Hawk verified live on 2026-08-09 against simulator v3.0.0), so the third component is up for one and down for the others, and altitude on the multirotor is minus that component.

Gotcha. Two handles in one process is the program where this bites hardest, because a distance between two positions in different frames is silently wrong rather than visibly wrong. Read coord_frame_id on each snapshot and convert before combining. examples/rust/src/bin/ex18_multi_robot.rs prints the tag beside each position and marks its own separation calculation as wrong when the two disagree.

Your frame and the robot's frame are different questions

ConnectOptions::coord_frame_id ("unity" by default) names the convention your outgoing vectors are written in. The robot converts an incoming vector out of that frame into its own before acting on it, so a body force sent in Unity axes does the right thing on a robot publishing frd.

Nothing converts what you read. Incoming state is in the robot's frame, tagged with the robot's coord_frame_id, and that is the asymmetry to keep in mind: commands are converted for you, states are not.

Quaternion order is [x, y, z, w]

The scalar part is last. This matches the wire's Vec4 field order, and it is the opposite of the [w, x, y, z] convention that most textbooks and several quaternion libraries use. Feeding an array in the wrong order into a rotation routine does not error; it produces a rotation that looks almost plausible, which is the worst possible failure mode. Check the order at every boundary where a quaternion enters or leaves the SDK.

The world-pose, body-twist split

Within Kinematics, half the fields are world frame and half are body frame:

This is physics, not a configuration choice, and there is no option that changes it. A position is only meaningful relative to some fixed origin, so pose is world. A velocity or an acceleration is what an onboard instrument measures, and instruments are bolted to the vehicle, so twist is body: ang_vel is what a gyroscope reads, which is body rates rather than Euler angle derivatives.

The same split holds in the estimate block, which carries a Kinematics of the same shape, so a comparison between estimate.kin and kin is field-by-field valid without any frame juggling. Kinematics works through what that means for each field.

Next: Rotation conversions

See also: Coordinate frames, Kinematics, More than one robot

Rotation conversions

Turning an attitude between quaternions, angles and matrices, and re-expressing a whole state in another frame.

cargo run -p vrobots-examples --bin ex36_rotations
./target/cpp-build/ex36_rotations
python examples/python/ex36_rotations.py

vrobots_sdk::rotations talks to nothing: pure math, shared with the C++ namespace vrsdk::rotations and the Python submodule vrsdk.rotations, so "convert this quaternion to roll, pitch and yaw" has one answer. ex36 creates its own truck, drives an arc so there is something non-zero to convert, and deletes it, so it takes no sys_id. Frames, axes and units owns the storage conventions; this page is the arithmetic on top of them.

Storage order is fixed; application order is an argument

Storage never changes. An Euler triple is [about x, about y, about z], reading as [roll, pitch, yaw] in a body frame whatever sequence the rotations are applied in; a quaternion is [x, y, z, w], scalar last; a matrix is row-major 3x3, so r[1][2] is row 1, column 2.

Application is the EulerOrder argument. Zyx applies yaw first, then pitch, then roll, and reads euler[2], euler[1], euler[0] in that sequence. It never reorders the array, so euler_to_quat([roll, pitch, yaw], Zyx) and the same triple under Xyz build different rotations from identical numbers. There is no [yaw, pitch, roll] layout here.

The values are swarmbotix.coordinates.EulerOrder's, so to_wire() is a cast. Zero is EULER_ORDER_UNSPECIFIED, not an order and with no variant here. Python spells the variants EulerOrder.ZYX; Rust and C++ spell them EulerOrder::Zyx.

Gotcha. vrobots_sdk::EulerOrder and rotations::EulerOrder are different types with the same name: the first is the wire tag, a transparent i32 keeping whatever a publisher sent, the second the math parameter, closed because a conversion handed a value that names no order has nothing correct to do. Cross with EulerOrder::from_wire(tag.0) and handle the None.

The six conversions

A quaternion here is Hamilton, unit and body-to-world: rotate_vec3 takes body components and returns world components. Euler angles are intrinsic Tait-Bryan, each rotation about an axis of the frame the previous one produced, matching the simulator.

Around them sit quat_multiply, quat_conjugate, quat_normalize, rotate_vec3, rotmat_multiply, rotmat_transpose, rotmat_det, rotmat_abs and rotmat_apply. quat_multiply(a, b) is "b first, then a", which is why ex35 post-multiplies to bias about a body axis. All are pure and total: no panics, no NaN out of finite input, no allocation, and a zero-length quaternion normalizes to IDENTITY_QUAT rather than erroring.

A frame is nine numbers, and the wire carries them

A convention is one signed-permutation matrix R whose rows are the Unity axes that map onto that frame's axes, so R * v re-expresses a Unity vector in frame components. That is what z/frames publishes, and it is all a peer needs: the inverse is the transpose, the handedness det(R), and north, east and down are R on the Unity world anchor.

AxisBasis is that matrix plus the frame's reporting order; FrameTransform is the R_to * R_from^T product between two of them. Both have constructors for the four built-ins, and AxisBasis::from_frame_def builds one from a definition read off the wire, the only route that reaches a frame with no Axes constant. "fru" is that case today, and a scene registering a convention at runtime is the same problem further out.

ex36 reads the definition with frame_def() and turns it into a basis. From examples/rust/src/bin/ex36_rotations.rs:

#![allow(unused)]
fn main() {
    let basis = AxisBasis::from_frame_def(&def).ok_or_else(|| {
        VrError::InvalidArgument(format!(
            "frame {:?} names no euler order and its axis convention has no built-in default, \
             so there is no order to report angles in",
            def.id
        ))
    })?;
}

// Throws when the definition names no order and its axis convention has
// no built-in default: there is then no order to report angles in.
const vrsdk::rotations::AxisBasis basis = vrsdk::rotations::AxisBasis::from_frame_def(def);

    basis = rotations.AxisBasis.from_frame_def(fdef)
    if basis is None:
        raise SystemExit(
            f"frame {fdef.id!r} names no euler order and its axis convention has no "
            "built-in default, so there is no order to report angles in"
        )

Rust returns Option, Python None and C++ throws, all saying the same thing: a basis with a guessed order extracts angles that look plausible and mean nothing. For the truck's "fru" the call succeeds, and the line under it reads

  det(R)=+1, right-handed=false

Gotcha. det(R) = -1 means right-handed, not left: the determinant is of the map out of left-handed Unity, so a right-handed frame is the one that flips the sign. is_right_handed() exists so nothing has to remember that.

Which rule applies depends on what the vector is

M * v is right for a polar vector and wrong for an axial one. An axial vector is defined by a cross product, so it picks up an extra sign when the basis flips handedness. The simulator splits its conversions the same way, so this is a rule the SDK agrees with rather than one it chose.

Between two frames of the same handedness det(M) = +1 and the first two rules coincide, which is why picking the wrong one survives testing until somebody crosses a flip. ex36 converts the live gyro both ways. From examples/rust/src/bin/ex36_rotations.rs:

#![allow(unused)]
fn main() {
    // axial: the gyro, and the mistake beside it
    let gyro = state.sensors.gyroscope.angular_velocity;
    let gyro_frd = t.apply_axial_vec3(gyro);
    let gyro_wrong = t.apply_vec3(gyro);
}

// axial: the gyro, and the mistake beside it
const vrsdk::Vec3 gyro = gyro_of(state);
const vrsdk::Vec3 gyro_frd = t.apply_axial_vec3(gyro);
const vrsdk::Vec3 gyro_wrong = t.apply_vec3(gyro);

    # axial: the gyro, and the mistake beside it
    gyro = state.sensors.gyroscope.angular_velocity
    gyro_frd = t.apply_axial_vec3(gyro)
    gyro_wrong = t.apply_vec3(gyro)

The truck reports in "fru" and the target is "frd", so M is diag(1, 1, -1) and det(M) = -1: every component of the axial answer is the negation of the polar one, and a body rate converted with the polar rule sends the robot spinning the other way. A roll rate of [0, 0, 1] in Unity axes converts to [-1, 0, 0] under apply_axial_vec3 and to [+1, 0, 0] under apply_vec3, which is the whole trap in two lines.

For a pair the Axes tags can name, convert_vec3, convert_axial_vec3, convert_inertia_vec3, convert_quat, convert_rotmat and convert_euler do the same in one call, each returning VrResult. Handed the truck's own axis_convention, which is UNSPECIFIED, they fail with InvalidArgument naming the three constants that exist. That is why frame_def() is there: the tag is a convenience and the nine numbers are the fact.

Gimbal lock has an answer, and it is the simulator's

With the middle angle at a quarter turn the two outer rotations act about the same axis and only their difference survives, so no unique triple is left. rotmat_to_euler and quat_to_euler return one anyway: the angle about z is pinned to zero, because heading is the meaningless quantity when the nose is vertical, and the whole determined combination goes into the other outer angle. Where z is itself the middle axis and cannot be pinned, the last angle of the sequence is zeroed instead. That matches CoordFrame.MatrixToEulerDeg, so a locked attitude decodes here to the triple the simulator shows.

ex36's last section builds an attitude from LOCKED_DEG, which is (0, 90, 40) degrees with the pitch exactly at the pole, and asks for the angles back through the same euler_to_quat and quat_to_euler pair the table above lists:

  in    roll=  +0.00 deg  pitch= +90.00 deg  yaw= +40.00 deg
  out   roll= -40.00 deg  pitch= +90.00 deg  yaw=  +0.00 deg

Different numbers, same rotation: the example rebuilds the matrix from the extracted triple and checks it element by element. A naive atan2 pair returns neither answer, because both its terms are rounding noise at the pole, which is how (0, 90, 40) comes back from a hand-inlined formula as (26.6, 90, 90).

Next: Reading state

See also: Frames, axes and units, Coordinate frames, Publishing estimates

Reading state

How one state snapshot is put together, and which page of this chapter describes each part of it.

The snapshot model

The simulator publishes one complete state sample per robot at 25 Hz. The SDK subscribes in the background, decodes each sample into an owned snapshot and stores it. You never poll a queue and you never register a callback: you read the latest snapshot whenever your loop wants it.

That read is the narrowest API in the SDK.

From crates/vrobots-sdk/src/robot.rs:

#![allow(unused)]
fn main() {
pub fn states(&self) -> Arc<State> {
    self.channel.snapshot.load_full()
}
}

[[nodiscard]] State states() const {
    vrsdk_state_t raw{};
    detail::check(vrsdk_robot_states(require(), &raw), "states");
    return State::from_raw(raw);
}

class VirtualRobot:
    @property
    def states(self) -> State: ...

The three differ only in how the copy is made and named. Rust hands back a reference-counted clone, so reading is a pointer bump. C++ copies the C struct into a value you can store and pass to another thread, and Python builds a State object. In Python it is a property, so it is mr.states with no parentheses; forgetting that is the most common transcription error when porting a loop from one of the other two.

This is a signature rather than a program: it returns immediately with the most recent decoded sample, hands you a reference-counted clone, and cannot fail. A State is a plain owned struct of fixed-size arrays and Vecs, with no borrows into the receive buffer, so it outlives the sample it came from and crosses to C++ and Python as a memory copy.

Three consequences follow, and the rest of the chapter is mostly their detail:

• A snapshot is never torn. You either see the previous sample in full or the next one in full, never half of each.
• A snapshot is never absent. connect blocks for the first sample before it returns, so states() is valid immediately afterwards.
• A snapshot is never an error. If the simulator stops, states() keeps returning the last sample it had, unchanged, forever. Detecting that requires wait_new_state, not an error check.

What one snapshot contains

Every sample carries a header, then four blocks that differ in what a real robot could know about itself.

flowchart TD
  S["State (one sample)"]
  S --> H["header: t_ns, elapsed, seq, sys_id, coord_frame_id"]
  S --> T["truth: simulator-exact"]
  S --> M["measured: robot-observable"]
  S --> B["believed: the robot's filter"]
  S --> A["actuator: none of the three"]
  T --> K["kin: pose, twist, accel"]
  T --> W["wrench: force, torque"]
  T --> E["env: gravity, air, geo, agl"]
  M --> SN["sensors: accel, gyro, mag, baro, gnss, flow"]
  B --> ES["estimate: kin, valid, timestamp"]
  A --> AC["pwm, normalized, measured"]

The split is the schema's whole purpose. kin, wrench and env are values no physical vehicle could measure. sensors is the noisy view of the same instant. estimate is what the robot's own filter believes. actuator belongs to none of them, because it is the command going out and the realised motion coming back.

The minimal read loop

The smallest useful program takes two fields out of the snapshot and paces itself.

From examples/rust/src/bin/ex01_hello_states.rs:

#![allow(unused)]
fn main() {
loop {
    let s = robot.states(); // immutable latest snapshot, never torn
    let [x, y, z] = s.kin.lin_pos;
    println!("State t={:.3} pos=({x:.3},{y:.2},{z:.2})", s.elapsed);
    robot.rate(HZ); // drift-compensated pacing, Hz
}
}

for (;;) {
    const vrsdk::State s = robot.states();  // latest snapshot, never torn
    const double* p = s.kin().lin_pos;
    std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)\n", s.elapsed, p[0], p[1], p[2]);
    robot.rate(HZ);  // drift-compensated pacing, Hz
}

while True:
    s = mr.states  # immutable latest snapshot, never torn
    x, y, z = s.kin.lin_pos
    print(f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f})")
    mr.rate(HZ)  # drift-compensated pacing, Hz

Rust and Python destructure the position into three names; C++ takes a pointer to the three-element array, because lin_pos is a plain C double[3] there.

At the example's 50 Hz against a 25 Hz stream, consecutive lines repeat the same sample about half the time, which is correct and costs nothing:

State t=12.480 pos=(0.031,0.85,-1.20)
State t=12.480 pos=(0.031,0.85,-1.20)
State t=12.520 pos=(0.032,0.85,-1.21)

Note. Reading the same snapshot twice is free and is not a bug. If duplicate processing would be a bug, for example when you differentiate or log, pace on the data instead: see Pacing your loop.

The rest of this chapter

Next: Truth, measured and believed

See also: Hello states, The shape of a program, Five rules that explain everything

Truth, measured and believed

The three epistemic categories a snapshot keeps apart, and the two differences between them that are worth measuring.

Why the schema separates them

A simulator can tell you exactly where a robot is. A robot cannot know that about itself. If both numbers live in the same struct under similar names, a control loop that accidentally reads the exact one works beautifully in simulation and fails on hardware, and nothing in the code looks wrong.

The message schema prevents that by construction, and the SDK's State mirrors it: truth, measurement and belief sit in separate blocks, and no field appears in two of them. Choosing a block is therefore a deliberate act. If your controller reads kin.lin_pos, you have decided to use ground truth, and that decision is visible in the source.

actuator is listed as none of the three on purpose. It is not a measurement of the world and not a belief about it: it is the echo of what you commanded beside what the device did. It has its own page, Actuators.

The two differences are the experiment

Because the blocks are published from the same instant, characterising a sensor or an estimator is a subtraction between two fields of one snapshot. Nothing has to be inferred, and no separate ground-truth log has to be aligned in time.

The SDK source states two of these outright:

The same construction extends to every other device. sensors.gyroscope.angular_velocity - kin.ang_vel is the gyro's noise realisation on that sample, because both are body-frame angular rates in rad/s. sensors.gnss.geo_point against env.geo_point is the receiver's position error.

Two cautions apply to every such diff. Both sides must be in the same frame, which is not automatic when a device overrides the robot's axis_convention (see Sensors). And the measured side must be fresh: a sensor slower than the state stream republishes its previous reading, so a diff taken every sample measures the same noise realisation several times over. Compare timestamps first, as Timestamps and sequence numbers describes.

Reading the believed block

estimate has the same Kinematics shape as the truth block, plus its own clock, its own frame, and a validity flag.

From examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
// -- believed -------------------------------------------------------
let e = &s.estimate;
println!(
    "BELIEVED  estimate  {}  frame={:?}",
    stamp(e.valid, e.timestamp),
    e.coord_frame_id
);
println!(
    "  lin_pos   {} m       (estimate.kin - kin IS the error)",
    v3(e.kin.lin_pos)
);
}

// -- believed -----------------------------------------------------
std::printf("BELIEVED  estimate  ");
stamp(r.estimate.valid, r.estimate.timestamp);
// The fixed C char arrays are NUL-terminated; the precision bounds
// the read even if a future field ever fills the buffer exactly.
std::printf("  frame=\"%.*s\"\n",
            static_cast<int>(sizeof r.estimate.coord_frame_id - 1),
            r.estimate.coord_frame_id);
v3("lin_pos", r.estimate.kin.lin_pos, "m", "(estimate.kin - kin IS the error)");
v3("lin_vel", r.estimate.kin.lin_vel, "m/s", "");

# -- believed -------------------------------------------------------
e = s.estimate
print(
    f"BELIEVED  estimate  {stamp(e.valid, e.timestamp)}  "
    f"frame={e.coord_frame_id!r}"
)
print(f"  lin_pos   {v3(e.kin.lin_pos)} m       (estimate.kin - kin IS the error)")
print(f"  lin_vel   {v3(e.kin.lin_vel)} m/s")

C++ reaches the block through s.raw, the copied C struct, where the other two have named fields on the snapshot: r.estimate.kin.lin_pos against s.estimate.kin.lin_pos. The fields, the units and the subtraction that gives the estimator error are identical.

Today that always prints an invalid stamp, zeroed vectors and an empty frame id:

BELIEVED  estimate  [INVALID t=0.000]  frame=""
  lin_pos   (  +0.000,  +0.000,  +0.000) m       (estimate.kin - kin IS the error)

It is not a decode failure and not a dropped block: a missing nested table decodes to its Default, which is all zeros, false and empty strings. The simulator runs no estimator and omits the field on purpose, for the same reason this page opens with. An estimate that silently mirrors truth would make estimate.kin - kin a tautology, so the block is absent rather than filled from the truth block.

Publishing the belief yourself

The believed block is therefore yours to supply, and it travels on its own topic rather than in the snapshot. publish_estimate puts a swarmbotix.states.EstimateState on vrobots/{sys_id}/z/estimate, where the fixed wing reads it under FW_EST_OBSERVER. Read sensors off z/state, run your filter, publish the result, and the subtraction against kin measures a real estimator error. Publishing estimates is that loop end to end.

Gotcha. estimate.valid is false until the filter converges, and an unconverged estimate that silently mirrors truth is the classic trap. A filter initialised from the simulator's own state reads as a perfect estimator for as long as nothing disturbs it, and your error metric reads zero because you are subtracting a number from itself. Gate every use of estimate on estimate.valid, and treat an error of exactly zero as evidence of a bug rather than of quality.

Which block should your code read

Note. Nothing in the SDK stops a controller reading kin. Early on that is often the right choice, because it separates a controller bug from a sensing problem. Make it a decision you can find later, not a default.

Next: Kinematics

See also: Publishing estimates, Frames, axes and units, A tour of the whole snapshot, Sensor noise

Kinematics

Every field of the pose, twist and acceleration block, the frame each one is expressed in, and the net wrench published beside it.

One struct, two uses

Kinematics appears twice in a snapshot: as State::kin, which is simulator truth, and as State::estimate.kin, which is the robot's belief. The fields, units and frames are identical in both, which is what makes estimate.kin - kin a meaningful subtraction. Everything on this page applies to both instances.

The one distinction inside the struct is the frame. Position and attitude are world quantities: they answer "where is the robot in the scene". Velocity, angular velocity and both accelerations are body quantities: they answer "what is the robot doing to itself". That split is physics rather than configuration, and no service changes it.

The wire carries these components as f32. The SDK widens them to f64, which is lossless, so the same struct reads naturally from Rust, C++ and Python.

The three conventions that catch people

Quaternion order is [x, y, z, w]. It matches the wire's Vec4 field order, not the [w, x, y, z] order that several maths libraries use for their constructors. Feeding the array into a library that expects scalar-first produces a rotation that is plausible, continuous and wrong, so it survives casual inspection.

ang_vel is not the derivative of Euler angles. It is the body rate vector, the quantity a rate gyro integrates. The two agree only in the trivial case of a single-axis rotation. If your controller wants roll, pitch and yaw rates in the aerospace sense, ang_vel is already that vector; if it wants the time derivatives of the Euler angles you extracted from quat, you must convert, and the conversion is singular near 90 degrees of pitch.

lin_acc is not what the accelerometer reads. kin.lin_acc is coordinate acceleration in body axes. The accelerometer reports specific force, which includes gravity, so a robot sitting still on the ground publishes lin_acc near zero and an accelerometer reading near 1 g. See Sensors.

Gotcha. A nested table missing from the wire decodes to that struct's Default, and Kinematics::default() sets quat to [0.0, 0.0, 0.0, 0.0], which is not the identity rotation [0, 0, 0, 1]. A zero quaternion is not a unit quaternion, so normalising it divides by zero. Check the norm before you rely on an attitude, particularly when decoding recorded payloads whose producer may not have filled the block.

Which frame the numbers are in

The vectors above are expressed in the convention named by the snapshot's own axis_convention and coord_frame_id, which is the robot's frame and not yours. Different robot types genuinely disagree: the examples report the truck publishing fru and the multirotor frd, so the third component of lin_pos is up for one and down for the other. Read coord_frame_id before you interpret a sign.

Axes::name() returns the registry id for the three built-in conventions and "" for anything else, so a runtime-registered frame keeps its identity in coord_frame_id while axis_convention carries whatever tag the publisher stamped.

Wrench: the physics engine's bottom line

State::wrench is truth, published beside the kinematics, and it is the net effect of everything acting on the body during that step: rotor thrust, gravity, contacts and drag combined.

Its use is diagnostic. Diffing the wrench against the force and torque your commanded actuation should have produced is how you observe unmodelled forces, and it is considerably more direct than inferring them from the acceleration.

Reading the truth block

The tour example prints the whole of kin and the wrench together, labelling each line with its frame.

From examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
// -- truth ----------------------------------------------------------
let k = &s.kin;
println!("TRUTH  kinematics");
println!("  lin_pos   {} m       (world)", v3(k.lin_pos));
println!(
    "  quat      [{:+.3},{:+.3},{:+.3},{:+.3}] (world, xyzw)",
    k.quat[0], k.quat[1], k.quat[2], k.quat[3]
);
println!("  lin_vel   {} m/s     (body)", v3(k.lin_vel));
println!(
    "  ang_vel   {} rad/s   (body -- what a gyro measures)",
    v3(k.ang_vel)
);
println!("  lin_acc   {} m/s^2   (body)", v3(k.lin_acc));
println!("  ang_acc   {} rad/s^2 (body)", v3(k.ang_acc));
println!(
    "  wrench    F={} N  T={} N.m",
    v3(s.wrench.force),
    v3(s.wrench.torque)
);
}

// -- truth --------------------------------------------------------
std::printf("TRUTH  kinematics\n");
v3("lin_pos", r.kin.lin_pos, "m", "(world)");
std::printf("  %-9s [%+.3f,%+.3f,%+.3f,%+.3f] (world, xyzw)\n", "quat", r.kin.quat[0],
            r.kin.quat[1], r.kin.quat[2], r.kin.quat[3]);
v3("lin_vel", r.kin.lin_vel, "m/s", "(body)");
v3("ang_vel", r.kin.ang_vel, "rad/s", "(body -- what a gyro measures)");
v3("lin_acc", r.kin.lin_acc, "m/s^2", "(body)");
v3("ang_acc", r.kin.ang_acc, "rad/s^2", "(body)");
v3("force", r.wrench.force, "N", "(total on the body)");
v3("torque", r.wrench.torque, "N.m", "");

# -- truth ----------------------------------------------------------
k = s.kin
print("TRUTH  kinematics")
print(f"  lin_pos   {v3(k.lin_pos)} m       (world)")
quat = ",".join(f"{c:+.3f}" for c in k.quat)
print(f"  quat      [{quat}] (world, xyzw)")
print(f"  lin_vel   {v3(k.lin_vel)} m/s     (body)")
print(f"  ang_vel   {v3(k.ang_vel)} rad/s   (body -- what a gyro measures)")
print(f"  lin_acc   {v3(k.lin_acc)} m/s^2   (body)")
print(f"  ang_acc   {v3(k.ang_acc)} rad/s^2 (body)")
print(f"  wrench    F={v3(s.wrench.force)} N  T={v3(s.wrench.torque)} N.m")

Rust and Python read quat as a four-element sequence; C++ indexes the C array. Every field name, frame and unit is the same in all three, and so is the rule that lin_pos and quat are world while the four rate and acceleration vectors are body.

For a stationary robot the twist and acceleration rows sit at zero and the attitude row shows the identity rotation with its scalar part last:

TRUTH  kinematics
  lin_pos   (  +0.031,  +0.852,  -1.204) m       (world)
  quat      [+0.000,+0.000,+0.000,+1.000] (world, xyzw)
  lin_vel   (  +0.000,  +0.000,  +0.000) m/s     (body)
  ang_vel   (  +0.000,  +0.000,  +0.000) rad/s   (body -- what a gyro measures)
  lin_acc   (  +0.000,  +0.000,  +0.000) m/s^2   (body)
  ang_acc   (  +0.000,  +0.000,  +0.000) rad/s^2 (body)
  wrench    F=(  +0.000,  +0.000,  +0.000) N  T=(  +0.000,  +0.000,  +0.000) N.m

Next: Sensors

See also: Frames, axes and units, Mass and inertia, A tour of the whole snapshot

Sensors

Every device in the measured block, its fields, its units, and the two common fields that reveal the rate it actually runs at.

What is in the block, and what is deliberately not

State::sensors is everything the robot can observe about itself. There are six devices: accelerometer, gyroscope, magnetometer, barometer, GNSS receiver and optical flow. Each is a separate struct with its own fields.

Two absences are deliberate. There is no imu grouping, because the accelerometer and the gyroscope are separate devices with separate clocks and separate noise models, and bundling them invites code that assumes they updated together. There is no attitude field anywhere in sensors, because attitude is never measured, only fused: if you want the robot's belief about its orientation, that is estimate, and if you want the simulator's, that is kin.quat.

Every device carries the same two fields, and they matter more than the readings do.

valid = false has three distinct causes that look identical from the client: the device is not mounted on this robot, it is mounted but has not produced a first reading, or it has lost its fix. Treat it as "do not use this number" and check the robot's configured sensor set if you expected otherwise.

The devices

Accelerometer

Specific force is the field people misread. It is not kin.lin_acc: subtracting gravity is your job, and doing it needs an attitude you do not have from this device alone.

Gyroscope

Magnetometer

Note. The unit of magnetic_field is not documented in state.rs. The tour example labels its printout gauss and says so explicitly; the sensor-noise service documents the magnetometer's noise parameter in tesla. Until this is checked against a running simulator, do not assume the reading and the noise setting share a unit (1 T is 10 000 G).

Barometer

The barometer is the one device with no axis_convention and no coord_frame_id, because it is a scalar sensor with nothing to orient.

GNSS

GeoPoint is latitude and longitude in degrees and altitude in metres, and the same struct appears in env.geo_point as the true position.

Gotcha. The receiver runs at roughly 5 Hz against a 25 Hz state stream, so the same fix is republished in about five consecutive snapshots. Presence is not freshness. To detect an update boundary, compare gnss.timestamp with the value you saw last, and only then treat the reading as new. Code that differentiates GNSS position once per state sample instead of once per fix produces a velocity that is zero four samples out of five and then spikes.

Optical flow

Optical flow is optional and deliberately a poor sensor: valid goes false over featureless ground. Robots mount it only when asked, so valid = false on a default robot usually means the device is not fitted rather than that it failed.

The per-device frame override

Five of the six devices carry their own axis_convention and coord_frame_id. They exist for one reason: a device whose mounting frame differs from the robot's, for example an IMU rotated in its bracket or a receiver quoting velocity in NED while the body publishes frd. When they are set, they win for that device's vectors only.

The practical rule is that a diff between a sensor vector and a truth vector is only meaningful once both are in the same frame. Read the device's coord_frame_id first; when it is empty, the device is in the robot's frame from the snapshot header.

Reading the measured block

The tour example prints each device with its reading, its validity and its own clock side by side, which is what makes the differing rates visible.

From examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
println!(
    "  gnss      lat={:.6} lon={:.6} alt={:.2} m  vel={} m/s (NED)",
    n.gnss.geo_point.latitude,
    n.gnss.geo_point.longitude,
    n.gnss.geo_point.altitude,
    v3(n.gnss.velocity)
);
println!(
    "            fix={} eph={:.2} epv={:.2} m  {}   [slowest device, ~5 Hz]",
    n.gnss.fix_type,
    n.gnss.eph,
    n.gnss.epv,
    stamp(n.gnss.valid, n.gnss.timestamp)
);
}

std::printf("  gnss      lat=%.6f lon=%.6f alt=%.2f m  vel=(%+.3f,%+.3f,%+.3f) m/s (NED)\n",
            n.gnss.geo_point.latitude, n.gnss.geo_point.longitude,
            n.gnss.geo_point.altitude, n.gnss.velocity[0], n.gnss.velocity[1],
            n.gnss.velocity[2]);
std::printf("            fix=%u eph=%.2f epv=%.2f m  ", n.gnss.fix_type, n.gnss.eph,
            n.gnss.epv);
stamp(n.gnss.valid, n.gnss.timestamp);
std::printf("   [slowest device, ~5 Hz]\n");

g = n.gnss.geo_point
print(
    f"  gnss      lat={g.latitude:.6f} lon={g.longitude:.6f} alt={g.altitude:.2f} m  "
    f"vel={v3(n.gnss.velocity)} m/s (NED)"
)
print(
    f"            fix={n.gnss.fix_type} eph={n.gnss.eph:.2f} epv={n.gnss.epv:.2f} m  "
    f"{stamp(n.gnss.valid, n.gnss.timestamp)}   [slowest device, ~5 Hz]"
)

The device path is the same in all three: sensors.gnss.geo_point.latitude, reached through s.raw.sensors in C++ and s.sensors in the other two. So is the valid and timestamp pair every device carries, which is what the shared stamp helper prints.

Run the tour at 1 Hz and the GNSS stamp advances once per printed block; run it at the state rate and the same stamp repeats:

  gnss      lat=37.400000 lon=-122.100000 alt=12.34 m  vel=(  +0.000,  +0.000,  +0.000) m/s (NED)
            fix=3 eph=1.20 epv=1.80 m  [valid t=1770000000.200]   [slowest device, ~5 Hz]

The helper stamp that formats the bracketed field is two lines long and is quoted in A tour of the whole snapshot.

Next: The environment block

See also: Sensor noise, Coordinate frames, Timestamps and sequence numbers

The environment block

The true world the sensor readings were noised from, field by field, including the one field the sources disagree about.

Truth about the world, not about the robot

State::env is a truth block, like kin and wrench. It holds the atmosphere and the geodetic position the simulator used when it generated that sample's measurements. Its value to you is almost entirely as the reference side of a subtraction: the barometer reading minus env.air_pressure is the barometer's error, and the GNSS position minus env.geo_point is the receiver's.

The block is small and every field is scalar or a fixed-size array, so reading it costs nothing.

gravity is a world-frame vector, not a scalar, so its sign tells you which way the frame's third axis points: positive where that axis counts downwards, negative where it counts upwards. That makes it a cheap runtime check that you have understood the frame the rest of the snapshot is in.

The two diffs this block exists for

Both are only meaningful when the measured side is fresh. The barometer and the GNSS receiver run at their own rates, so compare their timestamps before differencing. See Sensors.

Note. air_density and temperature are published as truth and are not derived from any sensor in the block: there is no thermometer and no air-data device in sensors. If your model needs density, this is where it comes from, and a real vehicle would have to estimate it.

The agl field, and what the sources say about it

agl is documented in the SDK as the true height above ground, to be compared with the barometer's pressure altitude. The examples say something incompatible with that.

The two are not reconcilable by reading the repository, so this book states both and resolves neither. The example header is explicit that the zero is a placeholder rather than a measurement, on the reasoning that env is the truth block and an invented height above ground would be worse than a visibly missing one.

Until it is checked against a running simulator, write code that survives either version: if agl is exactly zero while the robot is demonstrably not on the ground, fall back to the vertical component of kin.lin_pos with the sign that the snapshot's coord_frame_id implies.

Reading the world block

The tour example prints the environment in two lines, and labels the agl line with the example's own claim about it.

From examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
let env = &s.env;
println!("WORLD  environment");
println!(
    "  gravity   {} m/s^2   air {:.1} Pa {:.3} kg/m^3 {:.1} C",
    v3(env.gravity),
    env.air_pressure,
    env.air_density,
    env.temperature
);
println!(
    "  agl       {:.2} m    home lat={:.6} lon={:.6}   [agl is hard-coded 0 in sim v3.0.0 -- use -lin_pos[2]]",
    env.agl, env.geo_point.latitude, env.geo_point.longitude
);
}

std::printf("WORLD  environment\n");
std::printf("  gravity   (%+.3f,%+.3f,%+.3f) m/s^2   air %.1f Pa %.3f kg/m^3 %.1f C\n",
            r.env.gravity[0], r.env.gravity[1], r.env.gravity[2], r.env.air_pressure,
            r.env.air_density, r.env.temperature);
std::printf(
    "  agl       %.2f m    home lat=%.6f lon=%.6f   [agl is hard-coded 0 in sim "
    "v3.0.0 -- use -lin_pos[2]]\n",
    r.env.agl, r.env.geo_point.latitude, r.env.geo_point.longitude);

env = s.env
print("WORLD  environment")
print(
    f"  gravity   {v3(env.gravity)} m/s^2   air {env.air_pressure:.1f} Pa "
    f"{env.air_density:.3f} kg/m^3 {env.temperature:.1f} C"
)
print(
    f"  agl       {env.agl:.2f} m    home lat={env.geo_point.latitude:.6f} "
    f"lon={env.geo_point.longitude:.6f}   "
    f"[agl is hard-coded 0 in sim v3.0.0 -- use -lin_pos[2]]"
)

All three carry the same warning in the same place, because all three read the same field from the same message: agl is the one entry in this block you cannot use as it stands.

The printed agl value is the first thing to look at when you check this page's conflict, because a non-zero reading on a robot in the air settles it:

WORLD  environment
  gravity   (  +0.000,  +0.000,  +9.807) m/s^2   air 101325.0 Pa 1.225 kg/m^3 15.0 C
  agl       0.00 m    home lat=37.400000 lon=-122.100000   [agl is hard-coded 0 in sim v3.0.0 -- use -lin_pos[2]]

Next: Actuators

See also: Sensors, Known simulator issues, Kinematics

Actuators

The three index-parallel arrays that carry your last command back to you beside what the devices actually did.

Command in, motion out

State::actuator is the only block in a snapshot that is neither truth, measurement nor belief. Two of its arrays are an echo of the command you sent, and the third is the physical response of the devices. It is published on every state sample, at the state rate, whether or not you ever send a command.

The three arrays are index-parallel: entry i is the same device in each of them. There is no name, no id and no length field beyond the arrays' own lengths, so the mapping from index to physical device comes from the robot type, not from the snapshot.

measured is the array to read carefully, because its unit is a property of the device at that index rather than of the array: a rotor reports rad/s, a wheel reports rad/s, and a servo reports rad. Nothing in the snapshot tells you which, so the robot type decides. Chapter 7 gives the per-robot layouts.

Note. All three arrays default to empty rather than to a fixed length. A robot that has never been commanded, or a nested table missing from the wire, decodes to empty vectors. Index them with get, or check the length, before assuming a rotor count.

The echo is the only proof a command landed

Commands are fire and forget. They latch, they get no reply, and publishing to a topic nobody is subscribed to is not an error in zenoh, so a command sent while the simulator is closed still returns Ok. There is no acknowledgement anywhere in the command path.

That leaves exactly one way to confirm that a robot received what you sent: read actuator.pwm in the next state sample and compare it with what you commanded. The robust-loop example makes the point by printing the echo beside the sequence number in its status line.

From examples/rust/src/bin/ex19_robust_loop.rs:

#![allow(unused)]
fn main() {
let s = robot.states();
if samples.is_multiple_of(REPORT_EVERY) {
    let [x, y, z] = s.kin.lin_pos;
    let stats = robot.stats();
    println!(
        "ok  seq={} t={:.2}s pos=({x:.2},{y:.2},{z:.2}) echo={:?} \
         received={} gaps={} decode_errors={}",
        s.seq,
        s.elapsed,
        s.actuator.pwm,
        stats.received,
        stats.seq_gaps,
        stats.decode_errors
    );
}
}

const vrsdk::State s = robot.states();
if (samples % REPORT_EVERY == 0) {
    const double* p = s.kin().lin_pos;
    const vrsdk_state_stats_t st = robot.stats();
    std::printf(
        "ok  seq=%llu t=%.2fs pos=(%.2f,%.2f,%.2f) received=%llu gaps=%llu "
        "decode_errors=%llu\n",
        static_cast<unsigned long long>(s.seq), s.elapsed, p[0], p[1], p[2],
        static_cast<unsigned long long>(st.received),
        static_cast<unsigned long long>(st.seq_gaps),
        static_cast<unsigned long long>(st.decode_errors));
}

s = mr.states
if samples % REPORT_EVERY == 0:
    x, y, z = s.kin.lin_pos
    st = mr.stats
    print(
        f"ok  seq={s.seq} t={s.elapsed:.2f}s pos=({x:.2f},{y:.2f},{z:.2f}) "
        f"echo={s.actuator.pwm} received={st.received} gaps={st.seq_gaps} "
        f"decode_errors={st.decode_errors}"
    )

Rust and Python print actuator.pwm as a list; C++ has it as s.raw.actuator.pwm with a separate pwm_count, and s.pwm() is the convenience that turns the pair into a std::vector<std::uint32_t>. That count matters: the array is fixed-size and only the first pwm_count entries mean anything.

The echo= field is the pwm array verbatim, so a multirotor commanded to 1501 µs on all four rotors reports it back:

ok  seq=125 t=5.00s pos=(0.03,0.85,-1.20) echo=[1501, 1501, 1501, 1501] received=125 gaps=0 decode_errors=0

An echo that does not match what you sent is informative in itself. Values that stay at their previous setting across several samples mean your command did not reach the robot. An empty array means the robot has never been commanded at all. Values that differ from yours but do change when you change yours mean the robot received the command and did something to it, which is a question for the command chapter rather than this one.

Three arrays, three questions

The gap between normalized and measured is where actuator dynamics would show. A step in the command appears in pwm and normalized as soon as the robot has it, because those two are the command, while measured can only follow as fast as the modelled device does. Diffing the two over a step is how you observe that behaviour without instrumenting the simulator.

Where the shape differs: the Global Hawk

The three-array layout is uniform, but what the indices mean is not, and the fixed-wing Global Hawk is the case that breaks a naive reading.

Code that assumes measured is a homogeneous array of rotor speeds produces a thrust figure in the wrong units at index 6 and silently mixes radians with newtons in any aggregate. Read the robot type first.

Gotcha. Index-parallel does not mean equal length in general. Read the length of the array you are about to index rather than the length of pwm, particularly on robot types whose command surface is not a pulse width at all.

Next: Timestamps and sequence numbers

See also: Commands latch, Global Hawk, Rotors and thrust curves

Timestamps and sequence numbers

The four clocks and one counter that appear in a single snapshot, and which of them to use for which question.

Why there is more than one

A snapshot is stamped by the simulator at capture time, not by the SDK at read time, so every time value in it refers to the simulator's clock rather than to your process. Once you have that, the multiplicity is easy to justify: the header answers "when was this sample taken", each sensor answers "when did this device last produce a reading", and the estimate answers "how old is the filter's output". They are different questions and they have different answers on the same sample.

Reach for the right one and most freshness bugs disappear. Reach for the wrong one and you get code that looks correct and measures nothing.

t_ns is signed, which is deliberate: the useful operation on it is a difference, and a difference between two independent streams can legitimately be negative.

Elapsed, and what it does not do

elapsed is seconds since this robot's first state sample, computed by the decoder as (t_ns - epoch_ns) / 1e9. One epoch is shared by every stream on the robot, so a state elapsed and a camera frame elapsed are directly comparable. It is monotonic for as long as the simulator keeps publishing.

Two properties surprise people. It is not the simulator's run time, because the epoch is fixed at your handle's first sample, not at the simulator's start. And it does not reset when the simulator restarts: it keeps counting through the outage and comes back having jumped forward by however long the simulator was away. The robust-loop example measures exactly that, freezing at 5.84 s for the duration of an outage and resuming at 21.77 s on the first sample of the new run.

Note. State::decode(bytes, epoch_ns) exposes the same arithmetic for offline use. epoch_ns only affects elapsed; pass 0 when decoding a standalone recorded frame, and the frame's elapsed then equals its absolute unix time in seconds.

Sequence numbers

seq is a per-topic counter stamped by the publisher. Consecutive samples differ by one, so a jump means the samples in between never reached your process. That is the only ground truth for drops available to a client: a rate measured over a window cannot distinguish a publisher that slowed down from a network that dropped every third sample.

The sample-paced example computes both the wall gap and the skip count on each wakeup.

From examples/rust/src/bin/ex09_state_paced_loop.rs:

#![allow(unused)]
fn main() {
let s = robot.states();
let dt_ms = if last_t_ns == 0 {
    f64::NAN
} else {
    (s.t_ns - last_t_ns) as f64 / 1e6
};
let skipped = s.seq.saturating_sub(last_seq + 1);
last_seq = s.seq;
last_t_ns = s.t_ns;
}

// Exactly one new sample is waiting -- read it and do the work.
const vrsdk::State s = robot.states();
const double dt_ms =
    last_t_ns == 0 ? 0.0 : static_cast<double>(s.t_ns - last_t_ns) / 1e6;
const std::uint64_t skipped = s.seq > last_seq + 1 ? s.seq - last_seq - 1 : 0;
last_seq = s.seq;
last_t_ns = s.t_ns;

# Exactly one new sample is waiting -- read it and do the work.
s = mr.states
dt_ms = float("nan") if last_t_ns == 0 else (s.t_ns - last_t_ns) / 1e6
skipped = max(0, s.seq - (last_seq + 1))
last_seq, last_t_ns = s.seq, s.t_ns

t_ns is a signed 64-bit integer of nanoseconds and seq an unsigned 64-bit counter in all three, so the arithmetic is the same everywhere. Only the guard against the first iteration differs: Rust and Python use a NaN sentinel for the unknown first dt, C++ prints zero.

At a healthy 25 Hz the interval sits near 40 ms and skipped stays zero; a drop shows as a doubled interval and a non-zero skip on the same line:

seq=310 dt=  40.0 ms pos=(0.031,0.85,-1.20)
seq=311 dt=  40.1 ms pos=(0.031,0.85,-1.20)
seq=313 dt=  80.0 ms pos=(0.032,0.85,-1.21)  <- 1 sample(s) skipped

seq restarting from zero is not a drop. It means the publisher restarted, and the SDK recognises it as such: see Stream health.

Two streams, one clock

State arrives over zenoh and camera frames arrive over iceoryx2. They are independent streams with independent rates, and the SDK never pairs them. There is no combined callback, no synchronised read and no interpolation.

sequenceDiagram
    participant Sim as Simulator
    participant St as State stream (zenoh)
    participant Cam as Camera stream (iceoryx2)
    participant You as Your loop
    Sim->>St: State t_ns=T0
    You->>St: states()
    Sim->>Cam: Frame t_ns=T0+12ms
    Sim->>St: State t_ns=T0+40ms
    You->>Cam: latest()
    Note over You: lag = frame.t_ns - state.t_ns
    You->>You: accept or reject on lag

What makes the pairing possible at all is that both stamps are on the same clock: Frame::t_ns and State::t_ns are both simulator capture times in unix nanoseconds and are directly subtractable, and Frame::elapsed shares the state stream's epoch.

The fusion rule follows from that in one line: compare t_ns explicitly, decide a tolerance, and reject the pair when the lag exceeds it. A frame and a state sample that merely arrived near each other in your process are not simultaneous, because arrival order reflects transport and scheduling rather than capture time. Chapter 5 gives the freshness patterns in full.

Choosing between them

Next: Pacing your loop

See also: Freshness, Stream health, Measuring rates

Pacing your loop

The two ways to decide how often your loop body runs, and how to pick between them.

cargo run -p vrobots-examples --bin ex09_state_paced_loop
./target/cpp-build/ex09_state_paced_loop
python examples/python/ex09_state_paced_loop.py

Your clock or the data's clock

main owns the loop, so something in the body has to decide when the next iteration starts. The SDK offers two answers and they are not interchangeable.

rate(hz) sleeps until your next tick, with drift compensation, and then returns. Your clock drives the loop. states() hands back whatever the latest snapshot is at that moment, which at a tick rate above the 25 Hz stream means the same sample twice and at a tick rate below it means samples you never look at.

wait_new_state(timeout) blocks until a snapshot newer than the one you have arrives. The data drives the loop, so the body runs once per published sample: no duplicates, no skips, and no need to guess a rate that divides 25 Hz.

flowchart TB
  subgraph A["rate(hz): your clock"]
    A1["read states()"] --> A2["compute and command"]
    A2 --> A3["rate(hz) sleeps to next tick"]
    A3 --> A1
    A4["may read the same sample twice"] -.-> A1
  end
  subgraph B["wait_new_state(t): the data's clock"]
    B1["wait_new_state(timeout)"] --> B2{"new sample?"}
    B2 -->|Ok| B3["read states(), compute"]
    B2 -->|Timeout| B4["note the stall, hold last state"]
    B3 --> B1
    B4 --> B1
  end

Both signatures come from the same handle.

From crates/vrobots-sdk/src/robot.rs:

#![allow(unused)]
fn main() {
pub fn wait_new_state(&self, timeout: Duration) -> VrResult<()> {
}

void wait_new_state(double timeout_s = 0.2)

void rate(double hz)

def wait_new_state(self, timeout: float = 0.2) -> None: ...
def rate(self, hz: float) -> None: ...

Rust takes a Duration; C++ and Python take seconds as a plain double, and both default it to 0.2. Neither returns the sample in any of the three, so states() is still the read.

The call returns Ok(()) once a newer sample is available, at which point you still read it with states(). It does not hand you the sample, which keeps the read path identical in both styles.

A timeout is a status, not an error

VrError::Timeout from wait_new_state means "no new sample arrived in time". The session is healthy, the subscriber is intact, and the next call may well succeed. A paused simulator, a stopped simulator and a very busy machine all announce themselves this way, and none of them is a reason to exit.

Propagating that error out of main with ? is the single most common way to turn a paused simulator into a crashed program. Match on it instead, and let every other variant be fatal.

From examples/rust/src/bin/ex09_state_paced_loop.rs:

#![allow(unused)]
fn main() {
loop {
    match robot.wait_new_state(TIMEOUT) {
        Ok(()) => {
            // Exactly one new sample is waiting -- read it and do the work.
            let s = robot.states();
            let dt_ms = if last_t_ns == 0 {
                f64::NAN
            } else {
                (s.t_ns - last_t_ns) as f64 / 1e6
            };
            let skipped = s.seq.saturating_sub(last_seq + 1);
            last_seq = s.seq;
            last_t_ns = s.t_ns;

            let [x, y, z] = s.kin.lin_pos;
            println!(
                "seq={} dt={dt_ms:6.1} ms pos=({x:.3},{y:.2},{z:.2}){}",
                s.seq,
                if skipped > 0 {
                    format!("  <- {skipped} sample(s) skipped")
                } else {
                    String::new()
                }
            );
        }
        Err(VrError::Timeout(detail)) => {
            // Not a broken session: no sample arrived in time. The sim is
            // paused, stopped, or the machine is very busy. states() still
            // returns the last snapshot it had.
            let s = robot.states();
            println!(
                "no new state in {:?} ({detail}); still holding seq={} at t={:.3}",
                TIMEOUT, s.seq, s.elapsed
            );
        }
        Err(other) => return Err(other), // a real failure
    }
}
}

for (;;) {
    try {
        robot.wait_new_state(TIMEOUT_S);
    } catch (const vrsdk::Error& e) {
        if (e.code() != VRSDK_ERR_TIMEOUT) {
            throw;  // a real failure
        }
        // Not a broken session: no sample arrived in time. The sim is
        // paused, stopped, or the machine is very busy. states() still
        // returns the last snapshot it had.
        const vrsdk::State s = robot.states();
        std::printf("no new state in %.1fs; still holding seq=%llu at t=%.3f\n", TIMEOUT_S,
                    static_cast<unsigned long long>(s.seq), s.elapsed);
        continue;
    }

    // Exactly one new sample is waiting -- read it and do the work.
    const vrsdk::State s = robot.states();
    const double dt_ms =
        last_t_ns == 0 ? 0.0 : static_cast<double>(s.t_ns - last_t_ns) / 1e6;
    const std::uint64_t skipped = s.seq > last_seq + 1 ? s.seq - last_seq - 1 : 0;
    last_seq = s.seq;
    last_t_ns = s.t_ns;

    const double* p = s.kin().lin_pos;
    std::printf("seq=%llu dt=%6.1f ms pos=(%.3f,%.2f,%.2f)",
                static_cast<unsigned long long>(s.seq), dt_ms, p[0], p[1], p[2]);
    if (skipped > 0) {
        std::printf("  <- %llu sample(s) skipped", static_cast<unsigned long long>(skipped));
    }
    std::printf("\n");
}

while True:
    try:
        mr.wait_new_state(TIMEOUT)
    except vrsdk.VrError as e:
        if e.code != vrsdk.err.TIMEOUT:
            raise  # a real failure
        # Not a broken session: no sample arrived in time. The sim is
        # paused, stopped, or the machine is very busy. `states` still
        # returns the last snapshot it had.
        s = mr.states
        print(
            f"no new state in {TIMEOUT}s ({e.detail}); "
            f"still holding seq={s.seq} at t={s.elapsed:.3f}"
        )
        continue

    # Exactly one new sample is waiting -- read it and do the work.
    s = mr.states
    dt_ms = float("nan") if last_t_ns == 0 else (s.t_ns - last_t_ns) / 1e6
    skipped = max(0, s.seq - (last_seq + 1))
    last_seq, last_t_ns = s.seq, s.t_ns

    x, y, z = s.kin.lin_pos
    note = f"  <- {skipped} sample(s) skipped" if skipped else ""
    print(f"seq={s.seq} dt={dt_ms:6.1f} ms pos=({x:.3f},{y:.2f},{z:.2f}){note}")

Rust's match puts the two outcomes side by side; C++ and Python invert it, catching the timeout, re-raising everything else, and falling through to the work. The continue in the timeout arm is what keeps the shape equivalent: a caught timeout must not run the read below it.

Pausing the simulator while this runs switches the output from one line per sample to one line per timeout, and unpausing it switches back without a reconnect:

seq=310 dt=  40.0 ms pos=(0.031,0.85,-1.20)
seq=311 dt=  40.1 ms pos=(0.031,0.85,-1.20)
no new state in 200ms (no new state within 200ms (sys_id 1)); still holding seq=311 at t=12.440
no new state in 200ms (no new state within 200ms (sys_id 1)); still holding seq=311 at t=12.440
seq=312 dt=1240.3 ms pos=(0.031,0.85,-1.20)

The example sets TIMEOUT to 200 ms, five times the 25 Hz period. That is the useful shape for a timeout: long enough that ordinary jitter never trips it, short enough that a stall is noticed within a human's patience.

Gotcha. wait_new_state guarantees that the sample is newer, not that it is the next one. The SDK stores the newest sample received, so if two arrive between wakeups you see the second and seq jumps by two. That is why the example computes skipped from seq rather than assuming one iteration equals one sample.

Choosing

The two styles compose. A controller can run on rate(hz) and still call wait_new_state with a short timeout once a second to check that the stream is alive, which is the pattern Stream health builds on.

Note. rate(hz) is drift-compensated: its deadlines are absolute multiples of the period counted from the first call, not now + period, so the time your loop body spends working does not accumulate as a slow drift. An iteration that overruns its budget re-anchors to the present rather than firing a burst of zero-length sleeps to catch up. It is also deliberately infallible, so it needs no ? on the hot path: a non-finite or non-positive hz returns immediately with a warning instead of panicking.

Next: Stream health

See also: Timestamps and sequence numbers, The shape of a program, Measuring rates

Stream health

The counters that say whether your loop is really seeing every sample, and how to survive the simulator going away and coming back.

cargo run -p vrobots-examples --bin ex12_version_info
./target/cpp-build/ex12_version_info
python examples/python/ex12_version_info.py
cargo run -p vrobots-examples --bin ex19_robust_loop
./target/cpp-build/ex19_robust_loop
python examples/python/ex19_robust_loop.py

Why a rate is not enough

Measuring how many samples per second your loop processed answers a weaker question than it appears to. Twenty-five samples a second with three gaps and twenty samples a second with none are different experiments, and a rate alone cannot tell them apart: a publisher that slowed down and a network that dropped every fifth sample produce the same average.

The SDK counts the difference for you, continuously, from the sequence numbers the publisher stamps. stats() returns a copy of those counters.

seq_gaps and missed_samples answer different questions. Ten gaps of one sample each is jitter in the transport; one gap of ten samples is something that stopped. Both report missed_samples = 10.

Decode errors are counted, not raised

A malformed payload does not tear down the session, does not stop the subscriber and does not raise anywhere in your loop. One bad sample must not end a flight, so it increments decode_errors and the loop keeps running. The error itself is kept for inspection.

last_error() returning None means every payload so far decoded. A non-empty value beside a non-zero decode_errors is the actual reason, and it is almost always schema drift between your build and the simulator's.

From examples/rust/src/bin/ex12_version_info.rs:

#![allow(unused)]
fn main() {
let stats = robot.stats();
let last = robot.states();
println!(
    "\nstats after {:.1} s: received={} decode_errors={} seq_gaps={} \
     missed_samples={} last_seq={}",
    elapsed.as_secs_f64(),
    stats.received,
    stats.decode_errors,
    stats.seq_gaps,
    stats.missed_samples,
    stats.last_seq
);
println!(
    "  effective rate {:.1} Hz over the window; sim clock advanced {:.2} s",
    stats.received as f64 / elapsed.as_secs_f64(),
    last.elapsed - first.elapsed
);

// Counted, not raised. This is where a decode failure went.
match robot.last_error() {
    None => println!("  last_error: none -- every payload decoded"),
    Some(e) => println!("  last_error: [{}] {e}", e.code()),
}
}

const vrsdk_state_stats_t st = robot.stats();
const vrsdk::State last = robot.states();
std::printf(
    "\nstats after %.1f s: received=%llu decode_errors=%llu seq_gaps=%llu "
    "missed_samples=%llu last_seq=%llu\n",
    elapsed, static_cast<unsigned long long>(st.received),
    static_cast<unsigned long long>(st.decode_errors),
    static_cast<unsigned long long>(st.seq_gaps),
    static_cast<unsigned long long>(st.missed_samples),
    static_cast<unsigned long long>(st.last_seq));
std::printf("  effective rate %.1f Hz over the window; sim clock advanced %.2f s\n",
            static_cast<double>(st.received) / (elapsed > 0.0 ? elapsed : 1.0),
            last.elapsed - first.elapsed);

// Counted, not thrown. This is where a decode failure went.
if (const std::optional<vrsdk::Error> e = robot.last_error()) {
    std::printf("  last_error: [%d] %s\n", e->code(), e->what());
} else {
    std::printf("  last_error: none -- every payload decoded\n");
}

st = mr.stats
last = mr.states
print(
    f"\nstats after {elapsed:.1f} s: received={st.received} "
    f"decode_errors={st.decode_errors} seq_gaps={st.seq_gaps} "
    f"missed_samples={st.missed_samples} last_seq={st.last_seq}"
)
print(
    f"  effective rate {st.received / elapsed:.1f} Hz over the window; "
    f"sim clock advanced {last.elapsed - first.elapsed:.2f} s"
)

# Counted, not raised. This is where a decode failure went.
err = mr.last_error
if err is None:
    print("  last_error: none -- every payload decoded")
else:
    print(f"  last_error: [{err.code} {err.kind}] {err.detail}")

The five counters carry the same names everywhere. The absence is spelled three ways for the same reason: it is an optional, not an error. Rust returns Option<VrError>, C++ returns std::optional<vrsdk::Error>, and Python returns None or a VrError without raising it, which is the whole point of the field. In Python both stats and last_error are properties.

On a healthy link the effective rate lands near the stream's 25 Hz, the gap counters stay at zero, and the last line reports nothing:

stats after 2.0 s: received=50 decode_errors=0 seq_gaps=0 missed_samples=0 last_seq=361
  effective rate 25.0 Hz over the window; sim clock advanced 2.00 s
  last_error: none -- every payload decoded

The same example prints the simulator's schema_version beside the SDK's, which is the check worth making before you trust any of the numbers above. A version mismatch does not present as an error; it presents as fields that decode to plausible nonsense. Versions and pins covers that comparison.

Noticing that the simulator stopped

states() cannot tell you. It keeps returning the last snapshot it had, unchanged, forever, and a dead simulator therefore looks exactly like a perfectly stationary robot. That is the observer contract, and the cost of it is that a stall is not detectable from a data read alone.

Two things do detect it: elapsed (or seq) ceasing to advance, and a Timeout from wait_new_state. The second is the deliberate one.

From examples/rust/src/bin/ex19_robust_loop.rs:

#![allow(unused)]
fn main() {
Err(VrError::Timeout(_)) => {
    if healthy {
        println!("\nSTALLED: no new state in {TIMEOUT:?}. Not an error -- holding.");
        healthy = false;
        down_since = Some(Instant::now());
    }
    // states() still answers, with the LAST snapshot. Note that
    // `elapsed` is frozen: that, not an exception, is how a dead sim
    // looks from a data read.
    let s = robot.states();
    let down = down_since.map_or(0.0, |t| t.elapsed().as_secs_f64());
    println!(
        "    down {down:5.1}s -- stale snapshot still readable: seq={} \
         t={:.2}s (frozen)",
        s.seq, s.elapsed
    );

    // Publishing into an empty topic is not an error in zenoh, so
    // this keeps returning Ok. A command has no reply; only the echo
    // in the state stream ever proves anything landed.
    robot.set_mr_pwm([PWM_US; 4])?;
}
}

if (timed_out) {
    if (healthy) {
        std::printf("\nSTALLED: no new state in %.1fs. Not an error -- holding.\n",
                    TIMEOUT_S);
        healthy = false;
        down_since = std::chrono::steady_clock::now();
    }
    // states() still answers, with the LAST snapshot. Note that
    // `elapsed` is frozen: that, not an exception, is how a dead
    // sim looks from a data read.
    const vrsdk::State s = robot.states();
    const double down =
        std::chrono::duration<double>(std::chrono::steady_clock::now() - down_since)
            .count();
    std::printf(
        "    down %5.1fs -- stale snapshot still readable: seq=%llu t=%.2fs "
        "(frozen)\n",
        down, static_cast<unsigned long long>(s.seq), s.elapsed);

    // Publishing into an empty topic is not an error in zenoh, so
    // this keeps succeeding. A command has no reply; only the echo
    // in the state stream ever proves anything landed.
    robot.set_mr_pwm(hold);
    continue;
}

if healthy:
    print(f"\nSTALLED: no new state in {TIMEOUT}s. Not an error -- holding.")
    healthy = False
    down_since = time.perf_counter()
# `states` still answers, with the LAST snapshot. Note that
# `elapsed` is frozen: that, not an exception, is how a dead sim
# looks from a data read.
s = mr.states
down = time.perf_counter() - down_since
print(
    f"    down {down:5.1f}s -- stale snapshot still readable: "
    f"seq={s.seq} t={s.elapsed:.2f}s (frozen)"
)
# Publishing into an empty topic is not an error in zenoh, so this
# keeps succeeding. A command has no reply; only the echo in the
# state stream ever proves anything landed.
mr.set_mr_pwm([PWM_US] * 4)
continue

Each surface measures the outage with its own monotonic clock: Instant, steady_clock and time.perf_counter. The SDK does not supply one, because elapsed is the simulator's clock and it is exactly the thing that has stopped.

Close the simulator while that runs and the output changes character without the program exiting, with t frozen at whatever it reached:

STALLED: no new state in 500ms. Not an error -- holding.
    down   0.5s -- stale snapshot still readable: seq=146 t=5.84s (frozen)
    down   1.0s -- stale snapshot still readable: seq=146 t=5.84s (frozen)

Commands sent during the outage keep returning Ok, because publishing to a topic nobody is subscribed to is not an error in zenoh. Nothing in that path can tell you the robot is gone, which is the practical reason the actuator echo is the only proof a command landed.

Surviving a restart

Start the simulator again and samples resume on the same session. There is no reconnect logic in the example, because there is nothing to reconnect: the zenoh session, the subscriber and the command publisher all outlive the simulator's absence, and discovery is zenoh's job.

The restart is visible in two places, and neither is an error.

Leaving seq_gaps untouched across a restart is a deliberate decision. A counter that jumped by several thousand every time someone left Play mode would be useless for the thing it exists for, which is spotting real drops.

Gotcha. A loop that treats VrError::Timeout as fatal turns every pause of the simulator into a crashed program, and a loop that ignores it entirely spins on stale data without noticing. Handle it as a state change: note the transition once, keep using the last known snapshot, and log the recovery when samples come back.

Next: A tour of the whole snapshot

See also: Pacing your loop, Versions and pins, Appendix C: Error reference

A tour of the whole snapshot

One program that prints truth, measurement and belief side by side once a second, and what to read in the differences between them.

cargo run -p vrobots-examples --bin ex10_sensors_tour
./target/cpp-build/ex10_sensors_tour
python examples/python/ex10_sensors_tour.py

What this example is for

Every other example in the book reads two or three fields. This one walks the entire State, block by block, and prints it at 1 Hz because each iteration is a page of text. It is the capstone of the chapter for a specific reason: the value of the snapshot is not in any single field but in the fact that truth, measurement and belief for the same instant are published together, so characterising anything is a subtraction rather than an inference.

Read the output once with the simulator idle and once with the robot moving, and most of this chapter becomes concrete.

The header, and the two helpers

Each iteration opens with the snapshot's identity: name, id, sequence number, elapsed time, schema version, and the frame every vector below it is expressed in.

From examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
println!(
    "\n=== {} sys_id={} seq={} t={:.3}s schema={} frame={:?} ({}) ===",
    s.name,
    s.sys_id,
    s.seq,
    s.elapsed,
    s.schema_version,
    s.coord_frame_id,
    s.axis_convention.name()
);
}

const vrsdk::State s = robot.states();
const vrsdk_state_t& r = s.raw;

std::printf("\n=== %s sys_id=%u seq=%llu t=%.3fs schema=%u frame=\"%s\" ===\n",
            s.name.c_str(), s.sys_id, static_cast<unsigned long long>(s.seq),
            s.elapsed, r.schema_version, s.coord_frame_id.c_str());

s = mr.states

print(
    f"\n=== {s.name} sys_id={s.sys_id} seq={s.seq} t={s.elapsed:.3f}s "
    f"schema={s.schema_version} frame={s.coord_frame_id!r} "
    f"({s.axis_convention_name}) ==="
)

The C++ header line omits the convention name, because the C surface exposes axis_convention as a bare int32_t with no name lookup beside it. Rust spells the lookup axis_convention.name() and Python spells it axis_convention_name.

Printing coord_frame_id beside axis_convention.name() is not redundant. The string is authoritative and is the only way to name a frame registered at runtime; the convention is the enum tag beside it, and returns "" for anything outside the three built-in conventions.

Two small helpers do all the formatting, and the second one is the interesting one. Also from examples/rust/src/bin/ex10_sensors_tour.rs:

#![allow(unused)]
fn main() {
/// A 3-vector, aligned so a column of them reads as a column.
fn v3(v: [f64; 3]) -> String {
    format!("({:+8.3},{:+8.3},{:+8.3})", v[0], v[1], v[2])
}

/// A sensor's own validity and clock -- the two fields that reveal its rate.
fn stamp(valid: bool, timestamp: f64) -> String {
    format!(
        "[{} t={timestamp:.3}]",
        if valid { "valid" } else { "INVALID" }
    )
}
}

/// A 3-vector, aligned so a column of them reads as a column.
static void v3(const char* label, const double* v, const char* unit, const char* note) {
    std::printf("  %-9s (%+8.3f,%+8.3f,%+8.3f) %-8s %s\n", label, v[0], v[1], v[2], unit, note);
}

/// A sensor's own validity and clock -- the two fields that reveal its rate.
static void stamp(bool valid, double timestamp) {
    std::printf("[%s t=%.3f]", valid ? "valid" : "INVALID", timestamp);
}

def v3(v: Sequence[float]) -> str:
    """A 3-vector, aligned so a column of them reads as a column."""
    return "(" + ",".join(f"{c:+8.3f}" for c in v) + ")"


def stamp(valid: bool, timestamp: float) -> str:
    """A sensor's own validity and clock -- the two fields that reveal its rate."""
    return f"[{'valid' if valid else 'INVALID'} t={timestamp:.3f}]"

The C++ helpers print rather than return, because building strings with printf formatting would mean a scratch buffer per call. That is why the C++ output puts the label and unit inside v3 where the other two paste them at the call site.

Neither helper prints anything on its own: v3 renders one vector as ( +0.031, +0.852, -1.204) and stamp renders one device's validity and clock as [valid t=1770000000.960]. stamp is applied to every device, which is what makes the differing sensor rates visible in a static printout: the bracketed time next to the GNSS row sits still for about five iterations of a 25 Hz loop while the header's t advances on every one.

One iteration

The loop body prints five labelled sections in a fixed order: TRUTH, MEASURED, BELIEVED, WORLD and ACTUATORS. Each section's code is quoted on its own page of this chapter, and the whole body is one println! per line with no logic between them.

=== multirotor sys_id=1 seq=725 t=29.000s schema=1 frame="frd" (frd) ===
TRUTH  kinematics
  lin_pos   (  +0.031,  +0.852,  -1.204) m       (world)
  quat      [+0.000,+0.000,+0.000,+1.000] (world, xyzw)
  lin_vel   (  +0.000,  +0.000,  +0.000) m/s     (body)
  ang_vel   (  +0.000,  +0.000,  +0.000) rad/s   (body -- what a gyro measures)
  lin_acc   (  +0.000,  +0.000,  +0.000) m/s^2   (body)
  ang_acc   (  +0.000,  +0.000,  +0.000) rad/s^2 (body)
  wrench    F=(  +0.000,  +0.000,  +0.000) N  T=(  +0.000,  +0.000,  +0.000) N.m
MEASURED  sensors
  accel     (  +0.012,  -0.004,  -9.803) m/s^2  [valid t=1770000000.960]   [specific force: +1 g at rest]
  gyro      (  +0.001,  -0.002,  +0.000) rad/s  [valid t=1770000000.960]
  mag       ( +22.100,  +1.400, +42.300) gauss  [valid t=1770000000.960]
  baro      101318.4 Pa  alt=0.58 m (qnh 101325.0 hPa)  [valid t=1770000000.940]
  gnss      lat=37.400000 lon=-122.100000 alt=12.34 m  vel=(  +0.000,  +0.000,  +0.000) m/s (NED)
            fix=3 eph=1.20 epv=1.80 m  [valid t=1770000000.800]   [slowest device, ~5 Hz]
  flow      (  +0.000,  +0.000,  +0.000) m/s  [INVALID t=0.000]   [optional; mount it via srv/sensors]
BELIEVED  estimate  [INVALID t=0.000]  frame=""
  lin_pos   (  +0.000,  +0.000,  +0.000) m       (estimate.kin - kin IS the error)
  lin_vel   (  +0.000,  +0.000,  +0.000) m/s
WORLD  environment
  gravity   (  +0.000,  +0.000,  +9.807) m/s^2   air 101325.0 Pa 1.225 kg/m^3 15.0 C
  agl       0.00 m    home lat=37.400000 lon=-122.100000   [agl is hard-coded 0 in sim v3.0.0 -- use -lin_pos[2]]
ACTUATORS  command in, motion out
  pwm        [] us      (echo of the last command)
  normalized []
  measured   []   (rotor rad/s -- what the devices did)

Reading the differences

The printout is arranged so that the pairs worth subtracting sit near each other.

Four things the printout makes visible that a field table cannot.

The accelerometer disagrees with lin_acc on purpose. It reads specific force, so a robot at rest reports about 1 g and a robot in free fall reports zero, while kin.lin_acc does the opposite. Subtracting gravity is your job and it needs an attitude.

A sensor's stamp moves at the sensor's rate. Watch the GNSS row's bracketed time sit still while the header's t advances. Nothing else in the snapshot reveals that, and code that treats every sample's GNSS reading as new will differentiate a constant four times out of five.

An invalid block is not a failure. The test scene runs no estimator, so estimate arrives valid=false, zero-filled, with an empty frame id. Optical flow is optional and arrives the same way when it is not mounted. That is what "not present" looks like on the wire; a missing nested table decodes to its Default rather than raising.

The frame is on every line for a reason. coord_frame_id is the robot's, not yours. In a scene where the multirotor publishes frd, lin_pos[2] counts downwards and altitude is its negation, which is the substitute the example uses for env.agl.

Gotcha. Everything in TRUTH and WORLD is unavailable on a physical vehicle. The tour is a debugging and characterisation tool, and reading it is the fastest way to understand a robot; taking a control decision from those two sections is how a program that works in simulation stops working anywhere else.

Next: Sending commands

See also: Truth, measured and believed, Sensors, Supported virtual robots

Sending commands

Every actuator in the simulator is driven the same way: you publish a command, nothing replies, and the state stream is the only receipt you get.

The path a command takes

A command is a one-way publication onto a shared bus. Nothing in the diagram below travels back along the arrow it came in on.

sequenceDiagram
    participant You as Your loop
    participant SDK
    participant Cmd as z/cmd
    participant Robot
    participant St as z/state

    You->>SDK: set_mr_pwm([1501.0; 4])
    SDK->>SDK: validate client-side
    SDK->>Cmd: FlatBuffers put (zenoh)
    Note over Cmd: many-to-many: every peer's<br/>traffic to this robot lands here
    SDK-->>You: Ok(()) = published
    Cmd->>Robot: queued
    Note over Robot: next physics step:<br/>drain queue, latch, ack nothing
    Robot->>St: snapshot at 25 Hz
    St->>You: states().actuator echoes it

In words, five steps:

1. You call a typed method such as set_mr_pwm, or send_cmd for an id the SDK has no method for.
2. The SDK validates what it can client-side. A pulse width outside the band, a non-finite float or an empty slice returns VrError::InvalidArgument and nothing reaches the wire.
3. The SDK publishes one FlatBuffers payload to vrobots/{sys_id}/z/cmd over zenoh.
4. The robot drains its command queue at the start of its next physics step, and acknowledges nothing.
5. The only observable consequence is the actuator echo in the next state snapshot.

Ok means published, not acted on

Ok(()) from any command method means zenoh accepted the put. It says nothing about whether a robot read the message, understood the id, or did anything with it. The id space is shared across robot types, so a robot receiving an id it does not implement ignores it, and that is correct behaviour rather than a fault.

Gotcha. Wrong command id, wrong sys_id and wrong array length all present identically from outside: the state stream does not change. To tell them apart, print s.actuator.pwm (or s.actuator.measured on the plants that have no pulse widths) every iteration and watch whether it follows what you sent.

Which command drives which robot

Only catalog_key() is ever put on the wire. RobotType::from_catalog_key is case-insensitive and accepts synonyms ("car", "cart_pole", "invpen", "mass_spring_damper", "half_drone", "global_hawk", "rq4b"), which matters when a robot type comes from a configuration file rather than from Rust.

Floats narrow on the way out

The API is f64 everywhere, for the same reason the snapshots are: one float type across Rust, C++ and Python. The wire's float_val, float_arr, Vec3 and Vec4 are f32, so every float you send narrows once on the way out. Integer channels such as pulse widths do not: they are i32 on the wire and the SDK rounds into them.

Every vector carries a frame

Commands whose payload is a vector (vec3 or vec4) are stamped with the coord_frame_id and axis_convention from your ConnectOptions, and the robot converts them into its own axes using the physically correct rule for that command. A force and a torque convert differently, because a torque is a pseudovector and carries the handedness sign. Array payloads such as SET_MR_PWM and SET_FW_SURFACES carry no frame at all: they are per-channel numbers, and nothing is re-expressed.

Next: Commands latch

See also: Five rules that explain everything, Actuators, Appendix B: Command reference, Appendix D: Glossary

Commands latch

A command is a setpoint the robot holds, not an event it performs, and understanding that explains most of the surprises in this chapter.

A setpoint, not an impulse

The last command a robot received stays in effect until the next one arrives. Nothing decays it, nothing times it out, and nothing zeroes it when the process that sent it exits. Run a program that latches a pulse width, kill it, and the rotors keep spinning at that pulse width: the echo in the state stream reports the robot's holder, not the sender.

There is no watchdog anywhere in the command path. This is a deliberate simulator property, not an oversight, and it is the same assumption a real ESC makes about the flight controller upstream of it.

Your send rate is your own business

Because the value latches, the publish rate carries no meaning. A 5 Hz sender and a 50 Hz sender produce exactly the same behaviour if they send the same numbers, and physics runs at its own rate regardless. Publish at whatever rate your controller thinks at.

The practical consequence appears in every loop that waits for state. From examples/rust/src/bin/ex29_hello_cartpole.rs, the response to a stalled state stream is to do nothing at all:

#![allow(unused)]
fn main() {
        if let Err(VrError::Timeout(_)) = robot.wait_new_state(SAMPLE_TIMEOUT) {
            println!("no new state -- holding the last force (it latches)");
            continue;
        }
}

} catch (const vrsdk::Error& e) {
    if (e.code() != VRSDK_ERR_TIMEOUT) {
        throw;
    }
    std::printf("no new state -- holding the last force (it latches)\n");
    continue;
}

except vrsdk.VrError as e:
    if e.code != vrsdk.err.TIMEOUT:
        raise
    print("no new state -- holding the last force (it latches)")
    continue

Skipping the iteration is not a lost command. The previous force is still applied, so holding is the correct response to silence rather than a degraded one.

Releasing a command is itself a command

There is no "stop" verb. To stop pushing, send zero. From examples/rust/src/bin/ex28_hello_msd.rs, where the step force has to be explicitly withdrawn before the plant can ring back to equilibrium:

#![allow(unused)]
fn main() {
    // Release. The force LATCHES, so this zero is not optional.
    println!("   release (set_msd_force(0.0)) -- watch it ring back to equilibrium");
    for i in 0..RELEASE_SAMPLES {
        robot.set_msd_force(0.0)?;
}

// Release. The force LATCHES, so this zero is not optional.
std::printf("   release (set_msd_force(0.0)) -- watch it ring back to equilibrium\n");
for (int i = 0; i < RELEASE_SAMPLES; ++i) {
    robot.set_msd_force(0.0);

# Release. The force LATCHES, so this zero is not optional.
print("   release (set_msd_force(0.0)) -- watch it ring back to equilibrium")
for i in range(RELEASE_SAMPLES):
    robot.set_msd_force(0.0)

Leaving that call out does not "let the force fade": the mass sits pushed against its spring forever, and the printout looks like a plant that will not settle.

Gotcha. A Ctrl-C never releases anything. Examples that hand a robot back do it on the normal exit path only, so an interrupted run leaves the last force, the last pulse widths or the last deflections applied until someone else overwrites them.

What each command holds

Watching a latch hold

examples/rust/src/bin/ex31_globalhawk_direct.rs demonstrates the property directly by sending one pose and then sending nothing for roughly two seconds:

#![allow(unused)]
fn main() {
    // ===== latching, and no watchdog =====
    println!("\n-- nothing sent for ~2 s --");
    robot.set_fw_surfaces(&[
        DEFLECT_RAD,
        -DEFLECT_RAD,
        DEFLECT_RAD,
        -DEFLECT_RAD,
        0.0,
        0.0,
    ])?;
    for i in 0..HOLD_SAMPLES {
        if i % 25 == 0 {
            print_echo(&robot, "  latched");
        }
        robot.rate(HZ);
    }
    println!("  unchanged. A command is a setpoint; there is no failsafe behind it.");
}

// ===== latching, and no watchdog =====
const std::vector<double> latched = {DEFLECT_RAD, -DEFLECT_RAD, DEFLECT_RAD,
                                     -DEFLECT_RAD, 0.0,         0.0};
std::printf("\n-- nothing sent for ~2 s --\n");
robot.set_fw_surfaces(latched);
for (int i = 0; i < HOLD_SAMPLES; ++i) {
    if (i % 25 == 0) {
        print_echo(robot, "  latched");
    }
    robot.rate(HZ);
}
std::printf("  unchanged. A command is a setpoint; there is no failsafe behind it.\n");

# ===== latching, and no watchdog =====
latched = [DEFLECT_RAD, -DEFLECT_RAD, DEFLECT_RAD, -DEFLECT_RAD, 0.0, 0.0]
print("\n-- nothing sent for ~2 s --")
robot.set_fw_surfaces(latched)
for i in range(HOLD_SAMPLES):
    if i % 25 == 0:
        print_echo(robot, "  latched")
    robot.rate(HZ)
print("  unchanged. A command is a setpoint; there is no failsafe behind it.")

Each printed line has the shape below, and the point of the passage is that the panel numbers in successive lines are identical while nothing is being published:

  latched t=<seconds>s panels=[<six deflections, radians>] engine=<newtons> N  rates=(<p>,<q>,<r>) deg/s

What reset does to a latch

reset() re-latches the robot's initial command, the same thing the simulator's own Reset button does. It is a state reset, not a factory reset: configuration sent through the srv/* services (masses, noise models, rotor curves, skins) survives it.

A live publisher wins again one physics step later, so a control loop that keeps running through a reset barely notices: it sees one snapshot of the initial command and then its own values again. A program that resets and then stops sending sees the initial command stand indefinitely.

On the fixed wing, reset() also reverts the control mode and the estimate source, which is a larger change than re-latching a number and has its own section.

Next: Driving a multirotor

See also: Sending commands, Robot lifecycle, Pacing your loop

Driving a multirotor

SET_MR_PWM is the lowest actuation level the simulator offers, so flying a multirotor means writing the flight controller yourself.

cargo run -p vrobots-examples --bin ex02_hello_control
./target/cpp-build/ex02_hello_control
python examples/python/ex02_hello_control.py

The whole program

examples/rust/src/bin/ex02_hello_control.rs is the loop shape from chapter 2 with one command in it:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const SYS_ID: u32 = 1; // the multirotor in the test scene
const PWM_US: f64 = 1501.0; // microseconds per rotor, on the 1100-2000 band
const HZ: f64 = 100.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states();
        let [x, y, z] = s.kin.lin_pos;
        println!(
            "State t={:.3} pos=({x:.3},{y:.2},{z:.2}) echo={:?}",
            s.elapsed, s.actuator.pwm
        );

        // Do some COOL control here and publish -- PID/EKF is user code, NOT the SDK.
        let cool_control_result = [PWM_US; 4];
        robot.set_mr_pwm(cool_control_result)?;

        robot.rate(HZ);
    }
}

constexpr std::uint32_t SYS_ID = 1;  // the multirotor in the test scene
constexpr double PWM_US = 1501.0;    // microseconds per rotor, 1100-2000 band
constexpr double HZ = 100.0;

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();
        vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
        robot.connect();
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();
            const double* p = s.kin().lin_pos;

            // Do some COOL control here and publish -- PID/EKF is user code,
            // NOT the SDK.
            const std::vector<double> cool_control_result = {PWM_US, PWM_US, PWM_US, PWM_US};
            robot.set_mr_pwm(cool_control_result);

            // The echo: what the robot actually latched, from the state stream.
            const std::vector<std::uint32_t> echo = s.pwm();
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f)  pwm_echo=[", s.elapsed, p[0], p[1],
                        p[2]);
            for (std::size_t i = 0; i < echo.size(); ++i) {
                std::printf("%s%u", i ? "," : "", echo[i]);
            }
            std::printf("]  rotor0=%.1f rad/s\n",
                        s.actuator().measured_count > 0 ? s.actuator().measured[0] : 0.0);

            robot.rate(HZ);
        }
    } catch (const vrsdk::Error& e) {
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

SYS_ID = 1  # the multirotor in the test scene
PWM_US = 1501.0  # microseconds per rotor, on the 1100-2000 band
HZ = 100


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
    mr.connect()

    # ===== loop =====
    while True:
        s = mr.states
        x, y, z = s.kin.lin_pos
        print(
            f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f}) "
            f"echo={s.actuator.pwm}"
        )

        # Do some COOL control here and publish -- PID/EKF is user code, NOT the
        # SDK. `set_mr_pwm(a, b, c, d)` and `set_mr_pwm([a, b, c, d])` are the
        # same call.
        cool_control_result = [PWM_US] * 4
        mr.set_mr_pwm(cool_control_result)

        mr.rate(HZ)

One line per iteration. The four numbers at the end are actuator.pwm, which is your last command echoed back, so they are the proof the command landed:

State t=<seconds> pos=(<x>,<y>,<z>) echo=[1501, 1501, 1501, 1501]

PWM_US is barely off idle and will not lift the airframe. Raise it to 1700 to watch the drone climb.

Note. Nothing sits between these four pulse widths and the rotor thrust curves: no attitude stabilisation, no rate damping, no mixer. Equal values spin every rotor equally; differential values roll, pitch and yaw the airframe. Hover is wherever total thrust crosses weight for the current mass and curves, so there is no single "hover number".

The two methods

Both send command id SET_MR_PWM (300) with the values in int_arr. The array length must equal the airframe's rotor count, which is fixed at spawn.

The bindings need only one method each, because neither language has Rust's split between a fixed-size array and a slice.

void set_mr_pwm(const std::vector<double>& pwm)

def set_mr_pwm(self, *pwm: Union[float, Sequence[float]]) -> None: ...

C++ takes a std::vector<double> of any length, so {1501, 1501, 1501, 1501} covers the quadrotor and a two-element vector covers a HalfDrone. Python accepts either form: set_mr_pwm(a, b, c, d) and set_mr_pwm([a, b, c, d]) are the same call.

The band

The band is checked client-side by a shared check_pwm helper, so a value outside it, or a non-finite value, returns VrError::InvalidArgument before anything is published. Passing a normalised 0.7 by mistake names the channel and the band it missed:

pwm[0] = 0.7 is outside the 1100-2000 us pulse-width band (neutral is 1500; values look like microseconds, not normalised units)

The refusal is deliberate rather than defensive. The simulator's actuators are specified on 1100 to 2000 microseconds, so a value outside it is far more likely to be a unit mistake (a normalised 0.7, a raw newton figure) than an intent.

A wrong rotor count is not an error

The SDK checks each value but not the length, because it does not know the airframe. A slice of the wrong length is published happily, the simulator logs it and drops it, and the previously latched pulse widths stay in effect.

Gotcha. Sending five pulse widths to a quadrotor looks exactly like sending nothing: Ok(()) from the call, no error anywhere, and an echo that keeps reporting the old values. Compare s.actuator.pwm.len() against what you sent before assuming the simulator is asleep.

The one client-side length check is emptiness: an empty slice returns VrError::InvalidArgument with the message "set_mr_pwm needs one pulse width per rotor; got none".

The half drone

A HalfDrone is a two-rotor airframe that takes the same command id through set_mr_pwm_n with exactly two values, [left_us, right_us]. It is scene-authored rather than creatable, so attach to it by sys_id rather than asking the manager to spawn one.

Next: Driving the truck

See also: Commands latch, Actuators, Rotors and thrust curves, Multirotor

Driving the truck

SET_CAR carries three pulse-width channels, on a factory band that is not the multirotor's.

cargo run -p vrobots-examples --bin ex05_hello_car
./target/cpp-build/ex05_hello_car
python examples/python/ex05_hello_car.py

The whole program

examples/rust/src/bin/ex05_hello_car.rs is the same loop shape as the multirotor example with a different actuator in it:

use vrobots_sdk::{RobotType, VirtualRobot, VrError};

const SYS_ID: u32 = 0; // the truck in the test scene
const STEER_US: f64 = 1400.0; // left of centre
const THROTTLE_US: f64 = 1650.0; // light forward
const BRAKE_US: f64 = 1100.0; // released
const HZ: f64 = 50.0;

fn main() -> Result<(), VrError> {
    // ===== setup =====
    vrobots_sdk::init_logging("info");
    let robot = VirtualRobot::connect(RobotType::Truck, Some(SYS_ID))?;

    // ===== loop =====
    loop {
        let s = robot.states();
        let [x, y, z] = s.kin.lin_pos;
        // lin_vel is a BODY-frame vector, so no single component is "the speed";
        // its magnitude is.
        let [vx, vy, vz] = s.kin.lin_vel;
        let speed = (vx * vx + vy * vy + vz * vz).sqrt();
        println!(
            "State t={:.3} pos=({x:.3},{y:.2},{z:.2}) speed={speed:.2} m/s echo={:?}",
            s.elapsed, s.actuator.pwm
        );

        // A gentle left arc: steering left of centre, light forward throttle.
        robot.set_car(STEER_US, THROTTLE_US, Some(BRAKE_US))?;

        robot.rate(HZ);
    }
}

constexpr std::uint32_t SYS_ID = 0;      // the truck in the test scene
constexpr double STEER_US = 1400.0;      // left of centre
constexpr double THROTTLE_US = 1650.0;   // light forward
constexpr double BRAKE_US = 1100.0;      // released
constexpr double HZ = 50.0;

int main() {
    try {
        // ===== setup =====
        vrsdk::check_version();
        vrsdk::VirtualRobot robot(vrsdk::RobotType::Truck, SYS_ID);
        robot.connect();
        std::printf("connected to sys_id %u\n", robot.sys_id());

        // ===== loop =====
        for (;;) {
            const vrsdk::State s = robot.states();
            const double* p = s.kin().lin_pos;

            // A gentle left arc: steering left of centre, light forward
            // throttle, brake released.
            robot.set_car(STEER_US, THROTTLE_US, BRAKE_US);

            // Speed from the body-frame twist, so it is visible that the truck
            // really is moving rather than that the command was merely accepted.
            const double* v = s.kin().lin_vel;
            const double speed = std::sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]);

            const std::vector<std::uint32_t> echo = s.pwm();
            std::printf("State t=%.3f pos=(%.3f,%.2f,%.2f) speed=%.2f m/s  pwm_echo=[", s.elapsed,
                        p[0], p[1], p[2], speed);
            for (std::size_t i = 0; i < echo.size(); ++i) {
                std::printf("%s%u", i ? "," : "", echo[i]);
            }
            std::printf("]\n");

            robot.rate(HZ);
        }
    } catch (const vrsdk::Error& e) {
        std::fprintf(stderr, "error [%d] %s\n", e.code(), e.what());
        return 1;
    }
}

SYS_ID = 0  # the truck in the test scene
STEER_US = 1400.0  # left of centre
THROTTLE_US = 1650.0  # light forward
BRAKE_US = 1100.0  # released
HZ = 50


def main() -> None:
    # ===== setup =====
    vrsdk.init_logging("info")
    car = VirtualRobot(RobotType.TRUCK, sys_id=SYS_ID)
    car.connect()

    # ===== loop =====
    while True:
        s = car.states
        x, y, z = s.kin.lin_pos
        # lin_vel is a BODY-frame vector, so no single component is "the speed";
        # its magnitude is.
        speed = math.dist(s.kin.lin_vel, (0.0, 0.0, 0.0))
        print(
            f"State t={s.elapsed:.3f} pos=({x:.3f},{y:.2f},{z:.2f}) "
            f"speed={speed:.2f} m/s echo={s.actuator.pwm}"
        )

        # A gentle left arc: steering left of centre, light forward throttle.
        car.set_car(STEER_US, THROTTLE_US, BRAKE_US)

        car.rate(HZ)

The echo at the end of each line is the three channels coming back in the order they were sent:

State t=<seconds> pos=(<x>,<y>,<z>) speed=<m/s> echo=[1400, 1650, 1100]

The signature

#![allow(unused)]
fn main() {
pub fn set_car(&self, steer: f64, throttle: f64, brake: Option<f64>) -> VrResult<()>
}

void set_car(double steer, double throttle, std::optional<double> brake = std::nullopt)

def set_car(
    self, steer: float, throttle: float, brake: Optional[float] = None
) -> None: ...

The optional third argument is optional in the type system of all three, and C++ and Python also default it, so set_car(steer, throttle) is the two-channel form there.

The three channels go on the wire as int_arr, in that order, under command id SET_CAR (304).

The three channels

The truck's factory band is 1100 to 1900, unlike the multirotor's 1100 to 2000. The client-side validator is the shared pulse-width check, which accepts the wider band, so a value between 1900 and 2000 is published rather than refused. What the truck does with one is not documented.

Gotcha. The brake channel is the one people get wrong, because every other channel in the SDK is centred on 1500. Sending 1500 to the brake is not "no brake": it sits part way up a scale whose zero is 1100, and the symptom is a truck that accelerates far more slowly than its throttle suggests.

Passing None

brake: None sends the two-channel form, [steer_us, throttle_us], and brakes nothing. That is a different message from sending Some(1100.0) only in what is on the wire: both leave the truck unbraked.

Use None when your controller has no brake concept, and Some(...) when it does. Pick one form and keep to it: the two-channel message says nothing about the brake, and whether the robot then holds the brake value a previous three-channel message latched is not documented.

Stopping

Stopping is a command like any other. Throttle at 1500 is the idle brake, which is the truck's own resting state, and a brake channel at 1900 adds full braking on top of it. A controller that exits while commanding 1650 microseconds of throttle leaves the truck driving, because that is the value still latched.

Next: Single degree of freedom plants

See also: Commands latch, The truck drivetrain, Truck

Single degree of freedom plants

The mass spring damper and the cart pole each take one number, a force in newtons, and nothing else.

cargo run -p vrobots-examples --bin ex28_hello_msd
./target/cpp-build/ex28_hello_msd
python examples/python/ex28_hello_msd.py
cargo run -p vrobots-examples --bin ex29_hello_cartpole -- <sys_id>
./target/cpp-build/ex29_hello_cartpole <sys_id>
python examples/python/ex29_hello_cartpole.py <sys_id>

The two commands

Both send their value in float_val, and both validate only that it is finite. Both clamps are silent: nothing is refused, nothing is reported, and the state stream is the only place that says what force was actually applied.

The mass spring damper

An Msd is in the sandbox catalog, so connect(RobotType::Msd, None) spawns one. It is the only plant in the simulator whose response you can work out on paper first, because the SDK owns every letter of m*x'' + c*x' + k*x = F: m through set_physical_params, k and c through configure_msd, and F through set_msd_force.

From examples/rust/src/bin/ex28_hello_msd.rs, the start of one step-response run:

#![allow(unused)]
fn main() {
    println!("-- {label} --");
    if retune {
        robot.configure_msd(&MsdConfig::default().with_spring_k(k).with_damping_c(c))?;
    }
    // Home, at rest, with the force latch cleared -- otherwise the previous run's
    // step is still pushing.
    robot.set_msd_force(0.0)?;
    robot.reset()?;
}

std::printf("-- %s --\n", label);
if (retune) {
    auto config = vrsdk::msd_config();
    config.has_spring_k = true;
    config.spring_k = k;
    config.has_damping_c = true;
    config.damping_c = c;
    robot.configure_msd(config);
}
// Home, at rest, with the force latch cleared -- otherwise the previous run's
// step is still pushing.
robot.set_msd_force(0.0);
robot.reset();

print(f"-- {label} --")
if retune:
    robot.configure_msd(spring_k=k, damping_c=c)
# Home, at rest, with the force latch cleared -- otherwise the previous run's
# step is still pushing.
robot.set_msd_force(0.0)
robot.reset()

The three spell the optional configuration fields differently, and the difference is worth noticing because it is the pattern for every service in chapter 6. Rust chains with_* setters on a default; Python takes keyword arguments and omits what it does not set; C++ builds the plain C struct from vrsdk::msd_config() and sets a has_* flag beside each value. Forgetting the has_* flag in C++ means the field is silently not applied.

The progress lines that follow have this shape, one every 25 samples:

   t=<seconds>s x=<metres> m  x'=<m/s> m/s  disp=<metres> m  net F=<newtons> N

Note the order in that snippet. Clearing the force latch before the reset matters, because a reset with a step force still latched puts the mass back home and immediately starts pushing it again.

Predicting it before you run it

The example prints its own predictions, from the arithmetic in its header:

settles at   F / k          metres
period       2*pi*sqrt(m/k) seconds
damping      c / (2*sqrt(k*m))   -- < 1 rings, ~1 slides home, > 1 crawls

The actuator block carries the plant's own arithmetic rather than an echo of your command:

Gotcha. measured[0] is not the force you sent. It is what the spring and the damper left of it, which is why it crosses zero at every peak of the oscillation. To check that a clamp did not eat your command, compare against the displacement rather than against this channel.

The cart pole

A cart pole is scene-authored rather than creatable, so it takes a sys_id argument: find the live one with cargo run -p vrobots-sdk --bin vrobots -- topic list. Ids are allocated at scene load and keep incrementing, so no constant in an example could stay true.

One actuator, a force on the cart, and two things to control with it. That is what underactuated means: the pole is unactuated by design, and there is no command anywhere in the SDK that touches it. From examples/rust/src/bin/ex29_hello_cartpole.rs, the whole control output is three lines:

#![allow(unused)]
fn main() {
        let force = (-K_THETA * theta - K_THETA_DOT * theta_dot + K_X * x + K_V * v)
            .clamp(-MAX_FORCE_N, MAX_FORCE_N);
        robot.set_cartpole_force(force)?;
}

const double raw = -K_THETA * theta - K_THETA_DOT * theta_dot + K_X * x + K_V * v;
const double force = std::fmin(std::fmax(raw, -MAX_FORCE_N), MAX_FORCE_N);
robot.set_cartpole_force(force);

force = min(
    max(-K_THETA * theta - K_THETA_DOT * theta_dot + K_X * x + K_V * v, -MAX_FORCE_N),
    MAX_FORCE_N,
)
robot.set_cartpole_force(force)

Only the clamp differs, because only Rust has f64::clamp on the primitive. The control law and the newtons on the wire are identical.

Clamping in the controller as well as trusting the simulator's clamp is worth the line: the simulator's clamp is silent, so a controller that saturates without knowing it will report a force it never applied. Each printed line reports both:

t=<seconds>s  theta=<degrees> deg  theta'=<rad/s> rad/s  rail=<metres> m (world x=<metres>)  x'=<m/s> m/s  F=<commanded N> N  applied=<clamped N> N

The pole rides the actuator channels, which are exactly three:

theta is 0 upright and plus or minus pi hanging, wrapped into [-pi, pi], so a fallen pole may print either sign.

Latching is unforgiving here

On a plant this unstable, a controller that stops publishing is not neutral. The last force stays applied, and the pole is on the floor within about a second. Two habits follow: treat a late state sample as a reason to hold the last force rather than to send something new, and release the latch explicitly on the way out:

#![allow(unused)]
fn main() {
    robot.set_cartpole_force(0.0)?;
}

// ===== hand it back =====
// A command latches: without this the cart keeps pushing forever.
robot.set_cartpole_force(0.0);

# ===== hand it back =====
# A command latches: without this the cart keeps pushing forever.
robot.set_cartpole_force(0.0)

A Ctrl-C skips that line and leaves the cart pushing.

Next: Fixed wing control

See also: Commands latch, Mass spring damper and cart pole, Mass spring damper, Cart pole

Fixed wing control

The RQ-4B Global Hawk flies itself unless you take it, and five commands decide who is flying and what the airframe does.

cargo run -p vrobots-examples --bin ex31_globalhawk_direct -- <sys_id>
./target/cpp-build/ex31_globalhawk_direct <sys_id>
python examples/python/ex31_globalhawk_direct.py <sys_id>
cargo run -p vrobots-examples --bin ex33_fw_est_source -- <sys_id>
./target/cpp-build/ex33_fw_est_source <sys_id>
python examples/python/ex33_fw_est_source.py <sys_id>

The Global Hawk is scene-authored and lives in the IMU scene, not the sandbox, so both examples take the live sys_id as an argument. Find it with cargo run -p vrobots-sdk --bin vrobots -- topic list.

Two modes, and what reset does to them

stateDiagram-v2
    [*] --> Onboard
    Onboard: FW_ONBOARD_RATE (0, the default)
    Direct: FW_DIRECT_SURFACE (1)
    Onboard --> Direct: set_fw_ctrl_mode(1)
    Direct --> Onboard: set_fw_ctrl_mode(0)
    Direct --> Onboard: reset()
    Onboard --> Onboard: reset() also puts the estimate source back to truth
    note right of Onboard
        rate PIDs track SET_ANGVEL
        airspeed hold owns the engine
        set_fw_thrust pins it, bias trims it
        estimate source: truth or observer
    end note
    note right of Direct
        panels take SET_FW_SURFACES verbatim
        set_fw_thrust is the thrust, full stop
        thrust bias ignored, estimate source inert
    end note

Left alone the aircraft cruises at 72.8 m/s under the onboard rate loop plus airspeed hold and needs nothing from you. Transitions in both directions are bumpless on the simulator's side, so there is no need to ramp in or out.

The five commands

The rate setpoint is a sixth command, and it is not a SET_FW_* id: it is SET_ANGVEL (51), shared across the whole id space with the fixed wing as the only type that acts on it. set_angvel(&self, rates: [f64; 3]) -> VrResult<()> is its typed method. The triple is [p, q, r] in rad/s in your header frame, and the robot re-expresses it as an axial vector, so a conversion between two opposite-handed conventions flips its sign where a force's would not.

Six panels, no mixer

Each entry of set_fw_surfaces drives its own panel. Turning roll, pitch and yaw demands into deflections is your job.

Indices 2 and 3 are how you prove direct mode took: the simulator's own mixer has zero gain there and can never move them, so an inner flap that follows your command could only have come through the per-panel path.

The length must equal the panel count exactly. A wrong-length array makes the simulator drop the whole command, never apply it partially, and the previously latched deflections stay in effect. Client-side the SDK checks only that the slice is non-empty and that every value is finite.

The echo is in radians and newtons

actuator.measured has panels plus one entries:

measured[0..=5]  per-panel deflection, RADIANS
measured[6]      the engine, NEWTONS -- not normalised, not a pulse width

A simulator too old for the per-panel path publishes six entries rather than seven, so the channel count is the version check. examples/rust/src/bin/ex32_fw_rate_controller.rs refuses to run against one:

#![allow(unused)]
fn main() {
    let channels = robot.states().actuator.measured.len();
    if channels != PANELS + 1 {
        return Err(VrError::InvalidArgument(format!(
            "this robot publishes {channels} actuator channels; this mixer is written for \
             {PANELS} panels + an engine. Six channels means a simulator too old for the \
             per-panel path."
        )));
    }
}

const std::uint32_t channels = robot.states().actuator().measured_count;
if (channels != PANELS + 1) {
    std::fprintf(stderr,
                 "this robot publishes %u actuator channels; this mixer is written for "
                 "%zu panels + an engine. Six channels means a simulator too old for the "
                 "per-panel path.\n",
                 channels, PANELS);
    return 1;
}

channels = len(robot.states.actuator.measured)
if channels != PANELS + 1:
    raise SystemExit(
        f"this robot publishes {channels} actuator channels; this mixer is "
        f"written for {PANELS} panels + an engine. Six channels means a "
        f"simulator too old for the per-panel path."
    )

The count comes off a length in Rust and Python and off measured_count in C++, which is the same number: the C struct carries a fixed-size array plus its used length.

The check runs once, before the aircraft is touched, so an old simulator produces a named error at startup instead of a mixer that appears to do nothing.

Bumpless is not zeroed

Entering a mode seeds the latches from what the plant is doing now, including whatever thrust the airspeed hold happened to be carrying. Nothing jolts, and nothing is cleared either. A client that wants a particular thrust must send it after every mode entry. That is why the order in examples/rust/src/bin/ex31_globalhawk_direct.rs is mode first, thrust second:

#![allow(unused)]
fn main() {
    robot.set_fw_ctrl_mode(cmd::FW_DIRECT_SURFACE)?;
    robot.set_fw_thrust(CRUISE_N)?;
    println!("\nmode -> DIRECT_SURFACE, thrust -> {CRUISE_N} N\n");
}

// -- so the thrust command has to come AFTER the mode, every time.
robot.set_fw_ctrl_mode(vrsdk::FwCtrlMode::DirectSurface);
robot.set_fw_thrust(CRUISE_N);
std::printf("\nmode -> DIRECT_SURFACE, thrust -> %.0f N\n\n", CRUISE_N);

# so the thrust command has to come AFTER the mode, every time.
robot.set_fw_ctrl_mode(cmd.FW_DIRECT_SURFACE)
robot.set_fw_thrust(CRUISE_N)
print(f"\nmode -> DIRECT_SURFACE, thrust -> {CRUISE_N} N\n")

Only the spelling of the mode differs. C++ has a real enum class, vrsdk::FwCtrlMode, so the argument is FwCtrlMode::DirectSurface; Rust and Python pass the integer constant from cmd. The wire value is the same in all three.

mode -> DIRECT_SURFACE, thrust -> 3800 N

Skip the thrust line and the engine keeps whatever the autopilot left, which is usually about 3.8 kN at trim. That is a plausible-looking number, which is what makes the omission hard to spot.

In onboard mode set_fw_thrust means something slightly different: it takes the engine off airspeed hold and pins it, and it clears any thrust bias. The two overrides are last-writer-wins, and set_fw_thrust_bias(0.0) is the release back to plain airspeed hold.

What reset takes away

reset() reverts the control mode to FW_ONBOARD_RATE and the estimate source to FW_EST_TRUTH, and relaunches the aircraft at trim airspeed. This is deliberate: keeping direct control with the surface latches zeroed would relaunch the aircraft unflyable.

Gotcha. After a reset your surface commands are still publishing, still returning Ok(()), and doing nothing at all, because the mixer is flying the panels again. The symptom is the inner flaps sitting at 0 while the same command streams. Re-assert set_fw_ctrl_mode, then re-assert set_fw_thrust, in that order, after every reset.

Feeding the loop your own estimate

set_fw_est_source decides which attitude the onboard rate loop believes. The controller itself never knows which, because the swap happens upstream of it in the robot, which is exactly how a real flight computer gets fooled.

From examples/rust/src/bin/ex33_fw_est_source.rs, selecting the observer with nothing publishing an estimate:

#![allow(unused)]
fn main() {
    robot.set_fw_est_source(cmd::FW_EST_OBSERVER)?;
    println!(
        "\nsource -> OBSERVER. Nothing publishes z/estimate here, so within 0.5 s \
         the estimate is stale and the loop is fed truth again -- the simulator \
         logs a warning saying exactly that."
    );
}

// ===== phase 2: observer, with nobody publishing an estimate =====
robot.set_fw_est_source(vrsdk::FwEstSource::Observer);
std::printf(
    "\nsource -> OBSERVER. Nothing publishes z/estimate here, so within 0.5 s the "
    "estimate is stale and the loop is fed truth again -- the simulator logs a warning "
    "saying exactly that.\n");

# ===== phase 2: observer, with nobody publishing an estimate =====
robot.set_fw_est_source(cmd.FW_EST_OBSERVER)
print(
    "\nsource -> OBSERVER. Nothing publishes z/estimate here, so within 0.5 s "
    "the estimate is stale and the loop is fed truth again -- the simulator "
    "logs a warning saying exactly that."
)

Again C++ has the typed vrsdk::FwEstSource where the other two pass cmd::FW_EST_OBSERVER and cmd.FW_EST_OBSERVER.

The run ends with one line per phase:

steady yaw-rate tracking, commanded 0.05 rad/s:
  truth (source 0)                   mean r=<rad/s>  mean error=<rad/s>  |error|=<rad/s>
  observer (source 1), no publisher  mean r=<rad/s>  mean error=<rad/s>  |error|=<rad/s>
  after reset                        mean r=<rad/s>  mean error=<rad/s>  |error|=<rad/s>

The example measures yaw-rate tracking error across those three phases, and phase 2 matching phase 1 is the whole result: the loop asked for an estimate, found none fresh, and kept flying on truth.

Gotcha. An estimate older than 0.5 s is treated as stale and the loop silently falls back to truth. From the client there is no field that says which source is live, so an estimator publishing at 1 Hz behaves like one that is not publishing at all, and the only difference you can measure is that the tracking error stops changing.

This page is only the selector. The other half is publish_estimate, which puts a swarmbotix.states.EstimateState on vrobots/{sys_id}/z/estimate for FW_EST_OBSERVER to find, and examples/rust/src/bin/ex35_publish_estimate.rs is the paired publisher: it flies the same aircraft on a truth copy of its own attitude, then on a copy pitched up five degrees, and the airframe trims down to chase a nose position that was never real. Publishing estimates is that side in full, including the half-second staleness clock from the publisher's end.

Next: The generic command

See also: Publishing estimates, Commands latch, Reading someone else's commands, Global Hawk

The generic command

send_cmd reaches the whole command id space, including the ids the SDK has no typed method for.

cargo run -p vrobots-examples --bin ex08_generic_cmd
./target/cpp-build/ex08_generic_cmd
python examples/python/ex08_generic_cmd.py

One message, and cmd_id decides what it means

#![allow(unused)]
fn main() {
pub fn send_cmd(&self, cmd_id: u32, args: &CmdArgs) -> VrResult<()>
}

/// Publish any command by id -- the escape hatch for the whole
/// `VROBOTS_CMDS` space. Fill `args` with `vrsdk_cmd_args_default` first.
void send_cmd(std::uint32_t cmd_id, const vrsdk_cmd_args_t* args = nullptr)

def send_cmd(
    self,
    cmd_id: int,
    *,
    int_val: int = 0,
    float_val: float = 0.0,
    int_arr: Optional[Sequence[int]] = None,
    float_arr: Optional[Sequence[float]] = None,
    vec3: Optional[Sequence[float]] = None,
    vec4: Optional[Sequence[float]] = None,
    vec3_arr: Optional[Sequence[Sequence[float]]] = None,
    vec4_arr: Optional[Sequence[Sequence[float]]] = None,
) -> None: ...

Python has no CmdArgs type at all: the eight payload fields are keyword arguments on the call, and anything you omit stays off the wire. C++ passes the plain C struct by pointer, and every field in it is a borrowed pointer, so a null means "absent" and nothing is retained after the call returns.

Command is a union by convention: one message shape, and the id decides which payload fields mean anything. SET_MR_PWM reads int_arr and ignores the rest, SET_BODY_FORCE reads vec3. Fill the fields the target id reads and leave the others alone. Empty vectors and None vectors are omitted from the wire entirely rather than sent as zero length, so the receiver's "field absent" default applies.

Errors are VrError::Deleted if the robot was deleted and VrError::Publish if zenoh refuses the put. There is no validation here, which is the trade: every typed method validates its arguments, and send_cmd publishes whatever you build.

Sending an implemented id and an ignored one, side by side

examples/rust/src/bin/ex08_generic_cmd.rs sends two commands per iteration to the truck, on purpose:

#![allow(unused)]
fn main() {
    loop {
        // (1) An implemented id, built by hand. CmdArgs::ints fills int_arr,
        //     which is the field SET_CAR reads; everything else stays off the
        //     wire.
        let drive = CmdArgs::ints(&[STEER_US, THROTTLE_US, BRAKE_US]);
        robot.send_cmd(cmd::SET_CAR, &drive)?;

        // (2) An id nothing acts on, whose payload rides vec3. Same call, same
        //     Ok(()), no effect. `cmd::name` turns an id back into its schema
        //     name, which is what makes a log line readable.
        let gust = CmdArgs::default().with_vec3(GUST_N);
        robot.send_cmd(cmd::ADD_BODY_FORCE, &gust)?;
}

// ===== loop =====
for (;;) {
    // (1) An implemented id, built by hand. int_arr is the field SET_CAR
    //     reads; everything else stays NULL and off the wire.
    vrsdk_cmd_args_t drive;
    vrsdk_cmd_args_default(&drive);
    drive.int_arr = channels;
    drive.int_arr_len = 3;
    robot.send_cmd(SET_CAR, &drive);

    // (2) An id nothing acts on, whose payload rides vec3. Same call,
    //     no exception, no effect. Every payload field is a borrowed
    //     pointer: NULL means "absent" and nothing is retained after
    //     the call returns, so a stack array is fine.
    vrsdk_cmd_args_t force;
    vrsdk_cmd_args_default(&force);
    force.vec3 = gust;  // exactly 3 doubles
    robot.send_cmd(ADD_BODY_FORCE, &force);

# ===== loop =====
while True:
    # (1) An implemented id, built by hand. int_arr is the field SET_CAR
    #     reads; everything else stays off the wire.
    car.send_cmd(cmd.SET_CAR, int_arr=[STEER_US, THROTTLE_US, BRAKE_US])

    # (2) An id nothing acts on, whose payload rides vec3. Same call, no
    #     exception, no effect. `cmd.name` turns an id back into its schema
    #     name, which is what makes a log line readable.
    car.send_cmd(cmd.ADD_BODY_FORCE, vec3=GUST_N)

Python is the shortest because the payload is keyword arguments. C++ is the longest because each call needs a vrsdk_cmd_args_default first, and it also declares the two ids as its own constants: the C surface ships no cmd namespace, so SET_CAR = 304 and ADD_BODY_FORCE = 203 are written out from the schema and there is no cmd::name to turn one back into a label.

Both calls return Ok(()). Only one of them changes anything, and the printed lines say which:

sent SET_CAR(304) + ADD_BODY_FORCE(203) -> echo=[1500, 1600, 1100]
      SET_CAR landed (the echo is the receipt); ADD_BODY_FORCE was ignored -- wrench=(<fx>,<fy>,<fz>) N unchanged

cmd::name(cmd_id) -> &'static str maps an id back to its constant name for logging, and returns "" for an id that is not in the schema.

CmdArgs

#![allow(unused)]
fn main() {
#[non_exhaustive]
pub struct CmdArgs {
    pub int_val: i32,
    pub float_val: f64,
    pub int_arr: Vec<i32>,
    pub float_arr: Vec<f64>,
    pub vec3: Option<[f64; 3]>,
    pub vec4: Option<[f64; 4]>,
    pub vec3_arr: Vec<[f64; 3]>,
    pub vec4_arr: Vec<[f64; 4]>,
}
}

typedef struct vrsdk_cmd_args_t {
    int32_t int_val;
    double float_val;
    const int32_t *int_arr;
    size_t int_arr_len;
    const double *float_arr;
    size_t float_arr_len;
    const double *vec3;
    const double *vec4;
    const double *vec3_arr;
    size_t vec3_arr_len;
    const double *vec4_arr;
    size_t vec4_arr_len;
} vrsdk_cmd_args_t;

int_val: int = 0
float_val: float = 0.0
int_arr: Optional[Sequence[int]] = None
float_arr: Optional[Sequence[float]] = None
vec3: Optional[Sequence[float]] = None
vec4: Optional[Sequence[float]] = None
vec3_arr: Optional[Sequence[Sequence[float]]] = None
vec4_arr: Optional[Sequence[Sequence[float]]] = None

The eight fields in the table below are the same eight in all three. The C struct pairs every array with an explicit _len, and it flattens vec3_arr and vec4_arr into one double array of three or four values per entry, so vec3_arr_len counts vectors, not doubles. Python's are the keyword arguments of send_cmd rather than a type of their own.

The struct is #[non_exhaustive], so it cannot be built with a struct literal: a future schema field would otherwise be a breaking change across three languages. Build it with a shorthand or with CmdArgs::default() plus chained setters.

The builders are with_int_val, with_float_val, with_int_arr, with_float_arr, with_vec3, with_vec4, with_vec3_arr and with_vec4_arr. Each consumes self, returns Self and is #[must_use], so they chain and a dropped result is a compiler warning rather than a silent no-op.

Every command id

Live means a robot type acts on the id today. Not yet means it is defined on the wire and no robot type acts on it. Absent means the robot type it belongs to does not exist in the simulator.

The mode constants are argument values rather than ids: FW_ONBOARD_RATE (0) and FW_DIRECT_SURFACE (1) for SET_FW_CTRL_MODE, FW_EST_TRUTH (0) and FW_EST_OBSERVER (1) for SET_FW_EST_SOURCE.

When to reach for it

Use send_cmd for ids the SDK has no method for, SET_ANGVEL being the one that comes up in practice. Prefer the typed methods everywhere they exist: they fill the right field for you and they validate, and a pulse width outside 1100 to 2000 microseconds is refused before anything is sent, where send_cmd publishes it.

Next: Commands nothing acts on

See also: Sending commands, Fixed wing control, Appendix B: Command reference

Commands nothing acts on

Several ids are fully defined on the wire and implemented by no robot type, and from the client side they look exactly like a command you got wrong.

cargo run -p vrobots-examples --bin ex06_hello_throttle
./target/cpp-build/ex06_hello_throttle
python examples/python/ex06_hello_throttle.py
cargo run -p vrobots-examples --bin ex07_body_wrench
./target/cpp-build/ex07_body_wrench
python examples/python/ex07_body_wrench.py

The worked example: SET_MR_THROTTLE

SET_MR_THROTTLE is normalised per-rotor throttle: four values on 0 to 1 instead of four pulse widths. It is the command you would reach for to hover without thinking in microseconds, and no robot type acts on it. examples/rust/src/bin/ex06_hello_throttle.rs is therefore a lesson in what that looks like rather than a way to fly:

#![allow(unused)]
fn main() {
    loop {
        // Published exactly like set_mr_pwm: one put on vrobots/<id>/z/cmd, no
        // reply, latched until the next one arrives.
        robot.set_mr_throttle(THROTTLE)?;

        let s = robot.states();
        // The state frame is the robot's, not yours -- "frd" here, so lin_pos[2]
        // is DOWN, and altitude above the start point is its negation.
        let [_, _, down] = s.kin.lin_pos;
        println!(
            "sent {THROTTLE:?} -> alt={:.2} m  pwm={:?} normalized={:?} measured={:?}",
            -down,
            s.actuator.pwm,
            round3(&s.actuator.normalized),
            round3(&s.actuator.measured),
        );

        robot.rate(HZ);
    }
}

// ===== loop =====
for (;;) {
    // Published exactly like set_mr_pwm: one put on vrobots/<id>/z/cmd,
    // no reply, latched until the next one arrives.
    robot.set_mr_throttle(throttle);

    const vrsdk::State s = robot.states();
    // The state frame is the robot's, not yours -- "frd" here, so
    // lin_pos[2] is DOWN and altitude is its negation.
    const double alt = -s.kin().lin_pos[2];
    const vrsdk_actuator_t& a = s.actuator();

    std::printf("sent %.2f x4 -> alt=%.2f m  ", THROTTLE, alt);
    std::printf("pwm=[");
    for (std::uint32_t i = 0; i < a.pwm_count; ++i) {
        std::printf(i ? ",%u" : "%u", a.pwm[i]);
    }
    std::printf("] ");
    print_array("normalized", a.normalized, a.normalized_count);
    print_array("measured", a.measured, a.measured_count);
    std::printf("\n");

    robot.rate(HZ);
}

# ===== loop =====
while True:
    # Published exactly like set_mr_pwm: one put on vrobots/<id>/z/cmd, no
    # reply, latched until the next one arrives. Both call shapes work:
    # set_mr_throttle(a, b, c, d) and set_mr_throttle([a, b, c, d]).
    mr.set_mr_throttle(THROTTLE)

    s = mr.states
    # The state frame is the robot's, not yours -- "frd" here, so lin_pos[2]
    # is DOWN, and altitude above the start point is its negation.
    down = s.kin.lin_pos[2]
    norm = [round(v, 3) for v in s.actuator.normalized]
    meas = [round(v, 3) for v in s.actuator.measured]
    print(
        f"sent {THROTTLE} -> alt={-down:.2f} m  pwm={s.actuator.pwm} "
        f"normalized={norm} measured={meas}"
    )

    mr.rate(HZ)

The call succeeds in all three and nothing moves in any of them, which is the point of the example: an ignored command is indistinguishable from a delivered one at the call site.

Every line reports the same three actuator channels, and none of them moves in response to what was sent:

sent [0.6, 0.6, 0.6, 0.6] -> alt=<metres> m  pwm=<latched pulse widths> normalized=<latched normalised command> measured=<rotor rad/s>

There is no reply and no error, because the id space is shared across robot types and "not mine" is correct behaviour rather than a fault. Run ex02 alongside it: identical loop, an id the simulator implements, and the echo moves.

Not yet. set_mr_throttle is on the wire and acted on by nothing. Its payload field is the one the schema provides, but unlike set_mr_pwm it has never been confirmed against a consumer, so treat it as unverified until the simulator implements it. Today the way to fly a multirotor is set_mr_pwm.

The body wrench group

The wrench group is the disturbance-injection channel the schema reserves for wind gusts, payload drops and contact pushes. Three typed methods cover it, and today they are in the same category as SET_MR_THROTTLE.

SET_BODY_FT is asymmetric because the schema is. The typed method hides it; building the message by hand with CmdArgs does not.

From examples/rust/src/bin/ex07_body_wrench.rs, one verb per iteration so each printed line names exactly what went out:

#![allow(unused)]
fn main() {
        let sent = match step % 3 {
            0 => {
                robot.set_body_force(GUST_N)?;
                format!("set_body_force({GUST_N:?})")
            }
            1 => {
                robot.set_body_torque(TWIST_NM)?;
                format!("set_body_torque({TWIST_NM:?})")
            }
            _ => {
                robot.set_body_ft(GUST_N, TWIST_NM)?;
                format!("set_body_ft({GUST_N:?}, {TWIST_NM:?})")
            }
        };
}

// One verb per iteration, so each printed line names exactly what
// went out on the wire.
const char* sent = nullptr;
switch (step % 3) {
    case 0:
        robot.set_body_force(GUST_N);
        sent = "set_body_force(5, 0, 0)";
        break;
    case 1:
        robot.set_body_torque(TWIST_NM);
        sent = "set_body_torque(0, 0, 0.2)";
        break;
    default:
        robot.set_body_ft(GUST_N, TWIST_NM);
        sent = "set_body_ft((5,0,0), (0,0,0.2))";
        break;
}

# One verb per iteration, so each printed line names exactly what went
# out on the wire. Each takes three scalars or one sequence.
if step % 3 == 0:
    mr.set_body_force(GUST_N)
    sent = f"set_body_force({GUST_N})"
elif step % 3 == 1:
    mr.set_body_torque(TWIST_NM)
    sent = f"set_body_torque({TWIST_NM})"
else:
    mr.set_body_ft(GUST_N, TWIST_NM)
    sent = f"set_body_ft({GUST_N}, {TWIST_NM})"
step += 1

The three verbs are one for one across the surfaces. C++ takes std::array<double, 3> and Python takes either three scalars or one sequence; both hide the same schema asymmetry, in which set_body_ft puts the force in vec3 and the torque in vec3_arr[0].

The state block it prints afterwards is state.wrench, the total force and torque the simulator has on the body. That is where the effect will appear the day the simulator implements these ids. Until then it shows the robot's own actuators and nothing of yours:

set_body_force([5.0, 0.0, 0.0])
    state.wrench force=(<fx>,<fy>,<fz>) N  torque=(<tx>,<ty>,<tz>) N.m  in "<frame>"

The vectors are still frame-tagged on the way out, using the coord_frame_id from your connect options. An untagged vector would be taken at face value and would silently flip sign between opposite-handed conventions, which is why the SDK always tags.

Telling ignored from wrong

You cannot, from outside. There is no acknowledgement to inspect and no error to catch, so every one of these produces the same symptom:

The one tool that separates them is the echo. Print actuator.pwm, actuator.normalized and actuator.measured every iteration, and compare their contents and their lengths against what you sent. A channel that never moves while its neighbours do points at the id; an actuator block that never moves at all while elapsed keeps advancing points at the robot, which is either of a type that does not implement the command or not the robot you meant to attach to.

Confirm which robot is which with vrobots topic list before blaming a command.

Next: Reading someone else's commands

See also: The generic command, When nothing happens, Actuators

Reading someone else's commands

A robot's command topic is a shared bus, so you can subscribe to what other publishers are sending it and use that as your own controller's input.

cargo run -p vrobots-examples --bin ex32_fw_rate_controller -- <sys_id>
./target/cpp-build/ex32_fw_rate_controller <sys_id>
python examples/python/ex32_fw_rate_controller.py <sys_id>

The one place the SDK reads z/cmd

Everywhere else a command is write-only, and rightly so: it has no reply and the state stream is the proof. subscribe_setpoint is the exception. Zenoh's vrobots/{sys_id}/z/cmd is many-to-many, so the setpoints the simulator's in-game IMU panel publishes at 50 Hz are readable by anyone who subscribes to the same key.

That enables the experiment the fixed wing was built for. From examples/rust/src/bin/ex32_fw_rate_controller.rs, three streams meet in one program:

z/cmd    ->  SET_ANGVEL from the sim's IMU panel   the setpoint (read, not written)
z/state  ->  kin.ang_vel                           the measurement
z/cmd    <-  SET_FW_SURFACES + SET_FW_THRUST       your output

The operator keeps flying with the stick, the simulator's rate PIDs are bypassed, and your gains are in their place.

The two subscriptions

Both return VrError::Deleted if the robot was deleted and VrError::Session if zenoh will not declare the subscriber. Subscribe before you take the aircraft, so an input during the handover is not missed:

#![allow(unused)]
fn main() {
    // Subscribe BEFORE taking the aircraft: a stick input during the handover
    // would otherwise be missed, and the loop would start from "no setpoint".
    let setpoints = robot.subscribe_setpoint()?;
    println!(
        "attached to sys_id={} ({:?}); watching {} for {} (id {}), ignoring src_id={own_src_id}",
        robot.sys_id(),
        robot.robot_type(),
        setpoints.key(),
        cmd::name(setpoints.cmd_id()),
        setpoints.cmd_id()
    );
}

// Subscribe BEFORE taking the aircraft: a stick input during the
// handover would otherwise be missed, and the loop would start from "no
// setpoint".
vrsdk::SetpointStream setpoints = robot.subscribe_setpoint();
std::printf(
    "attached to sys_id=%u (GlobalHawk); watching %s for SET_ANGVEL (id %u), ignoring "
    "src_id=%u\n",
    robot.sys_id(), setpoints.key().c_str(), setpoints.cmd_id(), OWN_SRC_ID);

own_src_id = robot.options["src_id"]

# Subscribe BEFORE taking the aircraft: a stick input during the handover
# would otherwise be missed, and the loop would start from "no setpoint".
setpoints = robot.subscribe_setpoint()
print(
    f"attached to sys_id={robot.sys_id} ({robot.robot_type!r}); watching "
    f"{setpoints.key} for {cmd.name(setpoints.cmd_id)} (id {setpoints.cmd_id}), "
    f"ignoring src_id={own_src_id}"
)

Knowing your own src_id is where the surfaces diverge, and it matters here because it is how you filter your own traffic off the bus. Rust and Python can read the options back (robot.options["src_id"] in Python); the C++ surface has no accessor for them, so the example sets src_id explicitly in the connect options and keeps its own constant.

attached to sys_id=<id> (GlobalHawk); watching vrobots/<id>/z/cmd for SET_ANGVEL (id 51), ignoring src_id=122

Dropping the stream undeclares the subscription and nothing else. It does not stop anyone publishing, and the robot never learns that you were listening.

What a setpoint carries

The vector is taken exactly as the sender stamped it. The simulator converts its own copy, so a controller that skips the conversion disagrees with the simulator by a permutation and a sign or two. For SET_ANGVEL off the IMU panel the frame is the target robot's own, which for a fixed wing is FRD, so it reads as [p, q, r].

Filtering out your own traffic

Everything anyone sends to the robot arrives on this stream, your own commands included. Compare Setpoint::src_id against ConnectOptions::src_id, which defaults to 122:

#![allow(unused)]
fn main() {
        // --- the setpoint: latched, so read the current one every iteration ---
        let setpoint = setpoints.latest();
        let demand = match &setpoint {
            // Our own traffic comes back on this bus too. It is not a setpoint.
            Some(sp) if sp.src_id == own_src_id => [0.0; 3],
            Some(sp) => {
}

// --- the setpoint: latched, so read the current one every iteration
const std::optional<vrsdk::Setpoint> setpoint = setpoints.latest();
double demand[3] = {0.0, 0.0, 0.0};
if (setpoint && setpoint->src_id() != OWN_SRC_ID) {
    // Our own traffic comes back on this bus too. It is not a
    // setpoint.
    const std::array<double, 3> value = setpoint->value();
    demand[0] = value[0];
    demand[1] = value[1];
    demand[2] = value[2];

# --- the setpoint: latched, so read the current one every iteration ---
setpoint = setpoints.latest
if setpoint is None or setpoint.src_id == own_src_id:
    # Nobody has ever published one, or it is our own traffic coming back
    # on this bus. "Hold zero rates" is a decision.
    demand = (0.0, 0.0, 0.0)
else:
    demand = setpoint.value

Two absences collapse into one test in every surface: no setpoint has ever arrived, and the only setpoint is your own echo. Both mean "hold zero rates" here. Note that latest is a property in Python and a method in the other two.

Skipping that arm makes a controller chase its own output, which reads as a loop that will not settle rather than as a bug in the filter.

latest, not fresh

A None from fresh() does not mean the setpoint went away. The bus latches, and a publisher that stopped sending has not commanded zero, which is why latest() is almost always the right read here.

Gotcha. Before the first setpoint ever arrives, latest() is None. Whatever you substitute is a decision, not a default: ex32 holds zero rates, which is safe on an aircraft that is already trimmed and would not be on every plant.

Stream health

SetpointStats carries received, filtered, decode_errors, seq_gaps and last_seq. filtered climbing fast is normal: it counts every other peer's traffic to the same robot, which is everything that is not the one id you asked for. seq_gaps is only meaningful with a single publisher, since two senders interleaving their own sequences look like gaps. Decode errors are counted rather than returned and never tear the subscription down.

Next: Publishing estimates

See also: Fixed wing control, Stream health, Frames, axes and units

Publishing estimates

Putting your filter's belief on the wire so the fixed wing can fly it instead of the simulator's truth.

cargo run -p vrobots-examples --bin ex35_publish_estimate -- <sys_id>
./target/cpp-build/ex35_publish_estimate <sys_id>
python examples/python/ex35_publish_estimate.py <sys_id>

The Global Hawk is scene-authored and lives in the IMU scene, so the example takes the live sys_id as an argument. Find it with vrobots topic list.

The simulator deliberately leaves the estimate empty

Every snapshot carries an estimate block, and it is always zeroed with valid = false. That is not a gap. The simulator runs no filter of its own and omits the field on purpose, because an estimate mirroring truth would make estimate.kin - kin a tautology instead of an estimator error. Truth, measured and believed is that split from the reading side. The two topics divide along the same line and travel in opposite directions.

The loop is therefore: read sensors, run your filter, publish the result, repeat.

The two entry points

Both build the same wire message. From crates/vrobots-sdk/src/robot.rs:

#![allow(unused)]
fn main() {
    pub fn publish_estimate(
        &self,
        quat: [f64; 4],
        angular_rates: Option<[f64; 3]>,
        valid: bool,
    ) -> VrResult<()>
}

    void publish_estimate(const Quat& quat, std::optional<Vec3> angular_rates = std::nullopt,
                          bool valid = true)

def publish_estimate(
    self,
    quat: Sequence[float],
    angular_rates: Optional[Sequence[float]] = None,
    valid: bool = True,
) -> None: ...

C++ and Python default angular_rates to none and valid to true; Rust asks for all three. publish_estimate_euler(euler, order, angular_rates, valid) is the same call from angles: it runs rotations::euler_to_quat and hands the result to publish_estimate, and everything after the conversion is identical.

InvalidArgument comes back for a non-finite component or a quaternion whose norm is too near zero to be an attitude. There is no reply and no ack beyond that: the confirmation is the simulator's _Est cockpit gauges moving.

Publishing one

ex35 wraps the call in a helper so all four phases publish identically. From examples/rust/src/bin/ex35_publish_estimate.rs:

#![allow(unused)]
fn main() {
    // valid = true throughout. The gyro rates go along for the ride; nothing
    // reads them yet.
    robot.publish_estimate(quat, Some(s.sensors.gyroscope.angular_velocity), true)?;
    Ok(Some(quat))
}

// valid = true throughout. The gyro rates go along for the ride; nothing
// reads them yet.
robot.publish_estimate(quat, gyro_of(s), true);
return quat;

    # valid=True throughout. The gyro rates go along for the ride; nothing reads
    # them yet.
    robot.publish_estimate(quat, s.sensors.gyroscope.angular_velocity, True)
    return quat

It prints nothing of its own and returns once zenoh has accepted the put, exactly like a command; the only observable difference is what the aircraft does next.

It must keep coming

A command latches: send set_angvel once and the setpoint stands until you change it. An estimate does the opposite. The simulator ages it from arrival, in sim time, and stops trusting it after 0.5 seconds, so a publisher that pauses has handed the aircraft back to truth. Publish it every control iteration, at 20 Hz or better, which is why every loop in ex35 pairs one publish with one rate call:

#![allow(unused)]
fn main() {
    for _ in 0..SETTLE_SAMPLES {
        publish(robot, &robot.states(), estimator)?;
        robot.rate(HZ);
    }
}

for (int i = 0; i < SETTLE_SAMPLES; ++i) {
    publish(robot, robot.states(), estimator);
    robot.rate(HZ);
}

    for _ in range(SETTLE_SAMPLES):
        publish(robot, robot.states, estimator)
        robot.rate(HZ)

The example runs at 25 Hz, inside the window, and the settle loop prints nothing; the tracking window after it prints one progress line every two seconds.

Gotcha. valid = false is a gate, not a status flag. The simulator drops the message before it reads the quaternion and does not reset the age counter, so a stream of invalid estimates is indistinguishable, sim-side, from publishing nothing at all. Sending it while your filter is unconverged is still the honest thing to do; expect no way to tell it apart from a dead publisher.

The frame comes from the header

The SDK leaves the estimate's own frame pair unset, so it inherits the header's: the coord_frame_id and axis_convention from your ConnectOptions. Your quaternion must therefore be expressed in the frame you connected with, not in the robot's, and the two agree only if you make them. ex35 connects with ConnectOptions::default().with_frame("frd", Axes::FRD) precisely so s.kin.quat can go straight back out without conversion, and it checks states().coord_frame_id against the frame it stamped rather than assuming. Where they differ, convert first: Rotation conversions is the arithmetic, and an attitude is the M * C * M^T case.

Selecting it, and the four phases

Publishing alone changes nothing. set_fw_est_source(cmd::FW_EST_OBSERVER) is what makes the onboard loop read z/estimate instead of truth. Fixed wing control is that selector, this page is the publisher, and neither works without the other. ex35 walks all four combinations, measuring each the same way so the rows compare.

Phase 1 is what makes phase 3 mean anything. A truth-copy estimator is the one whose error is exactly zero, so if phases 1 and 2 already differed, the difference would be plumbing rather than estimation. reset() runs before phase 4, because the lie costs tens of metres of altitude and phase 4 is only comparable from a level start; that reset also clears the estimate source and the rate setpoint, so the example re-sends both. The run ends with one row per phase:

what each phase flew on, and what the airframe did about it:
  1 truth source, truth-copy estimate    pitch=<deg>  roll=<deg>  r=<rad/s>  alt=<m> (<m> over 6.0 s)
  2 observer source, truth-copy estimate pitch=<deg>  roll=<deg>  r=<rad/s>  alt=<m> (<m> over 6.0 s)
  3 observer source, +5.0 deg pitch lie  pitch=<deg>  roll=<deg>  r=<rad/s>  alt=<m> (<m> over 6.0 s)
  4 observer source, nothing published   pitch=<deg>  roll=<deg>  r=<rad/s>  alt=<m> (<m> over 6.0 s)

The onboard loop reads roll and pitch out of the believed attitude for two assists, a wings-leveller and an altitude hold that biases the pitch demand. Tell it the nose is five degrees higher than it is and the pitch assist trims five degrees of nose-down to correct an error that does not exist. The rate loop tracks its demand exactly as well as in phase 1: it is being asked for the wrong thing.

Note. Five degrees is deliberately small: the altitude assist is clamped at ten, so a lie inside the clamp settles the aircraft lower instead of departing.

Next: Cameras and images

See also: Fixed wing control, Truth, measured and believed, Rotation conversions

Cameras and images

Camera frames arrive on a second transport, at their own rate, with their own timestamps, and this chapter is how you read them.

Assume a vrobot ships with front_left and front_right mounted, at 720p rgba8. That is the starting assumption throughout this chapter and every camera example in the book: reading images means open_camera on one of that pair, which changes nothing in the simulator. It holds for the multirotor and the truck; robot types that carry no camera at all are the exception, and vrobots topic list settles it in one command. Creating a camera of your own is a separate, mutating operation, covered on Lens and mount pose and used by exactly one example.

Frames never leave the host

Camera frames ride iceoryx2 shared memory, so they are same-host only. This holds even when zenoh is talking to a simulator on another machine: states, commands and services cross the network happily, and the images do not follow. The simulator loans a buffer and your process reads the same physical pages, so there is no socket for a frame to travel over and no machine boundary it can cross.

The failure is quiet rather than loud. An iceoryx2 service that does not exist on your host is not an error at the far end, it is an absence, so mount_camera and open_camera wait for a publisher that is never going to appear and return VrError::Timeout after ConnectOptions::camera_timeout (5 s). A remote simulator, a typo in the format string and a camera nobody mounted all present as the same timeout.

Gotcha. If states work and cameras time out, check where the simulator is running before you check anything else. Run the simulator and your program on one machine to use cameras at all.

The path a frame takes

Six steps, from the render to the Frame in your hand:

1. Unity renders the camera.
2. The simulator requests a GPU readback and stamps the capture time, t_ns, at that moment.
3. It loans one shared-memory sample, a fixed 5760-byte prefix followed by the pixels, and publishes it on the camera's iceoryx2 service, vrobots/{sys_id}/i/cam/{name}/{res}_{fmt}.
4. The stream's reader thread, one per CameraStream, receives the sample and parses the prefix.
5. It copies the pixel tail out, flipping rows as it copies, because the wire is bottom-up and Frame::data is top-down. Then it releases the sample: the pixels you are handed never alias shared memory.
6. Your thread calls fresh() and gets an Arc<Frame>, an owned immutable snapshot that stays valid for as long as you hold it.

The reader thread sleeps about 2 ms when a receive finds nothing, rather than spinning. At 60 fps a frame period is about 16 ms, so the added latency is under 15% of one frame, and the SDK does not burn a core on a user's behalf.

How this differs from the state stream

The two streams share a clock and nothing else.

Frame::t_ns and State::t_ns are the same clock, and Frame::elapsed and State::elapsed count from the same epoch, the robot's first state sample. That is the only relationship the two streams have. The SDK never pairs a frame with a state. Code that fuses them subtracts t_ns explicitly, as page Two cameras at once shows.

The rest of this chapter

Everything here runs against a live simulator. If a camera call times out and the simulator is on this machine, vrobots topic list prints the streams that actually exist right now; the [i] lines are the camera services.

Next: Mount, open and unmount

See also: Two transports, one simulator, The topic namespace, The vrobots command

Mount, open and unmount

Three verbs with three different effects on the simulator, only one of which changes anything.

cargo run -p vrobots-examples --bin ex13_open_camera
./target/cpp-build/ex13_open_camera
python examples/python/ex13_open_camera.py
cargo run -p vrobots-examples --bin ex17_camera_pose
./target/cpp-build/ex17_camera_pose
python examples/python/ex17_camera_pose.py

Start by opening what is already there

Every vrobot ships with front_left and front_right mounted, at 720p rgba8. That is the default assumption behind every camera example in this book: to read images you open_camera one of those two, and the simulator is not changed in any way. Mounting is for the case the pair cannot serve -- a camera somewhere else on the robot, pointing somewhere else, through a different lens, or in a different format -- and among the examples only ex17_camera_pose does it.

That is not a limitation of the API, it is an ordering of it: the mutating verb is the one with a cleanup step, a name collision to avoid and a robot-wide resolution knob behind it, and none of that is worth paying for a picture the robot is already publishing.

The three verbs

The signatures, from crates/vrobots-sdk/src/robot.rs:

#![allow(unused)]
fn main() {
fn mount_camera(&self, name: &str, resolution: &str, format: &str) -> VrResult<CameraStream>
fn mount_camera_with(&self, name: &str, resolution: &str, format: &str, options: &CameraOptions) -> VrResult<CameraStream>
fn open_camera(&self, name: &str, resolution: &str, format: &str) -> VrResult<CameraStream>
fn unmount_camera(&self, name: &str) -> VrResult<()>
fn mounted_cameras(&self) -> Vec<CameraSpec>
}

[[nodiscard]] CameraStream mount_camera(const std::string& name,
                                        const std::string& resolution = "720p",
                                        const std::string& format = "rgb8",
                                        const vrsdk_camera_options_t* options = nullptr)
[[nodiscard]] CameraStream open_camera(const std::string& name,
                                       const std::string& resolution = "720p",
                                       const std::string& format = "rgb8")
void unmount_camera(const std::string& name)
[[nodiscard]] std::vector<std::array<std::string, 3>> mounted_cameras() const

def mount_camera(
    self,
    name: str,
    resolution: str = "720p",
    format: str = "rgb8",
    *,
    mount_position: Optional[Sequence[float]] = None,
    mount_euler_deg: Optional[Sequence[float]] = None,
    fx: Optional[float] = None,
    fy: Optional[float] = None,
    near_clip: Optional[float] = None,
    far_clip: Optional[float] = None,
) -> CameraStream: ...
def open_camera(
    self, name: str, resolution: str = "720p", format: str = "rgb8"
) -> CameraStream: ...
def unmount_camera(self, name: str) -> None: ...
def mounted_cameras(self) -> list[CameraSpec]: ...

Four verbs in Rust, three in the bindings: mount_camera_with has no counterpart, because C++ takes the options as an optional fourth argument and Python takes them as keyword-only arguments. C++ also has no CameraSpec type, so mounted_cameras gives back {name, resolution, format} as a three-element array of strings. Everything else, including the defaults of "720p" and "rgb8", matches across the three.

The two mount_* calls and unmount_camera reach the robot's srv/cameras; open_camera opens a subscriber, and mounted_cameras never leaves the process.

The lifecycle

A camera is either on the robot or not, and a camera that is on the robot was put there by this handle, by another client, or by the scene. Those three cases behave differently, and the difference is the whole page.

stateDiagram-v2
  state "Mounted by you" as Yours
  state "Mounted by someone else" as Theirs
  state "Theirs, and you are reading it" as Opened
  [*] --> Unmounted
  Unmounted --> Yours: mount_camera
  Yours --> Yours: mount_camera (reconfigure)
  Yours --> Unmounted: unmount_camera
  Unmounted --> Theirs: scene default or another client
  Theirs --> Theirs: unmount_camera refused
  Theirs --> Opened: open_camera
  Opened --> Theirs: stop or drop the stream
  Opened --> Opened: unmount_camera refused

Opened is a state of your subscription, not of the camera: the camera itself does not notice that you attached, and it keeps publishing for everyone when you drop the stream.

Mounting adds exactly one camera

mount_camera is an upsert of one camera. The request names that camera and asks the simulator to add it, or to reconfigure it if the name is already there. Every other camera on the robot is left exactly as it was: the scene's own cameras, another client's cameras, ones you attached with open_camera. Their streams do not blip.

Mounting a name that is already mounted reconfigures it, and if the resolution or format changes then the stream name changes with it, so the old stream ends and a new one begins. Anything still holding the old handle is reading a service that no longer exists.

From examples/rust/src/bin/ex17_camera_pose.rs, the one example that mounts:

#![allow(unused)]
fn main() {
let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

let options = CameraOptions::default()
    .with_mount_position(MOUNT_POSITION)
    .with_mount_euler_deg(MOUNT_EULER_DEG)
    .with_focal_length(FOCAL_PX)
    .with_clip(0.2, 500.0);

// mount_camera_with CREATES the camera on the robot (srv/cameras) and
// subscribes to its iox2 stream in one call.
let cam = robot.mount_camera_with(CAMERA, RESOLUTION, FORMAT, &options)?;
println!("camera stream: {}", cam.service_name());
}

vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
robot.connect();

vrsdk::CameraStream cam = robot.mount_camera(CAMERA, RESOLUTION, FORMAT, &options);
std::printf("camera stream: %s\n", cam.service_name().c_str());

mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=SYS_ID)
mr.connect()

cam = mr.mount_camera(
    CAMERA,
    RESOLUTION,
    FORMAT,
    mount_position=MOUNT_POSITION,
    mount_euler_deg=MOUNT_EULER_DEG,
    fx=FOCAL_PX,
    fy=FOCAL_PX,
    near_clip=0.2,
    far_clip=500.0,
)
print(f"camera stream: {cam.service_name}")

service_name is a method in Rust and C++ and a property in Python, and it reports the same string in all three.

With SYS_ID = 1, CAMERA = "tilt", RESOLUTION = "720p" and FORMAT = "rgb8", that prints the iceoryx2 service name, which is what vrobots topic list shows for the same stream:

camera stream: vrobots/1/i/cam/tilt/720p_rgb8

The ack from srv/cameras is a receipt, not a result. The confirmation that the camera exists is the stream appearing, which is what mount_camera waits for before it returns.

Opening changes nothing

open_camera opens the iceoryx2 subscriber and touches the simulator not at all. Two processes can open the same stream, neither disturbs the other, and neither has to own the camera. The price is that the name, resolution and format must match the publisher exactly: on iceoryx2 those three strings are the stream identity, and there is no type negotiation behind them.

Every vrobot ships front_left and front_right at 720p rgba8, which is Unity's native readback and not rgb8 -- a detail worth getting right, since rgb8 is the signature default and asking for it here is one of the two ways to earn the timeout below. Those are the cameras every camera example reads.

From examples/rust/src/bin/ex13_open_camera.rs, opening with the failure spelled out:

#![allow(unused)]
fn main() {
let cam = match robot.open_camera(CAMERA, RESOLUTION, FORMAT) {
    Ok(cam) => cam,
    Err(VrError::Timeout(detail)) => {
        // The whole point of the example: nothing is mounted under that
        // exact identity, and there is no way for the SDK to tell you which
        // of the three strings is wrong.
        eprintln!("no publisher for {CAMERA}/{RESOLUTION}_{FORMAT}: {detail}");
        eprintln!(
            "run `vrobots topic list` -- the [i] lines are the streams that \
             do exist. A camera another process mounted then unmounted is gone."
        );
        return Ok(());
    }
    Err(other) => return Err(other),
};
}

vrsdk::CameraStream cam;
try {
    cam = robot.open_camera(CAMERA, RESOLUTION, FORMAT);
} catch (const vrsdk::Error& e) {
    if (e.code() != VRSDK_ERR_TIMEOUT) {
        throw;
    }
    // The whole point of the example: nothing is mounted under that
    // exact identity, and there is no way for the SDK to tell you which
    // of the three strings is wrong.
    std::printf("no publisher for %s/%s_%s: %s\n", CAMERA, RESOLUTION, FORMAT, e.what());
    std::printf(
        "run `vrobots topic list` -- the [i] lines are the streams that do exist. A "
        "camera another process mounted then unmounted is gone.\n");
    return 0;
}

try:
    cam = mr.open_camera(CAMERA, RESOLUTION, FORMAT)
except vrsdk.VrError as e:
    if e.code != vrsdk.err.TIMEOUT:
        raise
    # The whole point of the example: nothing is mounted under that exact
    # identity, and there is no way for the SDK to tell you which of the
    # three strings is wrong.
    print(f"no publisher for {CAMERA}/{RESOLUTION}_{FORMAT}: {e.detail}")
    print(
        "run `vrobots topic list` -- the [i] lines are the streams that do "
        "exist. A camera another process mounted then unmounted is gone."
    )
    return

The missing publisher is a timeout in every surface, so it is caught the same way it is on wait_new_state: branch on the code, re-raise anything else. C++ pays one extra line for it, because CameraStream has to be declared outside the try to outlive it.

On a running simulator it attaches and reports the stream it found:

attached to vrobots/1/i/cam/front_left/720p_rgba8 (nothing in the sim changed)
spec: name=front_left resolution=720p format=rgba8 (3686400 bytes/frame)

Unmounting removes what you mounted

unmount_camera removes exactly the name it is given and stops that stream's reader thread. Every other camera on the robot keeps streaming. It refuses a name this handle did not mount, locally, with VrError::InvalidArgument, and the message lists what this handle did mount. That is what the end of ex13_open_camera demonstrates against the scene's own front_left:

unmount_camera refused, correctly: [2] invalid_argument: camera "front_left" was not mounted by this handle (mounted: []). unmount_camera only removes what mount_camera added -- a camera attached with open_camera belongs to whoever created it

mounted_cameras() returns the specs this handle asked for, in mount order. It is not a read-back: srv/cameras has no get verb, and the robot may well carry cameras this handle knows nothing about. vrobots topic list is the read-back.

Gotcha. A program that mounts must reach its unmount_camera call before exiting, which is why ex17_camera_pose runs for a fixed frame count rather than looping forever. Ctrl-C skips the cleanup and leaves the camera mounted in a simulator that outlives your process; unmount it by mounting the same name again from a short program, or restart the simulator. The examples that only open have no such deadline.

Next: Formats and resolution

See also: Freshness, Lens and mount pose, The vrobots command

Formats and resolution

The resolutions and pixel formats on offer, and the one setting that is robot-wide.

cargo run -p vrobots-examples --bin ex15_camera_formats
./target/cpp-build/ex15_camera_formats
python examples/python/ex15_camera_formats.py

Resolution

Bare heights are accepted because the service field is a number while the stream segment is a p-suffixed string. Widths are the 16:9 partners of the height: width() == height * 16 / 9.

Pixel format

Parsing is case-insensitive. Both strings are baked into the iceoryx2 service name, so a format change is an unmount and remount rather than a field write, and every consumer has to resubscribe under the new name.

What a frame costs

Resolution times format is the whole story, since the pixels are tightly packed and there is no compression anywhere on the path.

CameraSpec::data_size() returns that number for any combination. Sixteen times between the corners is worth having for anything that needs only luminance and geometry: optical flow, fiducials, horizon detection. The stream rides shared memory, so the saving is memory bandwidth rather than network, but at 60 fps it is 200 MB/s against 13 MB/s.

ex15_camera_formats opens front_left and prices the alternatives against the frames it is actually receiving. From examples/rust/src/bin/ex15_camera_formats.rs:

#![allow(unused)]
fn main() {
const CAMERA: &str = "front_left"; // every vrobot ships front_left and front_right
const RESOLUTION: &str = "720p"; // 360p | 720p | 1080p -- robot-wide
const FORMAT: &str = "rgba8"; // mono8 | rgb8 | rgba8 -- per camera
}

constexpr const char* CAMERA = "front_left";  // every vrobot ships front_left and front_right
constexpr const char* RESOLUTION = "720p";    // 360p | 720p | 1080p -- robot-wide
constexpr const char* FORMAT = "rgba8";       // mono8 | rgb8 | rgba8 -- per camera

CAMERA = "front_left"  # every vrobot ships front_left and front_right
RESOLUTION = "720p"  # 360p | 720p | 1080p -- robot-wide
FORMAT = "rgba8"  # mono8 | rgb8 | rgba8 -- per camera

The three strings are the stream identity on the wire, so they are literal strings in every surface: there is no enum for either, and a typo in one of them is a stream that does not exist rather than a compile error.

camera stream: vrobots/1/i/cam/front_left/720p_rgba8

frame 1280x720 = 921600 px, rgba8 at 4 B/px, step=5120 B/row, 3686400 B/frame
  mono8:    921600 B/frame
   rgb8:   2764800 B/frame
  rgba8:   3686400 B/frame  <- this stream

Getting the cheap end means creating a camera, since the pair the robot ships is rgba8 and nothing reconfigures a camera you do not own. That is mount_camera, covered on Lens and mount pose and used by ex17_camera_pose -- the one example in the book that mounts.

Resolution is robot-wide

Format is per camera. Resolution is one knob for the whole robot, shared by every camera on it, and it is the only part of a mount request that is not confined to the camera being named. Changing it restarts every stream on that robot under a new service name.

Three consequences, in the order people meet them:

1. Mounting a second camera at a different resolution from one this handle already holds is refused client-side with VrError::InvalidArgument, before anything reaches the wire. Mount them all at one resolution, or unmount the others first.
2. The guard can only see this handle's cameras. Mounting a camera at 360p does not unmount the robot's front_left and front_right, but it does move the resolution knob, so they come back at 360p under new names: vrobots/1/i/cam/front_left/360p_rgba8. Whatever was reading the 720p name is reading a service that no longer exists.
3. Unmounting does not put the knob back. unmount_camera removes one camera; resolution is a separate setting, and nothing in the SDK restores it. The robot streams at 360p until something sets it again or the simulator restarts.

The refusal names the cameras in the way:

invalid_argument: camera "wide" asks for 720p but "mono" is already mounted at 360p -- resolution is one setting for the WHOLE robot, and changing it restarts every stream under a new name. Mount them all at the same resolution, or unmount the others first

Gotcha. If a camera example times out on front_left at 720p, something mounted a camera at 360p and moved the robot with it. Open front_left at "360p" instead, or restart the simulator. ex17_camera_pose is the only example here that can cause it, and it mounts at 720p precisely so that it does not.

What is checked before anything is sent

CameraSpec::parse validates all three strings at the call site, because every one of them fails later as a stream that never arrives rather than as an error.

The name has to survive being pasted into an iceoryx2 service name. The simulator does not check it and does not error on a bad one: it mounts a camera whose service can never be opened.

Next: Inside a frame

See also: Mount, open and unmount, Two cameras at once, Appendix C: Error reference

Inside a frame

Row order, stride, channel order and the metadata that rides with every image.

A Frame is an owned, immutable snapshot. The reader thread copies the pixels out of the shared-memory sample and releases the sample immediately, so a Frame you hold stays valid for as long as you keep the Arc, however long that is.

Every field

intrinsics and mount ride with every frame, which is the point: a gimballed or re-mounted camera cannot desync from its images, and there is no camera-info topic to join by timestamp. Page Lens and mount pose covers both.

Three facts about the pixels

The SDK normalises geometry and nothing else.

Rows are top-down. Row 0 is the top of the picture. The wire is bottom-up, in Unity's render order, and the SDK flips while copying, which costs nothing: it is the same memcpy per row, in reverse order.

Stride is tight. step == width * bytes_per_pixel(), always, whatever padding the wire carried. data[y * step + x * bpp] is the first byte of pixel (x, y) with no special cases.

Channels are never swapped. rgb8 is R, G, B and rgba8 is R, G, B, A, exactly as the renderer produced them. Converting for the consumers that want BGR would tax the ones that do not, so the conversion happens at the call site that needs it: OpenCV users want cvtColor(..., COLOR_RGB2BGR) once, in their own code.

Gotcha. Brightness is the wrong way to check orientation outdoors. Measured on the test scene, the sky rows run B - R = +98 and the pale desert floor runs -25, so the ground is the brighter of the two and a brightness test reports the picture upside down. Compare blue against red instead.

Reading a row

row(n) is the shortest way to sanity-check orientation and channel order at once. From examples/rust/src/bin/ex03_hello_image.rs:

#![allow(unused)]
fn main() {
/// Mean `blue - red` across one row: strongly positive for sky, negative for
/// most ground. `0.0` for mono8, which has no channels to compare.
fn blueness(frame: &Frame, row: u32) -> f64 {
    let bpp = frame.bytes_per_pixel() as usize;
    if bpp < 3 {
        return 0.0;
    }
    let Some(pixels) = frame.row(row) else {
        return 0.0;
    };
    let mut sum = 0.0;
    let mut count = 0.0;
    // Channel order is R,G,B(,A) -- the SDK normalises rows and stride, never
    // channel order, so this is the renderer's own layout.
    for pixel in pixels.chunks_exact(bpp) {
        sum += f64::from(pixel[2]) - f64::from(pixel[0]);
        count += 1.0;
    }
    if count == 0.0 { 0.0 } else { sum / count }
}
}

/// Mean `blue - red` across one row: strongly positive for sky, negative for
/// most ground. 0 for mono8, which has no channels to compare.
static double blueness(const vrsdk::Frame& frame, std::uint32_t row) {
    const std::uint32_t bpp = frame.bytes_per_pixel();
    const std::uint8_t* pixels = frame.row(row);
    if (bpp < 3 || pixels == nullptr) {
        return 0.0;
    }
    double sum = 0.0;
    // Channel order is R,G,B(,A) -- the SDK normalises rows and stride, never
    // channel order, so this is the renderer's own layout.
    for (std::uint32_t x = 0; x < frame.width(); ++x) {
        sum += static_cast<double>(pixels[x * bpp + 2]) - static_cast<double>(pixels[x * bpp]);
    }
    return frame.width() > 0 ? sum / frame.width() : 0.0;
}

def sky_ness(img: np.ndarray, row: int) -> float:
    """Mean ``blue - red`` across one row.

    Strongly positive for sky, negative for most ground. The way to recognise
    sky is that it is *blue*, not that it is bright: in this scene the desert
    floor is the brighter of the two, so a brightness test reports the picture
    upside down. Returns 0.0 for mono8, which has no channels to compare.
    """
    if img.shape[2] < 3:
        return 0.0
    line = img[row].astype(np.int16)
    return float(np.mean(line[:, 2] - line[:, 0]))

Rust and C++ walk the raw bytes: frame.row(y) hands back one row and bytes_per_pixel gives the stride within it. Python does not walk bytes at all, because cam.image is a numpy (h, w, c) uint8 array, so the same subtraction is one slice. All three index channel 2 minus channel 0, which is blue minus red in the renderer's own RGB order.

Called on row 0 and row height - 1 of a forward-facing camera, it separates sky from ground and therefore confirms both facts at once:

Image front_left t=3.214 size=(1280x720) seq=42 lag_vs_state=8.4 ms
      sky-ness (B-R) top=+98 bottom=-25 (top-down: sky above ground), fov_y=61.9 deg

mono8 has one byte per pixel, so there is no channel order and no RGB against BGR question at all: data[y * step + x] is the intensity. Getting one means mounting a camera of your own, since the pair every vrobot ships is rgba8; ex15_camera_formats prices that trade, and ex17_camera_pose is the example that mounts.

Decoding a frame with no simulator

Frame::decode(payload, epoch_ns) does exactly what the reader thread does, on a slice you supply. That is the offline half of record and replay: vrobots record --camera writes those slices byte for byte, and this turns one back into a Frame. Pass a robot's first state timestamp as epoch_ns to line elapsed up with its states, or 0 to get elapsed as raw unix seconds. It fails with VrError::Decode if the slice is shorter than the 5760-byte prefix, declares more pixel bytes than it carries, or describes a shape that is not 1, 3 or 4 bytes per pixel.

Next: Freshness

See also: Saving a frame, Timestamps and sequence numbers, Recording and testing without the simulator

Freshness

fresh, latest and wait_new_frame, and which one a control loop wants.

cargo run -p vrobots-examples --bin ex13_open_camera
./target/cpp-build/ex13_open_camera
python examples/python/ex13_open_camera.py

Three ways to read one stream

This is deliberately not how states() behaves. A control loop wants the current state every iteration whether or not it changed, so states() always hands one back. An image pipeline wants to do its work once per frame, so fresh() hands each frame out exactly once. Polling fresh() at 100 Hz against a 60 fps stream returns a frame about 60 times a second and None the rest of the time, which is the shape of the loop in ex03_hello_image:

#![allow(unused)]
fn main() {
if let Some(frame) = cam.fresh() {
    // Some only if new since the last read
    seen += 1;
}

// A value only if new since the last read.
if (auto frame = cam.fresh()) {
    ++seen;

if cam.fresh:
    frame = cam.frame  # metadata for the image we are about to read
    img = cam.image  # numpy (h, w, c) uint8, top-down, RGB(A)
    seen += 1

Python is the one that splits the question from the answer. cam.fresh is a boolean property that asks, and cam.image or cam.read() is what consumes the freshness; a loop that tests cam.fresh and then never reads keeps seeing the same frame as new.

That fragment prints nothing on its own: it is the guard deciding whether the image half of the loop runs at all, and ex03_hello_image prints a state line from the else arm instead.

Each frame is handed out once even when two threads race on the same stream: the claim is a compare-and-exchange, so exactly one caller gets a given frame and the other gets None. Reach for latest() when you want the current picture regardless, for a viewer or a status line, and it will not steal a frame from the thread doing the real work.

Pacing on frames instead of the clock

wait_new_frame blocks until a frame newer than the last stored one arrives, so the loop body runs exactly once per rendered frame with no polling at all. From examples/rust/src/bin/ex13_open_camera.rs:

#![allow(unused)]
fn main() {
let mut seen = 0u64;
while seen < FRAMES {
    if let Err(VrError::Timeout(_)) = cam.wait_new_frame(Duration::from_millis(500)) {
        println!("no frame in 500 ms -- the camera stopped, or the sim is paused");
        continue;
    }
    let Some(frame) = cam.fresh() else { continue };
    seen += 1;
}

std::uint64_t seen = 0;
while (seen < FRAMES) {
    try {
        cam.wait_new_frame(TIMEOUT_S);
    } catch (const vrsdk::Error& e) {
        if (e.code() != VRSDK_ERR_TIMEOUT) {
            throw;
        }
        std::printf("no frame in %.1fs -- the camera stopped, or the sim is paused\n",
                    TIMEOUT_S);
        continue;
    }
    const std::optional<vrsdk::Frame> frame = cam.fresh();
    if (!frame) {
        continue;
    }
    ++seen;

seen = 0
while seen < FRAMES:
    try:
        cam.wait_new_frame(TIMEOUT)
    except vrsdk.VrError as e:
        if e.code != vrsdk.err.TIMEOUT:
            raise
        print(f"no frame in {TIMEOUT}s -- the camera stopped, or the sim is paused")
        continue

    frame = cam.read()  # consumes freshness; None if someone else got it
    if frame is None:
        continue
    seen += 1

wait_new_frame takes seconds as a double in C++ and Python where Rust takes a Duration, and the timeout is the same status rather than a failure in all three. Note the second guard: wait_new_frame returning does not guarantee the read succeeds, because another thread may have taken the frame in between, so the empty case is still handled.

While frames are arriving, nothing is printed by the timeout arm:

frame 1: seq=17 1280x720 3686400 bytes, mount=(+0.00,+0.00,+0.00) m
frame 11: seq=27 1280x720 3686400 bytes, mount=(+0.00,+0.00,+0.00) m

wait_new_frame returns VrError::Timeout when nothing new arrives in time, and the message names the service and the deadline:

timeout: no camera frame on vrobots/1/i/cam/front_left/720p_rgba8 within 500ms

Gotcha. A timeout here never means the stream is broken, and it is also how a stopped camera presents: a paused simulator, a camera someone else unmounted, and a genuinely slow render are the same event. Treat it as a condition to handle, not an error to propagate, which is why the example loop continues rather than returning.

The counters

stats() returns a CameraStats snapshot of the reader thread's counters.

Because seq is contiguous by construction, a non-zero seq_gaps is a real shared-memory drop rather than a skipped render. A malformed slice is counted and dropped, exactly like a malformed state payload: one bad frame must not end a flight. When decode_errors is non-zero, last_error() returns the most recent failure.

received counts what the thread stored, not what you read. A loop that polls more slowly than the camera renders sees fewer frames than received reports, and that difference is your loop falling behind, not a drop.

read 60 frame(s): received=60 decode_errors=0 seq_gaps=0 missed_frames=0

Stopping is not unmounting

stop() ends the reader thread, and Drop does the same and then joins it. Joining rather than detaching is deliberate: the thread holds an iceoryx2 subscriber, and letting it outlive the handle would leave a port attached to a service nobody can see. The wait is bounded by one poll interval, about 2 ms, plus the frame in flight.

Neither one unmounts the camera. The robot keeps rendering and publishing for everyone else, exactly as dropping a VirtualRobot leaves the robot running. unmount_camera is the verb for that, and it works only on cameras this handle mounted.

is_running() reports whether the reader thread is still alive. It is false after stop(), after unmount_camera removed this camera, or if the thread ended on its own.

Next: Lens and mount pose

See also: Mount, open and unmount, Pacing your loop, Stream health

Lens and mount pose

Where the camera sits, what it sees, and what the simulator substitutes for values it does not like.

cargo run -p vrobots-examples --bin ex17_camera_pose
./target/cpp-build/ex17_camera_pose
python examples/python/ex17_camera_pose.py

This is the page where a camera gets created rather than opened, and ex17_camera_pose is the one example in the book that does it. Everywhere else the assumption holds that every vrobot already ships front_left and front_right at 720p rgba8, and a reader just opens one. You reach for mount_camera when that pair cannot serve: a camera somewhere else on the robot, pointing somewhere else, through a different lens, or in a different format.

mount_camera uses the defaults: at the robot origin, looking along its forward axis, 600 px focal length. mount_camera_with takes a CameraOptions and configures the mount and the lens in the same call. Whatever it creates is yours to remove, and unmount_camera at the end of the run is what keeps the robot's own two cameras the only ones left.

What you can ask for

The two mount fields are tagged differently on purpose, matching the simulator. Position is a frame-tagged vector, re-expressed from the frame your headers declare into the robot's own. The euler triple is a Unity-local mount convention and is taken as written, in degrees, whatever frame you declared.

The lens is specified as intrinsics, not as a field of view. fx != fy renders anamorphic. A smaller focal length is a wider angle: fov_y = 2 * atan(height / (2 * fy)), which puts the 600 px default at about 61.9 degrees on a 720p frame and 400 px at about 84.0.

From examples/rust/src/bin/ex17_camera_pose.rs, the whole configuration:

#![allow(unused)]
fn main() {
let options = CameraOptions::default()
    .with_mount_position(MOUNT_POSITION)
    .with_mount_euler_deg(MOUNT_EULER_DEG)
    .with_focal_length(FOCAL_PX)
    .with_clip(0.2, 500.0);
println!("requested: {options:?}");

let cam = robot.mount_camera_with(CAMERA, RESOLUTION, FORMAT, &options)?;
println!("camera stream: {}", cam.service_name());
}

// Start from the documented defaults, then override.
vrsdk_camera_options_t options;
vrsdk_camera_options_default(&options);
for (int i = 0; i < 3; ++i) {
    options.mount_position[i] = MOUNT_POSITION[i];
    options.mount_euler_deg[i] = MOUNT_EULER_DEG[i];
}
options.fx = FOCAL_PX;
options.fy = FOCAL_PX;
options.near_clip = 0.2;
options.far_clip = 500.0;

vrsdk::CameraStream cam = robot.mount_camera(CAMERA, RESOLUTION, FORMAT, &options);
std::printf("camera stream: %s\n", cam.service_name().c_str());

cam = mr.mount_camera(
    CAMERA,
    RESOLUTION,
    FORMAT,
    mount_position=MOUNT_POSITION,
    mount_euler_deg=MOUNT_EULER_DEG,
    fx=FOCAL_PX,
    fy=FOCAL_PX,
    near_clip=0.2,
    far_clip=500.0,
)
print(f"camera stream: {cam.service_name}")

Only Rust needs a second entry point. C++ takes an optional fourth argument on mount_camera, defaulting to a null pointer, and Python takes the same settings as keyword-only arguments, so there is no mount_camera_with in either.

Start the C++ struct from vrsdk_camera_options_default, never from {}. A zeroed struct asks for a zero focal length and zero clip planes, which is not the defaults; Rust's CameraOptions::default() and Python's omitted keywords are the equivalents.

With MOUNT_EULER_DEG = [0.0, 0.0, 180.0] the camera is upside down, and the stream name is the usual one:

camera stream: vrobots/1/i/cam/tilt/720p_rgb8

What comes back

Both blocks ride with every frame, so a gimballed or re-mounted camera cannot desync from its images and there is no camera-info topic to join by timestamp.

frame.intrinsics is read back from the live camera rather than echoed from your request, because the simulator round-trips fx and fy through the Unity field of view. The numbers can differ slightly from the ones you sent, and the read-back is the authority on what rendered the frame.

Rendered images are an ideal pinhole. There are no distortion coefficients, because there is no distortion.

Degrees on the way in, radians on the way out, so the read-back needs converting before it is comparable to what you asked for:

#![allow(unused)]
fn main() {
// Degrees on the way in, radians on the way out: the wire is SI.
let euler_deg = [
    frame.mount.euler_rad[0].to_degrees(),
    frame.mount.euler_rad[1].to_degrees(),
    frame.mount.euler_rad[2].to_degrees(),
];
}

const vrsdk_mount_pose_t& m = frame->info.mount;
const vrsdk_intrinsics_t& in = frame->info.intrinsics;

// Degrees on the way in, radians on the way out: the wire is SI.
const double euler_deg[3] = {m.euler_rad[0] * RAD2DEG, m.euler_rad[1] * RAD2DEG,
                             m.euler_rad[2] * RAD2DEG};

m, i = frame.mount, frame.intrinsics

# Degrees on the way in, radians on the way out: the wire is SI.
euler_deg = tuple(round(math.degrees(a), 1) for a in m.euler_rad)

The asymmetry is the wire's, not the binding's: mount_euler_deg goes out in degrees and mount.euler_rad comes back in radians, in all three. Every surface converts at the same place, and the field names carry their units for exactly this reason.

That fragment prints nothing itself; it feeds the pose line ex17_camera_pose prints whenever the mount differs from the previous frame's.

The read-back is not the numbers you sent

You express the mount in your frame; the robot converts it into its frame and reports it back tagged with that frame. Against the test scene, the components move and change sign: