Mount, open and unmount

Three verbs with three different effects on the simulator, only one of which changes anything.

cargo run -p vrobots-examples --bin ex13_open_camera
./target/cpp-build/ex13_open_camera
python examples/python/ex13_open_camera.py
cargo run -p vrobots-examples --bin ex17_camera_pose
./target/cpp-build/ex17_camera_pose
python examples/python/ex17_camera_pose.py

Start by opening what is already there

Every vrobot ships with front_left and front_right mounted, at 720p rgba8. That is the default assumption behind every camera example in this book: to read images you open_camera one of those two, and the simulator is not changed in any way. Mounting is for the case the pair cannot serve – a camera somewhere else on the robot, pointing somewhere else, through a different lens, or in a different format – and among the examples only ex17_camera_pose does it.

That is not a limitation of the API, it is an ordering of it: the mutating verb is the one with a cleanup step, a name collision to avoid and a robot-wide resolution knob behind it, and none of that is worth paying for a picture the robot is already publishing.

The three verbs

Verb	Mutates the simulator	Needs the camera to exist first	Can it undo itself
`mount_camera` / `mount_camera_with`	yes, `srv/cameras`	no, it creates the camera	yes, with `unmount_camera`
`open_camera`	no, subscribe only	yes, exactly this name, resolution and format	nothing to undo
`unmount_camera`	yes, `srv/cameras`	it must be one this handle mounted	it is the undo

The signatures:

crates/vrobots-sdk/src/camera.rs:

#![allow(unused)]
fn main() {
pub fn mount_camera(
    &self,
    name: &str,
    resolution: &str,
    format: &str,
) -> VrResult<CameraStream>

pub fn mount_camera_with(
    &self,
    name: &str,
    resolution: &str,
    format: &str,
    options: &CameraOptions,
) -> VrResult<CameraStream>

pub fn open_camera(
    &self,
    name: &str,
    resolution: &str,
    format: &str,
) -> VrResult<CameraStream>

pub fn unmount_camera(&self, name: &str) -> VrResult<()>

pub fn mounted_cameras(&self) -> Vec<CameraSpec>
}

include/vrobots_sdk.hpp:

[[nodiscard]] CameraStream mount_camera(const std::string& name,
                                        const std::string& resolution = "720p",
                                        const std::string& format = "rgb8",
                                        const vrsdk_camera_options_t* options = nullptr)
[[nodiscard]] CameraStream open_camera(const std::string& name,
                                       const std::string& resolution = "720p",
                                       const std::string& format = "rgb8")
void unmount_camera(const std::string& name)
[[nodiscard]] std::vector<std::array<std::string, 3>> mounted_cameras() const

vrsdk/_vrsdk.pyi:

def mount_camera(
    self,
    name: str,
    resolution: str = "720p",
    format: str = "rgb8",
    *,
    mount_position: Optional[Sequence[float]] = None,
    mount_euler_deg: Optional[Sequence[float]] = None,
    fx: Optional[float] = None,
    fy: Optional[float] = None,
    near_clip: Optional[float] = None,
    far_clip: Optional[float] = None,
) -> CameraStream: ...
def open_camera(
    self, name: str, resolution: str = "720p", format: str = "rgb8"
) -> CameraStream: ...
def unmount_camera(self, name: str) -> None: ...
def mounted_cameras(self) -> list[CameraSpec]: ...

Four verbs in Rust, three in C++ and Python: mount_camera_with has no counterpart, because C++ takes the options as an optional fourth argument and Python takes them as keyword-only arguments. C++ also has no CameraSpec type, so mounted_cameras gives back {name, resolution, format} as a three-element array of strings. C++ and Python default the resolution and format to "720p" and "rgb8", where Rust, which has no default arguments, takes all three strings on every call. Everything else matches across the three.

The two mount_* calls and unmount_camera reach the robot’s srv/cameras; open_camera opens a subscriber, and mounted_cameras never leaves the process.

The lifecycle

A camera is either on the robot or not, and a camera that is on the robot was put there by this handle, by another client, or by the scene. Those three cases behave differently, and the difference is the whole page.

stateDiagram-v2
  state "Mounted by you" as Yours
  state "Mounted by someone else" as Theirs
  state "Theirs, and you are reading it" as Opened
  [*] --> Unmounted
  Unmounted --> Yours: mount_camera
  Yours --> Yours: mount_camera (reconfigure)
  Yours --> Unmounted: unmount_camera
  Unmounted --> Theirs: scene default or another client
  Theirs --> Theirs: unmount_camera refused
  Theirs --> Opened: open_camera
  Opened --> Theirs: stop or drop the stream
  Opened --> Opened: unmount_camera refused

Opened is a state of your subscription, not of the camera: the camera itself does not notice that you attached, and it keeps publishing for everyone when you drop the stream.

Mounting adds exactly one camera

mount_camera is an upsert of one camera. The request names that camera and asks the simulator to add it, or to reconfigure it if the name is already there. Every other camera on the robot is left exactly as it was: the scene’s own cameras, another client’s cameras, ones you attached with open_camera. Their streams do not blip.

Mounting a name that is already mounted reconfigures it, and if the resolution or format changes then the stream name changes with it, so the old stream ends and a new one begins. Anything still holding the old handle is reading a service that no longer exists.

From examples/rust/src/bin/ex17_camera_pose.rs, the one example that mounts:

#![allow(unused)]
fn main() {
let robot = VirtualRobot::connect(RobotType::Multirotor, Some(SYS_ID))?;

let options = CameraOptions::default()
    .with_mount_position(MOUNT_POSITION)
    .with_mount_euler_deg(MOUNT_EULER_DEG)
    .with_focal_length(FOCAL_PX)
    .with_clip(0.2, 500.0);
println!("requested: {options:?}");

let cam = robot.mount_camera_with(CAMERA, RESOLUTION, FORMAT, &options)?;
println!("camera stream: {}", cam.service_name());
}

examples/cpp/ex17_camera_pose.cpp:

vrsdk::VirtualRobot robot(vrsdk::RobotType::Multirotor, SYS_ID);
robot.connect();

vrsdk::CameraStream cam = robot.mount_camera(CAMERA, RESOLUTION, FORMAT, &options);
std::printf("camera stream: %s\n", cam.service_name().c_str());

examples/python/ex17_camera_pose.py:

mr = VirtualRobot(RobotType.MULTIROTOR, sys_id=1)  # sys_id 1 = multirotor, 0 = truck
mr.connect()

# the ONE example that mounts its own camera; all others open front_left/front_right
print(
    f"requested: position={MOUNT_POSITION} m, euler={MOUNT_EULER_DEG} deg, "
    f"fx=fy={FOCAL_PX} px"
)
cam = mr.mount_camera(
    "tilt",
    "720p",
    "rgb8",
    mount_position=MOUNT_POSITION,
    mount_euler_deg=MOUNT_EULER_DEG,
    fx=FOCAL_PX,
    fy=FOCAL_PX,
    near_clip=0.2,
    far_clip=500.0,
)
print(f"camera stream: {cam.service_name}")

The Python script passes the robot id, the camera name, the resolution and the format as literals where Rust and C++ use constants; all three print the requested pose and lens before they mount.

service_name is a method in Rust and C++ and a property in Python, and it reports the same string in all three.

With SYS_ID = 1, CAMERA = "tilt", RESOLUTION = "720p" and FORMAT = "rgb8", that prints the iceoryx2 service name, which is what vrobots topic list shows for the same stream:

camera stream: vrobots/1/i/cam/tilt/720p_rgb8

The ack from srv/cameras is a receipt, not a result. The confirmation that the camera exists is the stream appearing, which is what mount_camera waits for before it returns.

Opening changes nothing

open_camera opens the iceoryx2 subscriber and touches the simulator not at all. Two processes can open the same stream, neither disturbs the other, and neither has to own the camera. The price is that the name, resolution and format must match the publisher exactly: on iceoryx2 those three strings are the stream identity, and there is no type negotiation behind them.

Every vrobot ships front_left and front_right at 720p rgba8, which is Unity’s native readback and not rgb8 – a detail worth getting right, since rgb8 is the default in the C++ and Python signatures and asking for it here is one of the two ways to earn the timeout below. Those are the cameras every camera example reads.

From examples/rust/src/bin/ex13_open_camera.rs, opening with the failure spelled out:

#![allow(unused)]
fn main() {
let cam = match robot.open_camera(CAMERA, RESOLUTION, FORMAT) {
    Ok(cam) => cam,
    Err(VrError::Timeout(detail)) => {
        // The whole point of the example: nothing is mounted under that
        // exact identity, and there is no way for the SDK to tell you which
        // of the three strings is wrong.
        eprintln!("no publisher for {CAMERA}/{RESOLUTION}_{FORMAT}: {detail}");
        eprintln!(
            "run `cargo run -p vrobots-examples --bin ex11_topic_discovery`: its \
             [i] lines are the streams that do exist. A camera another process \
             mounted then unmounted is gone."
        );
        return Ok(());
    }
    Err(other) => return Err(other),
};
}

examples/cpp/ex13_open_camera.cpp:

vrsdk::CameraStream cam;
try {
    cam = robot.open_camera(CAMERA, RESOLUTION, FORMAT);
} catch (const vrsdk::Error& e) {
    if (e.code() != VRSDK_ERR_TIMEOUT) {
        throw;
    }
    // The whole point of the example: nothing is mounted under that
    // exact identity, and there is no way for the SDK to tell you which
    // of the three strings is wrong.
    std::printf("no publisher for %s/%s_%s: %s\n", CAMERA, RESOLUTION, FORMAT, e.what());
    std::printf(
        "run `vrobots topic list` -- the [i] lines are the streams that do exist. A "
        "camera another process mounted then unmounted is gone.\n");
    return 0;
}

examples/python/ex13_open_camera.py:

# rgba8: 4 channels, not rgb8; name/resolution/format are the stream identity
cam = mr.open_camera("front_left", "720p", "rgba8")
print(f"attached to {cam.service_name} (nothing in the sim changed)")
spec = cam.spec
print(f"spec: name={spec.name} resolution={spec.resolution} format={spec.format}")

The Python script does not catch the failure. When nothing publishes that exact name, resolution and format, open_camera raises vrsdk.VrError with code == vrsdk.err.TIMEOUT once camera_timeout (five seconds by default) has passed, and the script ends with that traceback. To handle it as the Rust and C++ programs do, catch vrsdk.VrError, compare e.code against vrsdk.err.TIMEOUT, and re-raise any other code.

The missing publisher is a timeout in every surface. Rust and C++ catch it the same way they catch one on wait_new_state: branch on the code, re-raise anything else. C++ pays one extra line for it, because CameraStream has to be declared outside the try to outlive it.

On a running simulator it attaches and reports the stream it found:

attached to vrobots/1/i/cam/front_left/720p_rgba8 (nothing in the sim changed)
spec: name=front_left resolution=720p format=rgba8 (3686400 bytes/frame)

The Python script prints the same two lines without the (3686400 bytes/frame) suffix, because its CameraSpec carries only the name, the resolution and the format.

Unmounting removes what you mounted

unmount_camera removes exactly the name it is given and stops that stream’s reader thread. Every other camera on the robot keeps streaming. It refuses a name this handle did not mount, locally, with VrError::InvalidArgument, and the message lists what this handle did mount. That is what the end of the Rust and C++ ex13_open_camera demonstrates against the scene’s own front_left:

unmount_camera refused, correctly: [2] invalid_argument: camera "front_left" was not mounted by this handle (mounted: []). unmount_camera only removes what mount_camera added -- a camera attached with open_camera belongs to whoever created it

In Python the same refusal raises vrsdk.VrError with code == vrsdk.err.INVALID_ARGUMENT and the same message. The Python ex13_open_camera.py does not call unmount_camera; it ends after printing its counters.

mounted_cameras() returns the specs this handle asked for, in mount order. It is not a read-back: srv/cameras has no get verb, and the robot may well carry cameras this handle knows nothing about. vrobots topic list is the read-back.

Gotcha. A program that mounts must reach its unmount_camera call before exiting, which is why ex17_camera_pose runs for a fixed frame count rather than looping forever. Ctrl-C skips the cleanup and leaves the camera mounted in a simulator that outlives your process; unmount it by mounting the same name again from a short program, or restart the simulator. The examples that only open have no such deadline.

Next: Formats and resolution

Keyboard shortcuts