Formats and resolution

The resolutions and pixel formats on offer, and the one setting that is robot-wide.

cargo run -p vrobots-examples --bin ex15_camera_formats
./target/cpp-build/ex15_camera_formats
python examples/python/ex15_camera_formats.py

Resolution

Bare heights are accepted because the service field is a number while the stream segment is a p-suffixed string. Widths are the 16:9 partners of the height: width() == height * 16 / 9.

Pixel format

Parsing is case-insensitive. Both strings are baked into the iceoryx2 service name, so a format change is an unmount and remount rather than a field write, and every consumer has to resubscribe under the new name.

What a frame costs

Resolution times format is the whole story, since the pixels are tightly packed and there is no compression anywhere on the path.

CameraSpec::data_size() returns that number for any combination. Sixteen times between the corners is worth having for anything that needs only luminance and geometry: optical flow, fiducials, horizon detection. The stream rides shared memory, so the saving is memory bandwidth rather than network, but at 60 fps it is 200 MB/s against 13 MB/s.

ex15_camera_formats opens front_left and prices the alternatives against the frames it is actually receiving. From examples/rust/src/bin/ex15_camera_formats.rs:

#![allow(unused)]
fn main() {
const CAMERA: &str = "front_left"; // every vrobot ships front_left and front_right
const RESOLUTION: &str = "720p"; // 360p | 720p | 1080p -- robot-wide
const FORMAT: &str = "rgba8"; // mono8 | rgb8 | rgba8 -- per camera
}

constexpr const char* CAMERA = "front_left";  // every vrobot ships front_left and front_right
constexpr const char* RESOLUTION = "720p";    // 360p | 720p | 1080p -- robot-wide
constexpr const char* FORMAT = "rgba8";       // mono8 | rgb8 | rgba8 -- per camera

CAMERA = "front_left"  # every vrobot ships front_left and front_right
RESOLUTION = "720p"  # 360p | 720p | 1080p -- robot-wide
FORMAT = "rgba8"  # mono8 | rgb8 | rgba8 -- per camera

The three strings are the stream identity on the wire, so they are literal strings in every surface: there is no enum for either, and a typo in one of them is a stream that does not exist rather than a compile error.

camera stream: vrobots/1/i/cam/front_left/720p_rgba8

frame 1280x720 = 921600 px, rgba8 at 4 B/px, step=5120 B/row, 3686400 B/frame
  mono8:    921600 B/frame
   rgb8:   2764800 B/frame
  rgba8:   3686400 B/frame  <- this stream

Getting the cheap end means creating a camera, since the pair the robot ships is rgba8 and nothing reconfigures a camera you do not own. That is mount_camera, covered on Lens and mount pose and used by ex17_camera_pose -- the one example in the book that mounts.

Resolution is robot-wide

Format is per camera. Resolution is one knob for the whole robot, shared by every camera on it, and it is the only part of a mount request that is not confined to the camera being named. Changing it restarts every stream on that robot under a new service name.

Three consequences, in the order people meet them:

1. Mounting a second camera at a different resolution from one this handle already holds is refused client-side with VrError::InvalidArgument, before anything reaches the wire. Mount them all at one resolution, or unmount the others first.
2. The guard can only see this handle's cameras. Mounting a camera at 360p does not unmount the robot's front_left and front_right, but it does move the resolution knob, so they come back at 360p under new names: vrobots/1/i/cam/front_left/360p_rgba8. Whatever was reading the 720p name is reading a service that no longer exists.
3. Unmounting does not put the knob back. unmount_camera removes one camera; resolution is a separate setting, and nothing in the SDK restores it. The robot streams at 360p until something sets it again or the simulator restarts.

The refusal names the cameras in the way:

invalid_argument: camera "wide" asks for 720p but "mono" is already mounted at 360p -- resolution is one setting for the WHOLE robot, and changing it restarts every stream under a new name. Mount them all at the same resolution, or unmount the others first

Gotcha. If a camera example times out on front_left at 720p, something mounted a camera at 360p and moved the robot with it. Open front_left at "360p" instead, or restart the simulator. ex17_camera_pose is the only example here that can cause it, and it mounts at 720p precisely so that it does not.

What is checked before anything is sent

CameraSpec::parse validates all three strings at the call site, because every one of them fails later as a stream that never arrives rather than as an error.

The name has to survive being pasted into an iceoryx2 service name. The simulator does not check it and does not error on a bad one: it mounts a camera whose service can never be opened.

Next: Inside a frame

See also: Mount, open and unmount, Two cameras at once, Appendix C: Error reference