Perception — Sensor Calibration

Kalibr Calibration Results

Spatial and temporal calibration of the ELP global shutter stereo camera with the STM LSM6DSOX IMU using Kalibr. The calibration estimates intrinsics, lens distortion, camera-IMU extrinsics, and the time offset between sensor clocks — all required for accurate stereo odometry in ORB-SLAM3 and visual-inertial fusion in RTAB-Map.

Calibration Quality

Residuals

Reprojection error measures how accurately the calibrated model reprojects 3D AprilGrid corners back onto the image plane. Values below ~1.5 px are considered good for a visual-inertial system. IMU residuals (gyroscope and accelerometer) are normalized by the noise model — values well below 1.0 indicate the IMU noise parameters are accurately characterized.

cam0 — Reprojection error scatter (px)

cam1 — Reprojection error scatter (px)

Camera Model

Intrinsics & Distortion

Both cameras use a pinhole model with radial-tangential (radtan) distortion. The focal lengths are nearly identical across both cameras (~877–881 px), confirming consistent optics. The large k1 value (~−0.40) indicates significant barrel distortion typical of wide-angle lenses, which Kalibr precisely corrects.

Stereo Geometry

Stereo Baseline & Camera-IMU Extrinsics

The stereo baseline is the physical distance between the two camera optical centers. Kalibr estimates the full 4×4 rigid-body transform between them. The camera-IMU extrinsics (T_ci) give the rotation and translation from the IMU frame to each camera frame — these are fed into RTAB-Map for visual-inertial fusion. ORB-SLAM3 uses only the stereo camera extrinsics (no IMU).

The ~51 ms timeshift means the camera timestamps lag behind the IMU clock by ~51 ms. This offset is relatively high because the stereo camera and the IMU are not hardware-synchronized — both devices feed data over USB, which introduces variable, software-managed latency on each side. Without a hardware trigger or shared clock, this kind of offset is expected and unavoidable. Kalibr estimates it precisely and corrects for it automatically, ensuring IMU pre-integration is correctly aligned with camera frames during visual-inertial fusion.

[[-0.99957  -0.01041   0.02726   0.01871]
 [-0.02719  -0.00681  -0.99961   0.00473]
 [ 0.01059  -0.99992   0.00652  -0.03436]
 [ 0.        0.        0.        1.     ]]

[[-0.99944  -0.01987   0.02710  -0.04121]
 [-0.02704  -0.00341  -0.99963   0.00478]
 [ 0.01996  -0.99980   0.00287  -0.03443]
 [ 0.        0.        0.        1.     ]]

IMU Characterization

IMU Noise Model

The STM LSM6DSOX IMU runs at 416 Hz. Kalibr estimates continuous-time noise density and random walk parameters for both the accelerometer and gyroscope. These values are used directly in the EKF (robot_localization) to correctly weight IMU measurements during EKF state estimation for RTAB-Map. ORB-SLAM3 uses only the stereo camera and does not consume IMU data.

Estimated IMU poses — 3D trajectory during calibration

IMU sample rate (avg dt = 2.40 ± 0.62 ms @ 416 Hz)

Predicted vs. measured specific force (accelerometer, 3 axes)

Accelerometer error (3 axes vs. error index)

Estimated accelerometer bias over time (3 axes)

Predicted vs. measured angular velocity (gyroscope, 3 axes)

Gyroscope error (3 axes vs. error index)

Estimated gyroscope bias over time (3 axes)

Calibration Target

AprilGrid Configuration

Kalibr uses an AprilGrid — a grid of unique AprilTag fiducial markers — as the calibration target. Unlike a standard checkerboard, each tag has a unique ID so partial occlusions and out-of-order detections are handled robustly.