Since the RGB and depth sensors sit on separate modules, their perspectives differ slightly. To unify them, the DepthVista SDK employs pre-calibrated parameters such as the intrinsic camera matrix (focal length and optical centre), distortion coefficients, and both transformation and rotation matrices. These correct for distortion and spatial misalignment.
Through the SDK’s SetRGBDMapping API, developers can easily apply this calibration and enable RGB-D mapping. This ensures that depth information overlays accurately onto RGB images, synchronising their fields of view and resolution.
Once aligned, the SDK can generate RGB-D point clouds, essential for 3D modelling and analysis. Using intrinsic and extrinsic matrices, each pixel’s colour is projected onto corresponding depth points. The GetDeviceCalibrationParams API provides these matrices, and the mapping can be executed using a sample script (rgbdMapping.py), which runs on both Windows and Linux.
Installing the SDK requires only a few terminal or command prompt steps, with installation scripts provided for both operating systems. Once installed, the Python script lets users:
- Select the connected camera device
- Stream synchronised RGB and depth data
- Capture images and 3D point clouds
- Exit cleanly from the application
The script’s internal workflow includes device selection, enabling RGB-D mapping, fetching frames, and generating point clouds. The point cloud itself is computed by translating 2D depth values and corresponding RGB pixels into 3D coordinates using the RGB camera’s focal length and principal point.
A common error related to missing libraries (e.g., libdc1394) can occur on Linux but is easily resolved with a package install command.
Overall, DepthVista’s RGB-D mapping feature provides developers with a robust pipeline for building spatially accurate 3D vision systems.
To read the full article, click here.
