top of page
Search

Three-Camera Open-VINS Server on the VOXL 2

Updated: May 7

Multi-camera tracking no longer requires a large production run or millions in VC capital to be within reach for your drone platform. The VOXL® 2 and its Open-VINS server now enable cost-effective and scalable multi-camera tracking solutions with visual-inertial odometry (VIO) and tracking. Although two-camera tracking is common, this guide will walk you through a three-camera setup on the VOXL 2 that provides expanded capabilities and added redundancy for your tracking needs. This tutorial will cover everything you need for the setup, but please refer to the official ModalAI documentation for a more detailed look at Open-VINS.

What You’ll Need

  • Drone (Starling V1 used in tutorial)

  • VOXL2 running voxl-SDK 1.4

  • 3 AR0144 tracking cameras

  • VOXL 2 breakout board

  • Linux computer for configuration

    • Possible on other OS’s as long as you have ADB on the system

Important Considerations

Why Triple Camera Tracking?

With the limited resources on a drone, we all know that more does not always equal better, but in the case of adding cameras for tracking, more provides some advantages. While two-camera setups are sufficient for many applications, a triple-camera configuration offers superior coverage and redundancy. Having cameras facing in different directions ensures that the system can maintain robust tracking regardless of flight orientation or lighting conditions. In our setup, we'll have forward, rear, and upward-facing cameras instead of the traditional down-facing configuration, as floor reflections can cause tracking issues in bright environments. This exposure issue, caused by the reflections, is also the reason why we will not be using two 45-degree cameras for this project. 

Camera Placement and Extrinsics.

Camera placement impacts overall system performance, the ability to detect errors, and the extrinsics configuration. Extrinsics are the literal geometrical relationships between sensors on the drone, and the extrinsics configuration affects the drone’s ability to correctly assess the translation and rotation of each camera as well as the temporal relationships between sensors. It may not be obvious, even the length of a wire can affect the extrinsics quite noticeably if they are not configured correctly.

The extrinsics consist of:

  • RPY: Three-parameter representations for roll, pitch, and yaw

  • Time offsets: Temporal alignment parameters that account for sensor synchronization differences handled by the VOXL-Open-VINS-server

Why That Matters

Accurate extrinsics ensures that when a visual feature is tracked across multiple cameras over time, the system can correctly associate camera frame changes with the drone’s motion.

Starting Fresh with Open-VINS

Although we are utilizing a Starling V1(Single camera tracking only), in order to make this tutorial applicable to non-Starling drones, we will start with a clean slate with no necessary configuration files. 


Initial Setup

Firmware

  1. Download the most recent VOXL SDK (we'll use version 1.4) from the ModalAI website

  2. Extract the SDK in your terminal with the following command:

tar -xvzf voxl2_SDK_1.4.tar.gz

  1. Connect the VOXL 2 to your computer using USB

  2. Power up the VOXL 2

  3. Navigate to the extracted SDK directory and run the installation script:

cd voxl2_SDK_1.4 && ./install.sh

  1. Follow the prompts and give the following responses

    • Select "yes" to factory install

    • Choose the firmware version "w/ M0173 Breakout Board..."

    • When prompted for camera configuration, select option 27 for triple camera tracking, hires, and TOF sensor

Camera Configuration

Connect to your VOXL 2 through an ADB shell or SSH connection, then configure the system for triple camera tracking by executing the command voxl-configure-cameras 27 in the terminal, This command will set up all three cameras (including the high-resolution camera) as well as the TOF (Time of Flight) sensor, preparing the hardware for the Open-VINS visual odometry implementation.



Configuring the Open-VINS Server

Now we are going to start setting up our Open-VINS server to accurately track the drone’s position and orientation from our triple camera setup, providing reliable odometry data to the autopilot system.


  1. In the VOXL 2 terminal, generate the necessary config files with this command:

voxl-configure-open-vins tracking_frd

  1. Disable the qvio server to prevent conflicts with multiple VIO sources:

voxl-configure-qvio disable

  1. Navigate to the configuration directory

cd /etc/modalai/

  1. If running VOXL SDK 1.4.open voxl-open-vins-server.conf and set:


"en_ext_feature_tracker": false

  1. Save the file and restart the service:

systemctl restart voxl-open-vins-server

Configuring the Extrinsics

The VOXL SDK comes with a great utility to help you configure your extrinsics. The voxl-configure-extrinsics command allows you to load from either a preset or a user file. Normally, you would go with the voxl-configure-extrinsics factory_starling_v1 command with the Starling 1 we are utilizing, but because we are setting up three cameras, we will need to customize the file. This will be dependent on your setup, so please reference the official ModalAI guide on configuring extrinsics to understand how to construct or modify existing templates to fit your project. We can’t cover every possible setup here, so instead, we’ll give you a light overview of the extrinsics structure.

"extrinsics": [{
                        "parent": "imu_apps",
                        "child":  "tracking_front",
                        "T_child_wrt_parent": [0.037, 0.000, 0.0006],
                        "RPY_parent_to_child":	[0, 90, 90]
                }, {
                        "parent": "imu_apps",
                        "child":  "tracking_down",
                        "T_child_wrt_parent": [-0.08825, -0.0045, 0.00269],
                        "RPY_parent_to_child":	[180, 0, 0]
                }, {
                        "parent": "imu_apps",
                        "child":  "tracking_rear",
                        "T_child_wrt_parent": [-0.092, 0.016, 0.0036],
                        "RPY_parent_to_child":	[0, -90, -90]
                }]

Okay, and let’s break each of its component parts down:

  • "parent": The reference frame from which the transformation is measured.

  • "child": The sensor or frame whose position and orientation are being defined relative to the parent.

  • "T_child_wrt_parent": The translation vector (in meters) specifying the child’s position offset from the parent.

  • "RPY_parent_to_child": The rotation angles (roll, pitch, yaw in degrees) describing the child’s orientation relative to the parent.

It’s not that complex once you understand how each of the parameters relate to each other. Also, you can see now, what you need to change if you are experiencing major issues during testing. Once your extrinsics are setup with each parent and child laid out in series with their translation and rotation relationships then issue this command, replacing the xx/yy/my_custom_file.conf with your own file’s destination and name:

voxl-configure-extrinsics custom /xx/yy/my_custom_file.conf

Configuring the VIO Camera File

The /etc/modalai/vio_cams.conf file determines which camera feeds will be used for tracking. First, we want to set ”enable: true. Then we will also need to rename our down camera to up for testing indoors.

"name":                 "tracking_up",
"pipe_for_preview":     "tracking_up",
"pipe_for_tracking":    "tracking_up",

What’s Next?

You are now ready for testing. Follow the testing section of the official VOXL Open-VINS Server documentation. It is recommended you start with hand testing and progress until you are ready to actually test in the field. One of the most important things we suggest paying attention to is the en_vio_always_on parameter. This should be set to true during hand testing and false during flight testing. This parameter determines how small vibrations are interpreted and can cause a fly away if set incorrectly.


Conclusion

Congratulations! You've now mastered the art of setting up triple camera tracking using the VOXL Open-VINS server on your drone platform. By following this tutorial, you've learned how to configure multiple cameras, understand and customize extrinsics, and set up the necessary configuration files. This empowers your drone to maintain reliable tracking even in challenging environments and poor lighting conditions. With this new level of spatial awareness, your platforms will be able to take on new complex navigation tasks with confidence and autonomy.

Comments

bottom of page