NEPI Engine – ZED Stereo Camera Systems
Introduction
This guide covers interfacing a StereoLab ZED Stereo Camera to a NEPI software enabled edge-compute hardware system using NEPI’s IDX driver system.
What you will need:
1) 1x NEPI-enabled device. This tutorial uses an edge-compute processor box that includes an NVIDIA Jetson Xavier NX embedded GPU module with NEPI Engine software installed. (Optional) If you are connecting to your NEPI device through a NEPI configured access point, your NEPI device will also need Wi-Fi support.
NOTE: See available off-the-shelf NEPI enabled edge-compute options at:
https://numurus.com/products-nepi-enabled-hardware/
2. 1x NEPI-enabled device connected to the internet and your PC. This can be done either with a network switch connected to the internet, the PC, and the NEPI device, or a PC connected directly to the NEPI device through and Ethernet cable, and the NEPI device connected to the internet through WiFi.
3. 1x StereoLabs Stereo Camera
Configuring the ZED Camera Driver
1) Open the NEPI device’s RUI SYSTEM/DRIVERS tab, and select the “IDX_ZED” camera driver from the dropdown menu. Make sure the driver is enabled and select the camera resolution from the Resolution dropdown menu.
NOTE: After making any changes to your Zed camera driver configuration you need to reboot your system.
Installing Zed Calibration File
While NEPI’s IDX driver system can autodetect StereoLab ZED stereo cameras, the driver requires that a valid calibration file for your camera in order to connect. See instructions below for installing a Zed calibration files.
1) NEPI will automatically download the calibration file from the StereoLab website when the camera is connected and your NEPI device is connected to the internet through an Ethernet connection. Before plugging in your ZED camera, connect your NEPI device to the internet open your NEPI device’s RUI SYSTEM/ADMIN tab and either connect your NEPI device to an Ethernet enabled switch and enable DHCP, or connect to an internet supported WiFi network. Wait for the “Internet Connected” icon at the top of the page to turn Green, then connect your ZED camera.
NOTE: Detailed instructions for connecting NEPI devices to the internet are available at:
https://nepi.com/nepi-tutorials/nepi-engine-connecting-and-setup/
NOTE: If NEPI’s is unable to automatically download the camera’s config file, you can install the zed calibration files using the zed tools installed on your system:
a) reboot your NEPI device with the ZED camera unplugged and make sure your NEPI device is connected to the internet from the NEPI RUI System/Admin tab.
b) from an ssh connected terminal, stop the NEPI ROS env by typing:
rosstop
c) type the following to try and download the config file:
cd /usr/local/zed/tools
./ZED_Explorer --dc
or manually download your camera’s calibration file from the StereoLab website, or create your own custom calibration file, and copy it to your NEPI device’s file system folder at
/usr/local/zed/settings/
d) Plug your ZED camera into your NEPI device and start the NEPI ROS environment back up by typing:
rosstart
2) Verify that your camera’s calibration file has been successfully installed by checking for a “.conf” file in the “user_cfg/zed_cals” folder on your NEPI device’s user storage drive. You can find instructions for accessing this drive at: https://nepi.com/nepi-tutorials/nepi-engine-user-storage-drive/
Camera Configuration
NEPI’s IDX driver interface provides capabilities for configuring your camera settings, accessing camera data streams, and saving camera data to the NEPI devices on-board user storage driver.
NEPI RUI Configuration
NEPI’s RUI DEVICES/IMAGING tab provides a convenient method for testing, configuring, and saving data from your Zed camera. For more information on using NEPI’s Imaging application see the following tutorial:
https://nepi.com/nepi-tutorials/nepi-engine-imaging-sensors/
API Configuration
All camera control and data interfaces available in the NEPI RUI DEVICES/IMAGING tab have ROS and other API backend interfaces for programmatic interfacing with your Zed Camera. For more information on API interfacing with your camera, see the “IDX Driver Interface” section in this NEPI Hardware Interfacing document:
https://nepi.com/documentation/nepi-engine-hardware-driver-interface-system/
You can view a list of the Zed Camera’s control and data API interfaces through an SSH connected terminal to your NEPI device and typing:
rostopic list | grep ZED_NAME_FROM_RUI
EXAMPLE:
rostopic list | grep zed2_34
Viewing Zed Data
NEPI RUI
RUI Image Viewer App
Open the NEPI device’s RUI DATA/IMAGE_VIEWER tab, select one or multiple camera streams to view simultaneously.
RUI NavPose Manager App
Open the NEPI device’s RUI NAVPOSE/NAVPOSE_MGR tab, select the Zed camera orientation stream you want connected to the NEPI NavPose solution and view Zed oreantation data in real time.
ROS RVIZ App
You can view the ZED cameras images, depth_map, and pointcloud data using the ROS rviz application on a NEPI ROS configured PC connected to your NEPI device following these instructions.
1) There are two options for running rviz,
a) Connect a display, keyboard, and mouse to your NEPI device and run rviz from a terminal on the NEPI device’s desktop.
b) Run rviz on a network connected PC with ROS installed and configured to communicate with the NEPI ROS environment. Follow the instructions in the tutorial link below to setup your Linux PC for NEPI ROS interfacing:
https://nepi.com/nepi-tutorials/nepi-engine-configuring-a-pc-for-nepi-ros-env/
2) Open a terminal on your Linux PC and type:
rviz
This will open the rviz ROS application.
3) Select the “Add ” button on the bottom of the left menu, select the “PointCloud2” item from the popup list. This will put a new PointCloud2 Display item in the left menu bar. Maximize the PointCloud2 Display item and click on the “Topic” selection line, then select the pointcloud you want to visualize from the dropdown list.
Automation Scripting
You can quickly create custom automation scripts that run within NEPI’s Automation Manager system that interface with Zed camera and other hardware to take actions such as robotic obstacle avoidance or target following. See the following tutorial to learn more about creating NEPI automation scripts:
https://nepi.com/nepi-tutorials/nepi-engine-automation-application/
