NEPI Engine – AI Application

Introduction

This tutorial covers NEPI Engine’s built-in AI application which supports loading AI models from NEPI’s on-board AI model library and connecting to available image streams.

What you will need

1) 1x NEPI-enabled device with internet access. This tutorial uses an edge-compute processor box that includes an NVIDIA Jetson Xavier NX embedded GPU module with NEPI Engine software installed.

NOTE: See available off-the-shelf NEPI enabled edge-compute options at: https://numurus.com/products-nepi-enabled-hardware/

2) 1x PC with internet access and configured to access the NEPI device’s RUI browser-based interface and user storage drive. This tutorial uses a Windows 11 PC and a USB GigE Ethernet adapter and Ethernet cable.

NOTE: Instructions for configuration a PC and connecting to a NEPI device’s RUI and user storage drive are provided in the NEPI Engine Getting Started tutorial at: https://nepi.com/nepi-tutorials/nepi-engine-connecting-and-setup/

3) 1x NEPI IDX supported 2D camera. This tutorial uses a USB webcam camera.

NOTE: See the NEPI Engine Hardware Interfacing tutorial “Imaging Sensors” for details on connecting a camera to your NEPI device at: https://nepi.com/tutorials/

Hardware Setup

1) Connect the NEPI IDX driver supported camera to your NEPI device.

2) Connect the NEPI device to your PC’s Ethernet adapter using an Ethernet cable, then power your NEPI device.

Loading and Running an AI Model

In this section we will show how to load an AI models from NEPI’s on-board AI model library, and connect it to an image stream being produced by a connected camera using NEPI’s built-in AI application.

NOTE: This AI application tutorial assumes you have at least one AI model loaded into your NEPI device’s on-board AI model library located on the NEPI device’s user storage drive in a folder called “ai_models”. To download the example AI models used in this tutorial, visit: https://www.dropbox.com/scl/fo/iep3i08frjlnthesyqeez/h?rlkey=lzrkvoheqglgtdezggvsra0vo&dl=0

For more information on deploying AI models onto your NEPI device see the NEPI AI Management System manual available at: https://nepi.com/documentation/nepi-engine-ai-management-system/

and NEPI Engine Custom AI Model Development tutorial available at: https://nepi.com/tutorials/

1) On your PC, open your NEPI device’s RUI and select the APPLICATIONS/AI tab from the top menu to open the NEPI AI application.

2) On the AI application page, there is an image view pane on the top left. On the top right of the page, there are dropdown menus for choosing the image source you want to feed into your AI model and the AI model you want to load from the on-board AI model library, selector dropdown boxes. Below the AI model selector dropdown is a slider that adjusts the detection threshold at which AI model detections are used based on the confidence value of each detection.  After setting these three parameters, press the “Start” button to load the selected model and connect it to the selected image topic. 

3) Once your AI model loads and connects to the selected image stream, you will see the detection data overlaid on the input image in the image viewing pane. 

NOTE: In addition to this detection image output stream, the AI application produces a ROS bonding box message with all the detection details for downstream applications such as automation scripts or controls systems to use the AI detection results.