Nuvo-9166GC + NPL54 with MobileSAM

Purposes

Autonomous logistics applications require higher safety and precise robotic control, enabled by AI-driven object detection and segmentation. This demo showcases a compact, rugged Edge AI computer with GMSL2 camera support and NVIDIA L4 inference capabilities.

This demo covers:

• Nuvo-9166GC: Intel 14th Gen rugged Edge AI computer supporting NVIDIA L4 and one add-on PCIe card

• PCIe-NPL54: Cost-effective 4-port GMSL2 frame grabber for select 8M/5M/3M/2M GMSL2 cameras

• AC-IMX390: IP67+IP69K SONY IMX390 GMSL2 automotive camera

Run Demo

• This demo should run automatically after reboot

Stop Demo

• Before shutdown the OS, please kindly press Ctrl+C on in the NPL54 Capture window to ensure the VDMA is stopped.

---

Preparation

• Demo Code GitHub Repo [link]

However, the demo code relies on many environmental settings ...

• Control the Nuvo-9166GC fan speed [link]

• Install the NVIDIA GPU driver [reference]

• PCIe-NPL54 work getting started guide [link]

• Install MobileSAM

cd ~/Desktop
git clone https://github.com/ChaoningZhang/MobileSAM.git

• Other Python packages

# Python OpenCV with GUI Support

sudo pip uninstall opencv-python opencv-python-headless -y
sudo pip uninstall opencv-python opencv-python
pip install opencv-python --no-cache-dir

# YOLOv8 Installation
sudo apt install -y libgtk2.0-dev pkg-config
sudo apt install -y qt6-wayland

pip install ultralytics opencv-python pillow torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

---

Recompile OpenCV with CUDA support

This section does not appear to be a mandatory part of the current demo.

During development, we tried to recompile OpenCV to enable CUDA support. The compilation process is quite complex and depends on the Ubuntu version and NVIDIA GPU architecture.

Install CUDA and CUDNN first

Major reference: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#network-repo-installation-for-ubuntu

DISTRO=ubuntu2204
ARCH=x86_64

wget https://developer.download.nvidia.com/compute/cuda/repos/$DISTRO/$ARCH/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb

sudo apt-get update
sudo apt-get install -y cuda-toolkit
sudo apt-get install -y cudnn9-cuda-12

Install gcc-10 & g++-10

sudo apt-get install gcc-10 g++-10

Remove Existing OpenCV

## Make sure the headless openCV is uninstalled
pip show opencv-python
pip uninstall -y opencv-python opencv-python-headless opencv-contrib-python

Download OpenCV Repo and Compile

Check the following link to ensure th CUDA_ARCH for the GPU card you're using https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/

CUDA_ARCH_BIN=8.0 = Ampere Architecture
CUDA_ARCH_BIN=8.9 = Ada Lovelace Architecutre 

cd ~/Desktop
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git

cd ~/Desktop/opencv
mkdir build
cd build

export CC=/usr/bin/gcc-10; export CXX=/usr/bin/g++-10

cmake \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_CUDA=ON \
-D WITH_CUDNN=ON \
-D WITH_CUBLAS=ON \
-D WITH_TBB=ON \
-D OPENCV_DNN_CUDA=ON \
-D OPENCV_ENABLE_NONFREE=ON \
-D BUILD_OPENCV_WORLD=OFF \
-D CUDA_ARCH_BIN=8.0 \
-D OPENCV_EXTRA_MODULES_PATH=/home/$USER/Desktop/opencv_contrib/modules/ /home/$USER/Desktop/opencv/ \
-D BUILD_EXAMPLES=OFF \
-D HAVE_opencv_python3=ON \

make 
sudo make install