NanoSAM on Orin NX w/ JetPack 5.x

This memo is intended for internal use to outline the steps for running the NanoSAM on PCIe-NX156U3.

NanoSAM

• https://segment-anything.com/

• https://www.jetson-ai-lab.com/vit/tutorial_nanosam.html

• https://github.com/NVIDIA-AI-IOT/nanosam

Prerequisites

• 1x USB camera supports at least 800 pixels width.

• 1x PCIe-NX156U3 or NRU-156U3-FT.

• At least 30GB of free space for Docker images.

---

Preparation (on PCIe-NX156U3)

### user name: nvidia

### Make sudo without password by 
sudo visudo
# In the final line, add
nvidia ALL=(ALL) NOPASSWD: ALL
# where 'nvidia' is your user name

### Prepare the jetson containers with conda
mkdir ~/Desktop/NanoSAM
cd ~/Desktop/NanoSAM

git clone https://github.com/dusty-nv/jetson-containers
cd jetson-containers

wget https://github.com/Archiconda/build-tools/releases/download/0.2.3/Archiconda3-0.2.3-Linux-aarch64.sh
sh Archiconda3-0.2.3-Linux-aarch64.sh
# yes 
source ~/.bashrc
conda create -n p3.8-nanosam python=3.8
conda activate p3.8-nanosam
sudo apt update; sudo apt install -y python3-pip

sudo ./run.sh $(./autotag nanosam)

### Inside Docker Container (Terminal #01)
sudo apt-get update
sudo apt install libcanberra-gtk-module libcanberra-gtk3-module vim
pip install pycocotools


### Outside Docker Container (Terminal #02)
# Add the missing model
# Download data/densenet121_baseline_att_256x256_B_epoch_160.pth from
# https://drive.google.com/open?id=13FkJkx7evQ1WwP54UmdiDXWyFMY1OxDU
cd ~/Downloads
containerID=$(sudo docker ps | grep 'nanosam' | awk '{ print $1 }')
sudo docker cp densenet121_baseline_att_256x256_B_epoch_160.pth $containerID:/tmp


### Inside Docker Container (Terminal #01)
cd /opt/nanosam
cp /tmp/densenet121_baseline_att_256x256_B_epoch_160.pth data/

# Edit the way to open camera to leverage Jetson's MJPEG decoder
vim /opt/nanosam/examples/demo_pose_tshirt.py
- cap = cv2.VideoCapture(0)
+ cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! image/jpeg, format=MJPG, width=800 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1", cv2.CAP_GSTREAMER)

# Make a script, /opt/run_demo.sh, to run the demo inside the docker
touch /opt/run_demo.sh
chmod +x /opt/run_demo.sh
vim /opt/run_demo.sh
# #!/bin/bash
# export LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0
# cd /opt/nanosam
# python3 examples/demo_pose_tshirt.py

    
### Outside Docker Container (Terminal #02)
# Preserve the docker status 
containerID=$(sudo docker ps | grep 'nanosam' | awk '{ print $1 }')
sudo docker commit -m "nanasam demo ready" $containerID demo/nanosam:v0.8

### If the demo is using GMSL2 camera, additional vi related paramters need to be passed into docker
# For example
# sudo docker run -it –rm –net=host –device /dev/video0:/dev/video0 –device /dev/media0:/dev/media0 –device /dev/capture-vi-channel0:/dev/capture-vi-channel0 –runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /lib/modules:/lib/modules demo/nanosam:v0.8
# 


### Inside Docker Container (Terminal #01)
exit

Demo Script (on PCIe-NX156U3)

# Assumed username = nvidia
touch /usr/src/run_demo.sh
chmod +x /usr/src/run_demo.sh
vim /usr/src/run_demo.sh

# #!/bin/bash
# cd /home/nvidia/Desktop/NanoSAM/jetson-containers
# source /home/nvidia/archiconda3/etc/profile.d/conda.sh
# conda activate p3.8-nanosam
# sudo ./run.sh $(./autotag nanosam) ./run_demo.sh

Misc

• Put the Demo, run_demo.sh, into Startup Applications with following steps, and put the following command inside the command prmopt. gnome-terminal -- /usr/src/run_demo.sh

• Disable the notifications if you have the host surface or over current notification is annoying. Also, put the Power mode into MAXN for best live demo performance.