NVIDIA Jetson Installation¶
Jetpack¶
Recommended Jetpack release for Nano: JetPack 4.6.0 Recommended Jetpack release for AGX and Orin: JetPack 5.1
Warnings¶
- NVIDIA Docker 2 is installed by default on JetPack
Supported NVIDIA devices¶
- Jetson Nano (CUDA compute 5.3)
- Jetson AGX NX (CUDA compute 7.2)
- Jetson Xavier AGX (CUDA compute 7.2)
- Jetson Orin Nano (CUDA compute 8.7)
- Jetson Orin NX (CUDA compute 8.7)
- Jetson Orin AGX (CUDA compute 8.7)
Supported TensorRT versions¶
- 7.1.3
- 8.0.1
- 8.2.1
- 8.4.0
- 8.4.1
- 8.5.2
Note: TensorRT 7.x dosen't support all CVEDIA-RT's model catalog. When this happens it will fallback to CPU inference.
Requirements¶
CVEDIA-RT will work with any Linux distro that can run dockers or with glibc 2.27 and newer.
CVEDIA-RT works on x86_64
, aarch64
, armv7a
and armhf
ISAs.
Docker install¶
- 16G disk space
- Docker 20.04 or newer
To run on GPU using NVIDIA, you will also need:
- CUDA 10.2+
- NVIDIA GPU with Compute Capability greater or equal to 5.2
Check Compute Capability to make sure your gpu is compatible.
If you don't have docker installed
Check install docker guide.
If you don't have NVIDIA Docker installed
Check the official install guide from NVIDIA.
Native install¶
- Debian-like distro with glibc >= 2.27 (run
ldd --version
to check) - ~8G disk space
- dpkg or similar package manager
To run on GPU using NVIDIA, you will also need:
- CUDA toolkit
Check Compute Capability to make sure your gpu is compatible.
Installation¶
Docker¶
By default CVEDIA-RT will run a GUI, that may require some X11 changes, the install.sh
script will check if your current settings are compatible and perform changes as needed.
You may need to restart your X11 session after changes are performed.
- Extract the
.tar.gz
file you downloaded:tar -xzvf <filename.tar.gz>
- Run install script:
sudo ./install.sh
Running with GUI¶
Run ./run.sh
to start.
Depending on the devices and platform you're running you may be required to run as sudo
or start the docker with privileged
mode, this is automatically decided by the startup script.
Running Headless / REST¶
Run ./run.sh --headless
to start.
CVEDIA-RT will be accessible via REST API on port 8080
Remote UI¶
As of 2022.11.0
release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.
Please check Remote UI for more info.
Running options
There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.
Solutions and persistence¶
When running CVEDIA-RT within docker, changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.
Native¶
CVEDIA-RT can run natively on linux. This deployment is experimental and may require you to install additional packages manually.
The main difference between this and the docker version is that the native will require additional supporting services (such as mqtt) and libraries (such as cuda, gstreamer, hailo, opencv) to be pre installed for it to work properly. This may also means if you don't have inference libraries installed or compatible with CVEDIA-RT, it will fallback to slow CPU inference.
This native install comes bundled with several static libraries that will not affect anything you may have installed locally.
Hard paths
The native install will be placed at a fixed path: /opt/cvedia-rt which will potentially overwrite any files you may have there.
To install:
- Install the deb package you downloaded:
dpkg -i <filename.deb>
- Install any potential missing dependencies
apt install -f
CVEDIA-RT deb package will add a few binaries to your path (/usr/bin
):
cvediart
listnndevices
benchmark
:modelforge
runinference
runinstance
Upgrading
CVEDIA don't yet provide a public package repository, however you can simply run a dpkg -i <filename>.deb
to upgrade any existing packages
Uninstalling
You can uninstall using the package manager apt remove cvedia-rt
Configuring
The native version port and services configuration relay in /opt/cvedia-rt/global.config
Running with GUI¶
Just call cvediart
, a window will show up.
Running Headless / REST¶
Run cvediart --headless
to start.
CVEDIA-RT will be accessible via REST API on port 8080
Remote UI¶
As of 2022.11.0
release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.
Please check Remote UI for more info.
Running options
There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.
Troubleshooting¶
If you're running in GUI mode and CVEDIA-RT crashes with FATAL [..] [glfwui.cpp:68] glfwCreateWindow ..
, you can try:
- On windows:
CVEDIA-RT.bat --dx11
- On windows:
CVEDIA-RT.bat --glfw
- On windows:
CVEDIA-RT.bat --sdl
- On linux with docker:
./run.sh --gl 0
- On linux native:
cvediart --sdl
- On linux native:
cvediart --glfw
If none of this options work, you can still run without UI then connect from a different machine using Remote UI:
- On windows:
CVEDIA-RT.bat --remote
- On linux with docker:
./run.sh --remote
- On linux native:
cvediart --remote
On linux CVEDIA-RT might fail to start if your desktop resolution is smaller than 1280x720
.
If you don't have a graphical interface or cannot run in this resolution, you can run headless (./run.sh --headless
on docker, or cvediart --headless
natively) or enable Remote UI (./run.sh --remote
on docker, or cvediart --remote
natively)
If you're having issues after the application has started, logs are automatically saved under the log
folder. Each session of CVEDIA-RT will create a new log entry in that folder, to submit support requests get the most recent log from that folder.
On linux for startup errors within docker, we recommend running test.sh
, this will create a detailed log of system capabilities which can then be used to submit support requests to our team.
For any other questions and inquiries, check the support page.
How to run your first project : Quick Start