Skip to content

Hailo Installation

CVEDIA-RT supports Hailo H8, Hailo H15 and all it's variants in either M.2, PCI-E and standalone versions.

Limitations

Some models are not available in this platform, when this happens CVEDIA-RT will automatically fallback to the next best backend.

Supported Hailo versions

HailoRT DFC CVEDIA-RT Version
4.13.0 3.23.0 2023.4.x
4.14.x 3.24.0 2023.5.x
4.16.0 3.26.0 2024.1.x
4.17.0 3.27.0 2024.2.0
4.17.0 3.27.0 2024.2.1
4.16.0 3.26.0 2024.2.2
4.16.0 3.26.0 2024.2.3

HailoRT Versions

CVEDIA-RT 2024.2.3 reverts back to an old version of HailoRT due stability issues

If you don't know what driver version you're running on windows...

Open Device Manager -> System Devices -> Hailo XX AI -> Properties, you should see the driver version in the General tab: Hailo Version

If you don't know what version you're running on linux...

Run ldconfig -v | grep hailo to find what hailort version you've currently loaded.

If you need to install a different driver version on linux...

  • Use apt-get to uninstall hailort
  • Reboot
  • Check if the hailo drivers are gone using ldconfig -vp | grep hailo, if you still see them delete the related files
  • Install the proper package
  • Reboot
  • Check if hailo drivers are loaded and have the correct version with ldconfig -vp | grep hailo

Supported Hailo Hardware arch

  • HailoH8
  • HailoH8L
  • HailoH8R
  • Hailo15M (HailoRT 4.16+)
  • Hailo15H (HailoRT 4.16+)

It's very important to assure that hailo is running at PCI-E 4x link width. There's a significant loss of performance if you run at 2x or 1x.

To make sure you're running at maximum speed:

On Windows

You can check on Device Manager -> System devices -> Hailo XX Accelerator -> Details -> PCI current link width

PCI-E Windows

You should see 0000004 , meaning 4x.

On Linux

Find PCI address of hailo:

# lspci | grep -i co-proc
0000:01:00.0 Co-processor: Device 1e60:2864 (rev 01)

Query For speed:

# lspci -vv -s 0000:01:00.0  | grep -i width
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <2us
                LnkSta: Speed 8GT/s (ok), Width x4

In this case LnkSta is the link stablished with the device with Speed 8GT/s and Width x4 meaning it's working at full speed. If it says downgraded it means it's working at slower than LnkCap (link capabilities).

Benchmarking Hailo

Besides CVEDIA-RT's own benchmarking tools, hailo has a special tool for monitoring the device within their hailortcli bundle.

You can monitor the device by setting HAILO_MONITOR=1 env variable, running RT then opening the monitor using haiortcli monitor.

For example on linux native installs, we can run CVEDIA-RT's benchmark tool in the background then run hailortcli monitor to see device utilization:

export HAILO_MONITOR=1
rtcmd inference benchmark -u hailo.auto://pva_det/rgb/medium_y6_mosaic_rot90_320x320/240326 -n 10000 -i 1 -t 16 -p 16 &> /dev/null &
hailortcli monitor

Will show:

HailoRTCLI Monitor

How to monitor when running RT inside docker

  • Call ./run.sh with -- -e HAILO_MONITOR=1 to add the ENV variable globally to the docker container
  • Shell into the container, eg: docker exec -it cvedia-rt /bin/bash (the container name might be different)
  • Install the same version of the release HailoRT within the container
  • Run hailortcli monitor inside the container while RT is using the hardware

Requirements

CVEDIA-RT will work with any Linux distro that can run dockers or with minimum libc6 2.27 on aarch64 and libc6 2.29 on x86 (2.31 for arm and 2.34 for x86 are strongly recommended) natively with deb packages.

CVEDIA-RT works on x86_64, aarch64, armv7a and armhf ISAs.

CVEDIA-RT been tested on Ubuntu 18, 20, 22 and 24 other platforms and systems are not supported / tested.

Ubuntu 24

Ubuntu 24 has issues when running rtstudio natively

Docker install

  • 16G disk space
  • Docker 18.04 or newer

To run on GPU using NVIDIA, you will also need:

  • CUDA 10.2+
  • NVIDIA GPU with Compute Capability greater or equal to 5.3 on aarch64 or 6.1 on x86

Check Compute Capability to make sure your gpu is compatible.

If you don't have docker installed

Check install docker guide.

If you don't have NVIDIA Docker installed

Check the official install guide from NVIDIA.

Native install

  • Any Debian-like distro, Ubuntu 20 or newer
  • ~8G disk space
  • dpkg or similar package manager

To run on GPU using NVIDIA, you will also need:

  • CUDA toolkit

Check Compute Capability to make sure your gpu is compatible.

Installation

Docker

By default CVEDIA-RT will run a GUI, that may require some X11 changes, the install.sh script will check if your current settings are compatible and perform changes as needed.

You may need to restart your X11 session after changes are performed.

  1. Extract the .tar.gz file you downloaded: tar -xzvf <filename.tar.gz>
  2. Run install script: sudo ./install.sh

Running with GUI

Run ./run.sh to start.

Depending on the devices and platform you're running you may be required to run as sudo or start the docker with privileged mode, this is automatically decided by the startup script.

Running Headless / REST

Run ./run.sh --remote to start.

CVEDIA-RT will be accessible via REST API on port 8080

Remote UI

As of 2022.11.0 release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.

Please check Remote UI for more info.

Running options

There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.

Solutions and persistence

When running CVEDIA-RT within docker, changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.

Native

As of 2024.1.0 releases CVEDIA-RT is now shipped thru our own apt repository. This allows you to update automatically or pin a specific version of RT. This works on all systems we support, including jetson and aarch64 distros.

Hard paths

The native install will be placed at a fixed path: /opt/cvedia-rt which will potentially overwrite any files you may have there.

Option 1: Installing using convenience script

curl -fsSLo - http://get.cvedia.com | sudo bash
apt install cvedia-rt -y

NX VMS

If you also want to install our NX VMS plugin, after installing mediaserver, run apt install cvedia-rt-nxplugin -y

Option 2: Installing by manually adding the apt repository

curl -fsSLo - http://apt.cvedia.com/key.gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/cvedia.gpg
sudo echo "deb http://apt.cvedia.com cvedia main" > /etc/apt/sources.list.d/cvedia.list
sudo apt update
sudo apt install cvedia-rt -y

LIBC6 older than 2.34

If you're running LIBC6 older than 2.34 you should use the legacy apt pool: sudo echo "deb http://apt.cvedia.com cvedia legacy" > /etc/apt/sources.list.d/cvedia.list

NX VMS

If you also want to install our NX VMS plugin, after installing mediaserver, run apt install cvedia-rt-nxplugin -y

Option 3: Installing using a debian package

Download the compatible .deb from https://rt.cvedia.com, then install it:

dpkg -i cvedia-rt_*.deb

NX VMS

If you also want to install our NX VMS plugin, after installing mediaserver, run dpkg -i cvedia-rt-nxplugin*.deb

CVEDIA-RT binaries

CVEDIA-RT deb package will add a few binaries to your path (/usr/bin):

  • rtcmd
  • rtstudio
  • rtservice

Uninstalling

You can uninstall using the package manager apt remove cvedia-rt

Running with GUI

Just call rtstudio, a window will show up.

Running Headless / REST

Run rtstudio --render Remote to start.

CVEDIA-RT will be accessible via REST API on port 8080

Remote UI

As of 2022.11.0 release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.

Please check Remote UI for more info.

Running options

There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.

Troubleshooting

If you're running in GUI mode and CVEDIA-RT crashes with FATAL [..] [glfwui.cpp:68] glfwCreateWindow .., you can try:

  • On windows: CVEDIA-RT.bat --render DX11
  • On windows: CVEDIA-RT.bat --render GLFW
  • On windows: CVEDIA-RT.bat --render SDL
  • On linux with docker: ./run.sh --gl 0
  • On linux native: rtstudio --render SDL
  • On linux native: rtstudio --render GLFW

If none of this options work, you can still run without UI then connect from a different machine using Remote UI:

  • On windows: CVEDIA-RT.bat --render Remote
  • On linux with docker: ./run.sh --remote
  • On linux native: rtstudio --render Remote

On linux CVEDIA-RT might fail to start if your desktop resolution is smaller than 1280x720.

If you don't have a graphical interface or cannot run in this resolution, you can run headless with Remote UI (./run.sh --remote on docker, or rtstudio --render Remote natively)

If you're having issues after the application has started, logs are automatically saved under the log folder. Each session of CVEDIA-RT will create a new log entry in that folder, to submit support requests get the most recent log from that folder.

On linux for startup errors within docker, we recommend running test.sh, this will create a detailed log of system capabilities which can then be used to submit support requests to our team.

For any other questions and inquiries, check the support page.