Skip to content

Google Cloud Virtual Machine Instances

You can run CVEDIA-RT headless or with a GUI using Google Cloud Virtual Machine Instances

  • When running headless you will need to use our REST API to interact with it.

  • When running with GUI you will need to open a VNC or RDC connection to it.

CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.

AMD GPUS

AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.

VM Setup process

Before setting up CVEDIA-RT, we need to create a new VM instance.

  1. Open Google Cloud VM Instances
  2. Click Create Instance
  3. Click Machine Family -> GPU
  4. GPU Type can be anything but NVIDIA K series, we recommend as a single NVIDIA T4. Running multiple GPUs in a single instance is not recommended, as it's more efficient to run multiple VMs with single GPUs.
  5. In Machine type, we recommend n1-standard-4 (4x vCPUs and 15G RAM) as minimum, depending on your workload you may need to request more vCPUs. A smaller instance won't be able to feed the GPU fast enough.
  6. In Boot disk Click on Switch Image then set:
    • Operating System to Ubuntu
    • Version to Ubuntu 20.04 LTS AMD64 (not to be confused with ARM64)
    • Disk Size to 100GB
    • Click on Select to save
  7. Click Create to schedule the vm

VM Setup

The setup above is a suggestion that works for our evaluation demo, depending on your workload it would be more efficient to setup more or less resources. For scalability we recommend Anthos or Cloud Run.

After the machine boots up, we need to setup docker, nvidia drivers and other dependencies. CVEDIA provides a script that does it all automatically.

Login via SSH as root then run:

curl -sk https://bin.cvedia.com/gcs_vm_install.sh | bash -

This script will automatically download, patch and install NVIDIA drivers, docker and NVIDIA docker runtime. It can also install X11 and configure a fully working desktop, if you want to run CVEDIA-RT in GUI natively (to do that, run declare -x DESKTOP=1 before calling the aforementioned install).

Finally, the current CVEDIA-RT redist package will also be downloaded and installed.

Running with GUI

Run ./run.sh to start.

Depending on the devices and platform you're running you may be required to run as sudo or start the docker with privileged mode, this is automatically decided by the startup script.

Running Headless / REST

Run ./run.sh --remote to start.

CVEDIA-RT will be accessible via REST API on port 8080

Remote UI

As of 2022.11.0 release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.

Please check Remote UI for more info.

Running options

There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.

Solutions and persistence

When running CVEDIA-RT within docker, changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.

Caching assets

All models will be downloaded at models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.

Model encryption

Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.

Notes

CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API /status to query for the instance metrics, allowing for easy scalability.