Google Cloud Virtual Machine Instances¶
You can run CVEDIA-RT headless or with a GUI using Google Cloud Virtual Machine Instances
When running headless you will need to use our REST API to interact with it.
When running with GUI you will need to open a VNC or RDC connection to it.
CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.
AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.
VM Setup process¶
Before setting up CVEDIA-RT, we need to create a new VM instance.
- Open Google Cloud VM Instances
- Click Create Instance
- Click Machine Family -> GPU
- GPU Type can be anything but NVIDIA K series, we recommend as a single NVIDIA T4. Running multiple GPUs in a single instance is not recommended, as it's more efficient to run multiple VMs with single GPUs.
- In Machine type, we recommend n1-standard-4 (4x vCPUs and 15G RAM) as minimum, depending on your workload you may need to request more vCPUs. A smaller instance won't be able to feed the GPU fast enough.
- In Boot disk Click on Switch Image then set:
Ubuntu 20.04 LTSAMD64 (not to be confused with ARM64)
- Click on
Createto schedule the vm
The setup above is a suggestion that works for our evaluation demo, depending on your workload it would be more efficient to setup more or less resources. For scalability we recommend Anthos or Cloud Run.
After the machine boots up, we need to setup docker, nvidia drivers and other dependencies. CVEDIA provides a script that does it all automatically.
Login via SSH as
root then run:
curl -sk https://bin.cvedia.com/gcs_vm_install.sh | bash -
This script will automatically download, patch and install NVIDIA drivers, docker and NVIDIA docker runtime. It can also install X11 and configure a fully working desktop, if you want to run CVEDIA-RT in GUI natively (to do that, run
declare -x DESKTOP=1 before calling the aforementioned install).
Finally, the current CVEDIA-RT redist package will also be downloaded and installed.
Running with GUI¶
./run.sh to start.
Depending on the devices and platform you're running you may be required to run as
sudo or start the docker with
privileged mode, this is automatically decided by the startup script.
Running Headless / REST¶
./run.sh --headless to start.
CVEDIA-RT will be accessible via REST API on port
2022.11.0 release you can run CVEDIA-RT in remote GUI mode. To run in this mode you don't need to install a window manager nor x11.
Please check Remote UI for more info.
There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.
Solutions and persistence¶
When running CVEDIA-RT within docker, changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.
All models will be downloaded at
models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.
Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.
CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API
/status to query for the instance metrics, allowing for easy scalability.