Azure Container Instances (ACI)¶
You can run CVEDIA-RT using Azure ACI.
ACIs will work for both GPU and non GPU workloads. Note that CVEDIA-RT only supports NVIDIA GPUs.
- 1 CPU core
- 512M Memory
Deployment on ACI¶
- Create a new
Azure Container instance
Other Registry, get CVEDIA-RT docker image tag from docker hub and paste.
OS Typeshould be
Size, if you have availability we recommend running with one GPU enabled (any NVIDIA GPU will work). For testing the minimum requirements above apply.
Networkingtab, you should expose port
80(this is the default)
Advancedtab, you should add a new environment variable key:
- Confirm by clicking
- Go to the resource group where you created the container
- Open the container instance
- You can use the public ip provided to configure CVEDIA-RT using REST API.
All models will be downloaded at
models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.
Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.
CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.
AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.
CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API
/status to query for the instance metrics, allowing for easy scalability.