Skip to content

Azure Container Instances (ACI)

You can run CVEDIA-RT using Azure ACI.

ACIs will work for both GPU and non GPU workloads. Note that CVEDIA-RT only supports NVIDIA GPUs.

Minimum Requirements

  • 1 CPU core
  • 512M Memory

Deployment on ACI

  1. Create a new Azure Container instance
  2. Select Image Source -> Other Registry, get CVEDIA-RT docker image tag from docker hub and paste.
  3. OS Type should be Linux
  4. Size, if you have availability we recommend running with one GPU enabled (any NVIDIA GPU will work). For testing the minimum requirements above apply.
  5. In Networking tab, you should expose port 80 (this is the default)
  6. In Advanced tab, you should add a new environment variable key: RUN_UI value: 0
  7. Click Review+Create
  8. Confirm by clicking Create

Accessing

  1. Go to the resource group where you created the container
  2. Open the container instance
  3. You can use the public ip provided to configure CVEDIA-RT using REST API.

Caching assets

All models will be downloaded at models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded every time you restart an instance.

Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.

Notes

CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on the fly.

AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.

CVEDIA-RT container automatically provide health metrics back to the cluster, on top of that you can use the API /status to query for the instance metrics, allowing for easy scalability.