Skip to content

Amazon webservices ECS / EKS

You can run CVEDIA-RT using Amazon Elastic Container Service (ECS) or Amazon Elastic Kubernetes Service (EKS).

For scalability and price we recommend using a headless ECS cluster with GPU accelerated instances, this depends on availability.

For testing, you can use a simple CPU instance.

Deployment on ECS / EKS

  1. Create a Elastic Container Registry (ECR)
  2. Copy / Register CVEDIA-RT docker image into your ECR
  3. Create a ECS or EKS Cluster
  4. Create a new service deployment, using the ECR image
  5. Expose port 80

Caching assets

All models will be downloaded at models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.

Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.

Notes

CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.

AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.

CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API /status to query for the instance metrics, allowing for easy scalability.