Amazon webservices ECS / EKS¶
You can run CVEDIA-RT using Amazon Elastic Container Service (ECS) or Amazon Elastic Kubernetes Service (EKS).
For scalability and price we recommend using a headless ECS cluster with GPU accelerated instances, this depends on availability.
For testing, you can use a simple CPU instance.
Deployment on ECS / EKS¶
- Create a Elastic Container Registry (ECR)
- Copy / Register CVEDIA-RT docker image into your ECR
- Create a ECS or EKS Cluster
- Create a new service deployment, using the ECR image
- Expose port
All models will be downloaded at
models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.
Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.
CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.
AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.
CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API
/status to query for the instance metrics, allowing for easy scalability.