Amazon webservices EC2¶
You can run CVEDIA-RT using Amazon EC2 instances.
EC2 instances are ideal for larger workloads using GPUs.
For testing, we recommend using ECS / EKS instead.
CVEDIA-RT can run on any NVIDIA GPU with Compute Capability greater or equal to 5.2.
- 16G disk space
- Docker 20.04 or newer
- NVIDIA Docker 2
- CUDA 10.2+
- NVIDIA GPU with Compute Capability greater or equal to 5.2
Deployment on EC2¶
- Create a new EC2 instance with ubuntu 18+ as base image
- Install docker and nvidia-docker (when using GPU)
- Download CVEDIA-RT linux redist package
- CVEDIA-RT will be accessible via REST API on port
NVIDIA docker base images
Depending on availability EC2 may have a pre-configured nvidia-docker base AMI image.
Installing NVIDIA docker
Check the official install guide from NVIDIA.
There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.
All models will be downloaded at
models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.
Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.
Solutions and persistence¶
On linux CVEDIA-RT runs within docker, by default changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.
CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.
AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.
CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API
/status to query for the instance metrics, allowing for easy scalability.