Skip to content

Amazon webservices EC2

You can run CVEDIA-RT using Amazon EC2 instances.

EC2 instances are ideal for larger workloads using GPUs.

For testing, we recommend using ECS / EKS instead.

CVEDIA-RT can run on any NVIDIA GPU with Compute Capability greater or equal to 5.2.

Requirements

  • 16G disk space
  • Docker 20.04 or newer
  • NVIDIA Docker 2
  • CUDA 10.2+
  • NVIDIA GPU with Compute Capability greater or equal to 5.2

Deployment on EC2

  1. Create a new EC2 instance with ubuntu 18+ as base image
  2. Install docker and nvidia-docker (when using GPU)
  3. Download CVEDIA-RT linux redist package
  4. Call ./run.sh --remote
  5. CVEDIA-RT will be accessible via REST API on port 80

NVIDIA docker base images

Depending on availability EC2 may have a pre-configured nvidia-docker base AMI image.

Installing NVIDIA docker

Check the official install guide from NVIDIA.

Running options

There's several ways to run CVEDIA-RT, please refeer to Running Options for all available options.

Caching assets

All models will be downloaded at models/ within the folder you extracted CVEDIA-RT redist. Adding this folder to persistent storage will avoid models from being redownloaded everytime you restart a instance.

Note: Models are encrypted and signed for the specific platform they're running, if the VM you're running is ephemeral cached models will not work.

Solutions and persistence

On linux CVEDIA-RT runs within docker, by default changes on solutions are ephemeral, if you want to make your solution changes persist, please check solution persistence.

Notes

CVEDIA-RT supports any NVIDIA GPU instance, you should use instances with a single GPU attached. Models for the specific platform you're using will be downloaded on fly.

AMD GPU acceleration is not supported, CVEDIA-RT will fallback to CPU inference.

CVEDIA-RT container automatically provide health metrics back to the cluster, in the top of that you can use the API /status to query for the instance metrics, allowing for easy scalability.