Modelforge¶

Modelforge is a CVEDIA proprietary API to make AI models available on any platform without any extra heavy lifting by the user of CVEDIA-RT. Simply choose the desired AI model to load, and Modelforge will make it available for your platforms.

Using `listnndevices` to find available devices¶

CVEDIA-RT package comes with a standalone application called listnndevices. When running this application you will get a list of all the available devices on the machine.

listnndevices

Using `modelforge` to query available models for each device¶

CVEDIA-RT package comes with a standalone application called modelforge. When running this application you will get a list of all the models that can be automatically downloaded for the machine's device.

The query must be made specifying what device to use, based on the available devices listed on the listnndevices application.

Usage: modelforge.exe --uri [DEVICE]

Example: modelforge.exe --uri tensorrt.1://

modelforge

Seeing verbose output

If the -v flag is used when running the modelforge command, a verbose debug output will be presented

Using the model on the application¶

After choosing the model from the list returned by the Modelforge application, you can set it on the project settings JSON file in the corresponding inference section:

Using auto: on the device section of the json

If the user specifies auto: as the device for the model to load, CVEDIA-RT will automatically choose the better device for that machine to run the model.

modelfile_json

When the instance starts, CVEDIA-RT will automatically reach the Model Forge servers and download the model corresponding to the configured settings, which will then use for inference.

Using a local model instead of CVEDIA's models¶

You can use your own models instead of ones delivered by our system.

Example: tensorrt.1:///path/to/my/model.engine

Note 3x /. This will instruct CVEDIA-RT to load the file from a local path.

A model should have a .json file with its configuration parameters:

{
    "channel_layout": "",
    "comment": "Model name",
    "mean": 128,
    "normalize_input": false,
    "postprocess": "passthru",
    "preprocess": "default",
    "softmax": false,
    "stddev": 57,
    "transpose_input": false
}

You can find more on how to configure it in inference