Customizing Solutions ¶

Customizing a Solution can be accomplished by editing its Lua code.

All CVEDIA-RT solutions are broken down in different folders and .json files that are reflected in the instance list sidebar:

Placeholder

Example

If we want to change the Crowd Estimation solution, we will be able to find it at solutions/crowd-estimation/.

CVEDIA-RT on Dockers

When running CVEDIA-RT within docker (Linux deployments) solutions are embedded within the docker image. You can run extract_solutions.sh to expose all included solutions. This script will copy all solutions from the docker image to the solutions folder, which is automatically mounted when you start CVEDIA-RT using run.sh.

Solution ¶

Solution Configuration ¶

A solution configuration is set by the base_config.json file present in the solution root folder. This file sets the common properties for all the instances that use that solution. When an instance starts, it will inherit the solution configuration from the base_config.json file and will add or replace its own settings.

Accessing Solution Configuration ¶

As an example, we show how to define a my_var variable in the JSON and retrieve it from the code.

base_config.json or instance.json:

"Global": {
    "my_var": "my var content"
}

And inside the Lua code, we can retrieve it by using:

local lua_var = instance:getConfigValue("Global/my_var")

Solution Lua Code ¶

Every solution must have a solution.lua file. This file is responsible for creating the solution scope interfaces and modules. It also defines the options to expose in the Input Configuration panel and in the Export configuration.

A base template for this file is the following:

dofile(luaroot .. "/api/api.lua")

---@param solution Solution
function onStartup(solution)
    print("Starting solution")
    local detectInst = api.factory.inference.create(solution, "PVADetector")
    detectInst:loadModelFromConfig()
    solution:registerUiCallback("onInputSinkConfig", "onInputSinkConfig", inputSinkConfig)
end

---@param solution Solution
---@param instance rt_instance
function onStartInstance(solution, instance)
    print("Running instance " .. instance:getName())
end

---@param solution Solution
---@param instance rt_instance
function onStopInstance(solution, instance)
    print("Stopping instance " ..  instance:getName())
end

---@param solution Solution
function onShutdown(solution)
    print("Shutting down solution")
    api.factory.inference.delete(solution, "PVADetector")
    solution:unregisterUiCallback("onInputSinkConfig", "onInputSinkConfig")
end

function inputSinkConfig(input)
    local config = {
        {name = "Detector regions", configkey = "DetectorRegions", introtext = "Create custom detection regions to replace the full frame detection", itemprefix = "Region", type = "rectangle"}
    }

    return config
end

Instance script ¶

An instance main code file is called index.lua.

Info

For testing and debugging purpose, it's possible to use a different main file for the instances. It can be set in the JSON file of an instance, or in the base_config.json of the solution, by setting the lua_script property of the Global object:

{
    "Global": {
        "lua_script": "lua/custom_index.lua"
    },
}

index.lua Structure ¶

File imports ¶

The first thing index.lua does is importing other lua files it may need.

dofile(luaroot .. "/api/api.lua")
dofile(project_root .. "classifier_tracker.lua")
dofile(project_root .. "utils.lua")
---(...)

Lua state global vars

Lua state always has the variables luaroot and project_root defined for the main lua API folder inside CVEDIA-RT and the current lua project folder path.

onStartInstance()¶

Next the onStartInstance function is defined, this function is the first being called when starting the instance and should be where all the workers are defined.

Workers

In CVEDIA-RT a Worker is an instance thread that has its own onInit and onRun functions. This methods allows multithreading for the instances.

function onStartInstance(name, config)
    local instance = api.thread.getCurrentInstance()
    instance:createWorker("onInit", "onRun")
end

onInit()¶

The onInit() function runs once when starting the Instance.

Initializing modules

Usually a module can be loaded by calling api.factory.[MODULENAME].create(instance, [MODULENAME_ON_JSON]).

[MODULENAME_ON_JSON] is used to read and write the settings of the module in the instance or solution config JSON.

function onInit(instance)

    local inputInst = api.factory.input.create(instance, "Input")
    inputInst:setSourceFromConfig()

    local outputInst = api.factory.output.create(instance, "Output")
    outputInst:loadHandlersFromConfig()

    zoneInst = api.factory.zone.create(instance, "Zone")
    zoneInst:registerKnownZones()

    -- Create other modules (...)

    return true
end

onInit() return value

onInit() must return true, signaling the engine that initialization was completed successfully.

onRun()¶

The onRun() function runs continuosly and it's where the video analitycs logic is implemented

function OnRun(currentTime)

    local instance = api.thread.getCurrentInstance()

    -- Get next frame. If the instance is paused we must ignore the skip_frame functionality
    local inputFrames = input:readMetaFrames(instance:isPaused())

    if #inputFrames == 0 then
        -- No frames available yet due to FPS syncronization
        instance:sleep(1) -- avoid CPU burn
        return InstanceStepResult.INSTANCE_STEP_INVALID
    end

    local inputimg = inputFrames[1].image
    local metadata = inputFrames[1].metadata -- data received from the REST API or other protocols

    if inputimg == nil or not inputimg:is_valid() then
        instance:sleep(1) -- avoid CPU burn
        return InstanceStepResult.INSTANCE_STEP_INVALID
    end

    -- Run detector
    -- DetectorRegions is the name given in the inputSinkConfig function for storing the detection region
    -- settings, as defined on the solution.lua code
    ---@type Rect[]
    local regions = instance:getConfigValue("DetectorRegions")

    local detectInst = api.factory.inference.get(instance:getSolution(), "PVADetector")

    ---@type Detection[]
    local detections = detectInst:runInference(regions, inputimg)
    if #detections > 1 and #regions > 1 then
        detections = api.utils.NMS(detections, 0.5)
    end

    Instance:writeOutputSink("source", OutputType.Image, inputimg, "Input image")
    Instance:writeOutputSink("detections", OutputType.BBox, detections, "Raw detections (untracked)")
    Instance:flushOutputSink()

    return InstanceStepResult.INSTANCE_STEP_OK
end

Ouptut Sinks

For more information on the Output Sink check here.

onRun() currentTime parameter

The onRun() function always receives a parameter that indicates the real timestamp at which the function was called. This can be used for timing and performance data gathering.