Skip to content

Release notes - CVEDIA-RT 2022.11.0

New Solutions

Security

This release introduces a new solution that is focused on perimeter security and video analytics. It’s a complete and flexible solution ready to be deployed in production environments.

Many types of events are included:

  • Intrusion Detection
  • Crowding Detection / Estimation
  • Line Crossing / Counting
  • Tailgating
  • Occupancy
  • Object Left / Removed

Events can distinguish between people, vehicles, animals, or just general movement, which means you can eliminate false alarms at the source.

Exporting the events out of CVEDIA-RT is possible through one of the supported protocols:

  • JSON file
  • MQTT
  • CSV file
  • JPG or PNG
  • MP4
  • RTSP / RTMP

Because this solution is designed to run on any hardware with any number of cameras we’ve included an AI configuration panel that helps you configure the performance/accuracy trade-off.

Engine Updates

AI Models

  • Added Crowd Estimation
  • Improved Vehicle, Person, and Animal Detector
  • Improved Model Vehicle Classifier in RGB and Thermal
  • Improved 3D Bounding Box Classifier
  • Improved Face Detector
  • Improved Package Detector

New User Interface

To improve the overall user experience and usability of CVEDIA-RT the User Interface has been drastically reworked. The goal was to make it easier to setup cameras, configure solutions, and export event data. To support this there are several new panels available:

Input panel

The new Input Configuration panel allows you to easily draw and configure areas, lines, and detection regions.

Each solution defines what type of events can be monitored. And each event type can be tuned based on specific needs, such as count thresholds, time limits, etc.

The playback controls are now displayed on the Input panel

Output preview

The new Output panel is now a powerful tool to visualize, debug, or present, all the inference results and other runtime information.

You can also greatly customize the look of bounding boxes, zones, lines, etc… and chose what to show or hide on the fly, without having to interact with the Lua scripts.

Data Visualizer

The Data Visualizer panel shows the real-time data that are produced by the running instance. It also shows the events produced by the solution and the JSON data of the events.

Note: The Data Visualizer is meant to show real-time data. If the panel is not open or is not rendered, it will not “collect” events. So when you open it, it will not show historical events.

Config panel

Each solution can expose different and specific settings that you can customize.

For example, in the Security solution, you can choose what Detector to use, the sensor modality RGB or Thermal, and the Motions and Movement detection settings.

In addition to the solution settings, you can access all the loaded modules, like Tracker, Motion, Detector, etc…

Exporting Data and Events

Events are not part of the output sink and you can export them in the same way as any data that is being visualized in the Output panel.

Inside the Config panel, the solutions can expose the exporting options.

Here you can enable any of the pre-configured export settings.

You can edit the path, the output streaming URL, and other parameters specific to the exporting format you enable.

Additionally, you can also add and remove more export options.

Supported formats and protocols:

  • JSON file
  • MQTT
  • CSV file
  • JPG or PNG
  • MP4
  • RTSP / RTMP

Normalized Coordinates

All the coordinates are now normalized in the range [0,1] instead of being absolute in pixels. This allows setting zones and lines independent from the input resolution, which could change at runtime for performance reasons.

Remote UI

You can now access the CVEDIA-RT UI even when running headless on the cloud or edge devices.

The new Remote UI Client can connect to a running CVEDIA-RT instance and show the complete user interface you would have when running on a PC.

Hardware & Platforms

Hailo

The performance of our models running on the Hailo chipset is significantly improved.

Added support for HailoRT 4.6.0, 4.8.1, 4.9.0, and 4.10.0.

Additionally, many more solutions can run without limitations. For now, the unsupported ones are Crowd Estimation and Fall Detection.

Qualcomm Snapdragon RB5

Added support for Qualcomm Snapdragon RB5.

All our solutions (except for Crowd Estimation and Fall Detection) run smoothly on the RB5.

Jetson Nano 4GB

Optimized the solutions to run on NVIDIA Jetson Nano with less available memory.

Documentation

Our online documentation has been reorganized and extended with more detailed information for users and developers.

RTSP

Major rework has been done to improve RTSP input and output:

  • fixed start/stop issues
  • automatic reconnect
  • better defaults for low latency
  • added missing codecs
  • support for Nvidia accelerated io

RTMP

Added support for RTMP on the GStreamerWriter module and added a manual of GStreamer pipelines.

LUA Scripting

  • Added the concept of a solution orchestrator with a dedicated solution.lua script file. It can be used to load unique shared instances of the inference modules and to configure the UI options for the Input panel and for the exporting of the data.
  • Refactored the API for an easier and better-organized namespacing approach.
  • Improvement of the method definitions to support autocompletion in Lua
  • Triggers were split into Zones and Tripwires, these two modules now contain the features that pertain only to each specific usage.
  • Refactored Lua API, changes:
  • sapi → api in all Lua interface functions
  • Moved the lua folder out of assets folder
  • Removed the config node from config files (as all properties are accessible with instance:getConfigValue)
  • rt:createWorker does not receive the name parameter anymore
  • project_root var now points to the Lua folder in the project
  • Module creation now uses the following syntax:

    detectInst = api.factory.inference.create(instance, "Detector") -- To create the module in the instance scope
    detectInst = api.factory.inference.create(solution, "Detector") -- To create the module in the solution scope
    
  • Input and Inference now have a SetSourceFromConfig method in Lua, to set source from the config file

  • Instance:registerSink("xxx") is not needed anymore.
  • Refactored Lua sink system:

    Instance:writeToSink("Detector", { image = outputimg, meta = detections }) is now used as Instance:writeOutputSink("detections", OutputType.BBox, detections, "Raw detections") followed by Instance:flushOutputSink()

  • Data for Zones and Tripwires now live on the instance data, and are managed on the Input window (view Zones Configuration) and shared between modules, and not on the modules themselves.

  • Added solution scope to Lua projects
  • New module registry system for Lua interface creation
  • New Input handler register system
  • Refactored the Input playback system
  • Use of normalized coordinates to define zones, triggers and all other internal systems
  • Added Dyanmic Strings feature to the engine.
  • Add Output Handlers for TXT and CSV on the WriteData plugin. These will allow outputting data with predefined formats.
  • Added a documentation page for exporting data, explaining the different ways in which data can be exported from CVEDIA-RT

Fixes/Improvements

  • Bug: UI Fails to start when window position is out of bounds
  • Bug: Variable Input Size doesn't work in CRT
  • Bug: Error with directory input containing large amout of images
  • Some file can get corrupted when saving to disk
  • Bug: Concave polygon drawing doesn't work properly in CRT
  • Tracker - track jumps between two objects next to each other
  • Tracker: source box not showing in the tracker output
  • Debug stuff in solutions
  • UTF-8 Chars in Win path of RT causes not loading of the Plugins
  • .gitkeep is left in release directory structure
  • Tracking box remains for too long after the target exits the view
  • Detector region is rendered (but not used) after restarting an instance
  • Images from different solutions info pages are being mixed up
  • runinference tool doesnt work with local onnx
  • Vehicle classifier does not have softmax enabled
  • Setting FPS player playback affects the frame rate of the UI
  • Thermal triggers left double count, only first time
  • Unexpected errors in the log when saving an instance state
  • Set FPS does nothing
  • Browse dialog is not pre-configured
  • PVA motion text is too large
  • disable_ui setting don't work properly
  • Multiple TRT instances go out with a boom
  • Trigger zone setting is lost when selecting another zone right after
  • Wrong merging of base_config into instance json file
  • 3D Bounding Box is still slow
  • base_config-only solutions
  • Starting a invalid instance = forever locked
  • Playback forward button jumps 2 frames
  • RT hangs when scrubbing in video
  • Lua crashes when there's too many objects found
  • Wrong value shown in the detector config UI
  • 16 instance limit
  • The new save_state REST API method doesn't work
  • Memory leak is back
  • Invalid base64 encoded strings break MQTT
  • CVEDIA-RT hangs when starting an instance with invalid json
  • Contact us button in rt.cvedia.com has an invalid link
  • Many links in the current published documentation are broken
  • CVEDIA-RT output handlers config
  • CVEDIA-RT crashes and wipe global.config
  • Saving crops is unreliable
  • SegFaults on instance stop (linux)
  • CVEDIA-RT internal headers
  • RT hangs and crashes if invalid data are passed to trigger::checkZones
  • Repeat mode not working on release version
  • Cannot properly stop RTSP instances
  • Solving proxy issue with CVEDIA-RT
  • Properly shutdown hailo inference
  • CVEDIA-RT in hailo new versions
  • Input:playlistReachedEnd() is not working
  • runinference cluttered output
  • CVEDIA-RT JSON editor
  • Playback skips the last frame / image
  • Motion vector reset doesn't work
  • Adding a key to an array turns it into a map silently breaking runInference()
  • Test GStreamer plugins
  • Check and fix tracker issues on the Demo solution
  • Save on a Solution level module, it should save on the instance json file, an not in memory only
  • RT hangs when moving the confidence threshold slider from min to max (new config panel for PVADetector)
  • If a detection region has zero height it causes the solution to halt
  • Inspector panel disappears
  • Crash on instance duplication
  • Reloading instances doesn't see new files
  • UI Exceptions while input is starting
  • solution.lua code get cached on first run and doesn't update
  • Random characters appear in the Input window left menu
  • The saved layout is not applied correctly
  • JSON editor style
  • File picker All files don't work
  • .mov is not a video
  • Implement MQTT new factory logic
  • the accordion menu of the input window cannot be closed
  • RT crashes when playing the playback bar
  • custom layout with Data windows crashes RT
  • ONVIF discovery gets stuck
  • ONVIF finds the same camera multiple times
  • RT UI hangs while downloading models
  • Clicking an empty area causes errors
  • Playback weird behavior
  • onInit pointer fight / Zones input edit / display
  • Remove xtensor buffer because it only gets in the way
  • Regions not being listed on startup if they were serialized in an open state
  • Writing to a unnamed outputsink breaks all others
  • Unescaped chars on the URI for RTSP and ONVIF are breaking RT
  • 3D Bounding boxes are not using the selected colors. They are always red.
  • Removing the last tripwire point from the UI crashes RT
  • Clicking on the output windows causes the UI to flash with an error in the log
  • Creating tripwires and missing fields
  • Editing zones
  • Creating new Zones causes empty exception
  • Changing tripwire direction has no effect
  • Custom projects folder don't work
  • Documentation missing links / detached pages
  • Each sink data entry needs to have a unique "id" field for UI to work
  • runinference does not working anymore. missing width height error
  • Tripwires new issues
  • Detections visuals
  • Shape discrepancies
  • Interface locks up when creating trip wires
  • Tripwire editor wire colors
  • Tripwire segfaults when moving shapes around
  • Data visualizer POT is broken
  • Privacy demo doesn't work well with custom detection regions
  • Crash when calling getTrackById with ID that doesn't exist
  • onLastStop() does not delete the loaded models
  • typo in the event manager actions
  • TimeStep when going backwards
  • Stop / Starting a instance crashes
  • RT crashes when resetting motion detection
  • Unexpected list of default Input options
  • Regression: the solution.lua code is cached and doesn't see the changes between runs.
  • Different hailo models in different solutions / instances
  • Classifier:runInference(toClassify) Fatal error
  • Smart-home rgb_pva_motion_outdoor hangs without crashing at random
  • Fatal error when stopping any solution
  • Video playlist doesn't go to the next video
  • From the second run of an instance, the Input window doesn't show any configure option
  • cv:resize exception inside runInference
  • Vanishing tracks when video changes
  • ONVIF cameras are not properly saved to global.config (end up in error state on startup)
  • Typo in CUAV
  • Crowd estimation - Shibuya - Dynamic input error
  • Inspecting detection breaks UI
  • Can't go to next frame
  • ITS solutions are not using the latest models
  • ITS - Vehicle counting indicator
  • ITS - Lane occupancy is not accurate
  • OOM crash when start/stop crowd counting multiple times
  • UI Scale factor
  • Image corruption on very busy crowds
  • Cannot open GUI on Jetson
  • Amba Preset Build on RT Develop Branch cannot build AmbaOut plugin - Old LUA API still in use
  • RunInference producing random results on ONNX file
  • Different inference results between local ONNX and ModelForge
  • Last broken tests
  • UI Scaling and spacing
  • Zones/Tripwires can't be fetched in lua before the second frame
  • Unable to change zone transparency in lua
  • CLI: listnndevices
  • Crashes when opening the Input Window
  • TRT inference crashes on shutdown (jetson)
  • ONVIF discovery on linux
  • Add camera UI is too small
  • New masking regions are no longer transparent
  • Layout -> Delete layout
  • Layout -> Profiles = segfault
  • Add instance button
  • UI: Invisible dots
  • UI Inspector causes lua exceptions
  • bindEvent causes segfault on linux when stopping instance
  • Fonts are too big
  • Memory leak on playlists
  • Unable to get any state/config info from instance on onStartInstance
  • crowd-estimation people_walking instance doesn't run
  • Editing multiple tripwire/zone names goes funky
  • onStartInstance seems to be called in the main thread which hangs the app if I try to load a network
  • Events being shown as duplicated in the data visualizer
  • A new "Tripwire" object is added to the instance config when clicking on File > Save Selected
  • Basic PVA selection issue
  • Unable to access zones in lua
  • In gstreamer use avdec264 when nvidia dec is not available
  • Unable to access zones in lua
  • New zones should have transparency by default
  • Smart-Home: PVA Motion - Tripwire config
  • Crash when selecting a bbox from a detector in the output window
  • Video playlist order
  • Benchmark json exports
  • Can't select tracks in the output windows
  • UI: Layout jumps when ticking checkboxes
  • The event viewer shows a crop from an old frame
  • Exporting to MQTT is terribly slow
  • Export dynamic strings: sys.time
  • RTSP server crashes on exit
  • Remote CvediaRT on Amba - Not unloading / reusing model on Cavalry
  • When building Amba in CI, old assets directory gets copied
  • Input configuration accordion counters dont match the number of triggers inside
  • None of the videoreader config UI settings work
  • No example on how to export events with images from Security to disk.
  • security | vehical_tripwire_crossing
  • The dashboard is always empty
  • The timestamp passed to composeFrame needs to be in seconds with milliseconds after the decimal point.
  • CV25: Cavalry crashes when stopping instance
  • MQTT Works from LUA Only
  • Security should only use instance:getTimestep instead of system time
  • Vehical_area_occopany
  • Video playback is much faster than original video
  • When opening the configuration of the Input module, the video accelerates
  • writerData doesn't check if folder exists on linux
  • Privacy solution: Clicking readme segfaults on linux
  • inputInst:getCurrentFrame() is not returning the current frame
  • runinstance doesnt work
  • runinstance.exe -c or -p?
  • runinstance returns error code even when it runs successfully
  • running an instance through CLI takes 7 times longer than running it through UI
  • Export data is duplicated if there are multiple output handlers
  • Major memory leak when exporting data that use OutputHandlerProcess scripts
  • The Output and Data Visualizer panels show also custom sink types
  • Calling tracker methods inside the new IoU callback causes a deadlock
  • buffer.copy() doesn't copy frame ID and time
  • Cannot connect to RTSP output on Windows
  • Scrubbing or playback reset causes a negative timestep
  • Track suddenly disappearing
  • Security: Object removed, Object Left and Smart-home
  • Security: Person area crowding -> gigantic tracking box
  • writerData doesn't check if folder exists on linux
  • Better shell interface for ./cvediart
  • Network Backends should be destroyed when clicking on File > Reload Instances
  • Tripwires trigger cooldown
  • Ability to associate source boxes and tracks
  • Save layout should save states of toolbars
  • Evaluate input plugin granularity
  • Draw polygons and bbox in C++ instead of LUA
  • Implement Configurable Tripwire Bandwidth (C++)
  • Imgui on DX11
  • Use normalized coordinates to define zones and triggers and for all other internal systems
  • Tripwire temporal bandwith
  • CVEDIA-RT Instance status (API)
  • Playback TakeScreenshot gives no feedback
  • global.config proper documentation
  • Instance API naming inconsistency
  • Multiple Dimension Model Output
  • Major refactoring on RT
  • Entire codebase covered by unit tests
  • Motion: Ability to specify the learningRate when using MOG2 background subtraction
  • Visual feedback (Popup ?) for when trying to load a video with unsupported codec
  • gstreamer writer should support gstreamer:/// pipelines
  • Load multiple hailo models at once
  • Implement P2PNet
  • Implement new input config settings (FPS)
  • Add counting data window for bboxes
  • CVEDIA-RT binary should have a --version option
  • Trip wire colors
  • Trip wires directionality should be visible at output
  • Fix screencap to work on linux
  • Allow disabling of kalman filter
  • Allow override of kalman identity matrices
  • Ablility to export txt data and inference image output with the same filename as input image
  • drawCircle expects int radius but checks for float >= 2
  • Remote UI: Port is not configurable
  • Cloud deployment documentation
  • Smart-Home: PVA Motion - Wires are not clear
  • Add a 'details' icon to the event viewer to view the event content
  • Documentation User guide: Exporting data vs Output
  • Ability to check targets for a specific zone
  • writeData should have a image sink
  • runInference on a classifier should generate id as it happens for detectors
  • Allow for optional list of wires to check for when calling checkTripwires()
  • Expose createBuffer in Lua
  • Allow the tracker IoU matrix to be overriden from Lua
  • Increase ImGui slider precision for Tracker process noise cov
  • Include Base demo in the release
  • Validate that you can create custom configuration panel in the Config window
  • Remove bookmarks feature in the input player
  • Test and evaluate SNPE models
  • Review and update all the RT solutions for the next release
  • Create a Base Demo with exporting settings
  • Add Fisheye to Pano Video/Image transformation
  • Implement {sys.count} in the OutputHandler to export unique json filenames
  • AMBA: Implement Encoding Check in Amba Reader
  • Remove the legacy instance menu
  • Set Linear as default interpolation
  • Show events inside the Data Visualizer window
  • Support drawing of tracks using different colors in the Output window
  • Support playback bar on folder of images
  • Allow to overwrite render setting by passing settings in the sink
  • Update ITS to show different colors for the lanes depending on the number of vehicles
  • Re-save all the tripwires and zones in all the instances of all demos
  • Extend the input configuration to support a group by feature
  • Extend the input configuration to allow setting default values for new entries (zone, tripwires, etc...)
  • Prevent showing cog icon beside tripwires and zone if the onTriggerConfig callback is not set
  • Expose the distance of a track
  • Expand the UI callback to send all the trigger config in the context
  • Show numbers beside input configuration groups to show if there is something configured inside
  • Revisit the place for the trigger interaction instructions
  • Automatically open only the input configuration that has entries
  • Add Occupancy input configuration
  • Add Intrusion input configuration
  • Occupancy should trigger on enter and exit events
  • Review the movement detection using the parking lot demo video
  • Generate a guid id for each track in the Tracker plugin
  • Allow the UI to show only tracks with events
  • Expose a Lua API to generate GUID
  • Build RT With MQTT on Amba devices
  • Update Basic PVA Detection to use Output Handler scripts
  • Allow output handlers to be enabled/disabled from the JSON configuration
  • Convert the output handler vector to a map
  • Remove counters from zones where needed in Security by setting the default label to "None"
  • Update solutions to export buffers using {image=buffer}
  • Add help menu to RT
  • Document the new Export panel
  • Add the export panel to Sicurity
  • Add fall detection event
  • Add the export panel to Fall detection
  • Add gun detected event
  • Add the export panel to Gun Detection
  • Refine the event data for Security