Advanced IO with GStreamer ¶

Using GStreamer you can build custom pipelines for decoding / encoding pretty much any media source.

CVEDIA-RT supports loading custom gstreamer plugins both on windows and linux, this includes proprietary plugins provided by NVIDIA and others.

You can use gstreamer for both input and output pipelines, CVEDIA-RT will sit in the middle processing the feed as a appsink and exporting it as a appsrc.

Example Input pipelines ¶

Following there's a list of example pipelines, you should add them to your Input/uri parameter or in the UI at the gstreamer configuration.

appsink name

Any input pipelines need to end with a appsink name=cvdsink for CVEDIA-RT to see it

Generic automatic pipeline ¶

gstreamer:///urisourcebin uri=rtsp://user:password@ip:port/address ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

Note

This pipeline will try to automatic resolve the input uri type and codecs needed. It may not work for all types of media or if you don't have the gstreamer plugins needed to decode it.

GStreamer plugin ranks

If you have custom plugins for streamer you may need to change their rank (priority), CVEDIA-RT has a builtin system for this in rtconfig.json.

RTSP with h264 ¶

Windows:

gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph264depay ! h264parse ! openh264dec ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

Linux:

gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph264depay ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

RTSP with h265 ¶

Linux:

gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph265depay ! avdec_h265 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

h265 on windows

GStreamer on windows doesn't ship with opensource h265 codecs, only proprietary and paid codecs are available.

MJPEG ¶

gstreamer:///souphttpsrc location=http://webcam01.ecn.purdue.edu/mjpg/video.mjpg is-live=true ! multipartdemux ! jpegdec ! videoconvert ! video/x-raw,format=BGR ! appsink drop=true name=cvdsink

HTTPS WEBM with MKV mux and h264 frames ¶

gstreamer:///souphttpsrc ssl-strict=false location=https://user:pass@ip/path/to/stream?params is-live=true ! matroskademux ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=true name=cvdsink

Example Output pipelines ¶

Following there's a list of example pipelines, you should add them to your Output/uri parameter or in the UI at the gstreamer configuration.

appsrc sink name

Any output pipelines need to start with a appsrc name=cvedia-rt for CVEDIA-RT to see it

Write h264 MP4 to disk ¶

Windows:

gstreamer:///appsrc name=cvedia-rt ! videoconvert ! openh264enc ! queue ! filesink location=output/file.mp4

Linux:

gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc ! queue ! filesink location=output/file.mp4

RTMP server ¶

This requires a RTMP server (such as red5 or live555) to sink data.

Lowest latency:

gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=1 ! video/x-h264,profile=baseline ! flvmux latency=0 start-time-selection=1 streamable=true ! rtmpsink location='rtmp://ip:port/path live=true'

High quality:

gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc ! video/x-h264,profile=high ! flvmux start-time-selection=1 streamable=true ! rtmpsink location='rtmp://ip:port/path live=true'

Adding your own plugins ¶

On Windows:

Add your plugins to files\gstreamer_plugins. CVEDIA-RT ships with the latest (currently 1.19.2) gstreamer provided by vcpkg

On Linux:

Plugins are automatically loaded from your local /usr/lib/$(uname -p)-linux-gnu/gstreamer-1.0/ folder. You can define a custom path by defining GST_PLUGIN_PATH env variable.

NVIDIA Jetson platform

On Jetson your GPU accelerated plugins depends on Jetpack, which may be using an older version of gstreamer. Those plugins are not upgradable as they highly depend on jetpack.