Advanced IO with GStreamer¶
Using GStreamer you can build custom pipelines for decoding / encoding pretty much any media source.
CVEDIA-RT supports loading custom gstreamer plugins both on windows and linux, this includes proprietary plugins provided by NVIDIA and others.
You can use gstreamer for both input and output pipelines, CVEDIA-RT will sit in the middle processing the feed as a appsink and exporting it as a appsrc.
Example Input pipelines¶
Following there's a list of example pipelines, you should add them to your Input/uri parameter or in the UI at the gstreamer configuration.
appsink name
Any input pipelines need to end with a appsink name=cvdsink for CVEDIA-RT to see it
Generic automatic pipeline¶
gstreamer:///urisourcebin uri=rtsp://user:password@ip:port/address ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink
Note
This pipeline will try to automatic resolve the input uri type and codecs needed. It may not work for all types of media or if you don't have the gstreamer plugins needed to decode it.
GStreamer plugin ranks
If you have custom plugins for streamer you may need to change their rank (priority), CVEDIA-RT has a builtin system for this in rtconfig.json.
RTSP with h264¶
Windows:
gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph264depay ! h264parse ! openh264dec ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink
Linux:
gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph264depay ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink
RTSP with h265¶
Linux:
gstreamer:///rtspsrc location=rtsp://user:password@ip:port/address latency=0 ! rtph265depay ! avdec_h265 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink
h265 on windows
GStreamer on windows doesn't ship with opensource h265 codecs, only proprietary and paid codecs are available.
MJPEG¶
gstreamer:///souphttpsrc location=http://webcam01.ecn.purdue.edu/mjpg/video.mjpg is-live=true ! multipartdemux ! jpegdec ! videoconvert ! video/x-raw,format=BGR ! appsink drop=true name=cvdsink
HTTPS WEBM with MKV mux and h264 frames¶
gstreamer:///souphttpsrc ssl-strict=false location=https://user:pass@ip/path/to/stream?params is-live=true ! matroskademux ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=true name=cvdsink
Example Output pipelines¶
Following there's a list of example pipelines, you should add them to your Output/uri parameter or in the UI at the gstreamer configuration.
appsrc sink name
Any output pipelines need to start with a appsrc name=cvedia-rt for CVEDIA-RT to see it
Write h264 MP4 to disk¶
Windows:
gstreamer:///appsrc name=cvedia-rt ! videoconvert ! openh264enc ! queue ! filesink location=output/file.mp4
Linux:
gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc ! queue ! filesink location=output/file.mp4
RTMP server¶
This requires a RTMP server (such as red5 or live555) to sink data.
Lowest latency:
gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=1 ! video/x-h264,profile=baseline ! flvmux latency=0 start-time-selection=1 streamable=true ! rtmpsink location='rtmp://ip:port/path live=true'
High quality:
gstreamer:///appsrc name=cvedia-rt ! videoconvert ! x264enc ! video/x-h264,profile=high ! flvmux start-time-selection=1 streamable=true ! rtmpsink location='rtmp://ip:port/path live=true'
Adding your own plugins¶
On Windows:
Add your plugins to files\gstreamer_plugins. CVEDIA-RT ships with the latest (currently 1.19.2) gstreamer provided by vcpkg
On Linux:
Plugins are automatically loaded from your local /usr/lib/$(uname -p)-linux-gnu/gstreamer-1.0/ folder. You can define a custom path by defining GST_PLUGIN_PATH env variable.
NVIDIA Jetson platform
On Jetson your GPU accelerated plugins depends on Jetpack, which may be using an older version of gstreamer. Those plugins are not upgradable as they highly depend on jetpack.