Debugging GStreamer Pipelines: A Comprehensive Guide for CVEDIA-RT Users¶

Navigating the waters of multimedia applications, especially with intricate streaming protocols like RTSP, can sometimes be challenging. For CVEDIA-RT users, a flawless GStreamer pipeline configuration is indispensable. This post aims to guide you through the intricacies of the GST_DEBUG environment variable on both Windows and Linux platforms. Additionally, we'll introduce the ffprobe and ffplay diagnostic tools, vital instruments in troubleshooting pipeline issues.

Jetson Orin

If you are looking to solve a connectivity issue with the RTSP input on the Jetson Orin, you can try the following pipeline:

gstreamer:///rtspsrc protocols=tcp location="rtsp://[YOUR URI HERE]" short-header=true do-retransmission=false ! rtph264depay ! h264parse ! avdec_h264 ! videorate ! videoconvert ! video/x-raw,format=BGR ! appsink drop=true name=cvdsink

Please note that this pipeline is specific to the h264 encoding.

Peeling back GStreamer with GST_DEBUG¶

The GST_DEBUG environment variable grants you a magnifying lens into GStreamer's inner workings. Through this variable, you can modulate debug message verbosity and category, aiding in diagnosing pipeline snags.

Debug Levels:

none (0): No messages.
error (1): Only error messages.
warning (2): Warnings and errors.
info (3): Informational messages and lower.
debug (4): Debug messages and lower.
log (5): Log messages and everything else.

Setting Up GST_DEBUG¶

Windows

Command Prompt:
```
set GST_DEBUG=3
```
System-wide: Incorporate GST_DEBUG as an environment variable through the System Properties.
Batch File: A .bat file can be created for temporary debug configurations for specific applications.

Linux

Terminal Session:
```
export GST_DEBUG=3
```
System-wide: Integrate the line export GST_DEBUG=... into your .bashrc or .zshrc, depending on your shell.

Deciphering streams with ffprobe and ffplay¶

From the FFmpeg suite, the ffprobe and ffplay tools emerge as saviors for diagnosing stream troubles.

ffprobe: An insightful tool for dissecting multimedia streams. It provides a detailed breakdown of the stream's components.
```
ffprobe rtsp://...
```
ffplay: An essential utility to quickly verify the health of an RTSP feed.
```
ffplay rtsp://...
```

Where to find and download ffprobe and ffplay¶

Both ffprobe and ffplay are part of the FFmpeg suite. To acquire them:

Windows

Visit the FFmpeg official website.
Navigate to the Windows section and select a link from the provided options.

Tip

A recommended source is the Gyan Doshi FFmpeg builds where you can download the ffmpeg-release-essentials.zip file.

Linux

Most Linux distributions have FFmpeg available in their repositories. You can typically install it using your package manager. For example, on Ubuntu or Debian:

sudo apt update
sudo apt install ffmpeg

Probing Streams with ffprobe and ffplay¶

Two indispensable tools from the FFmpeg suite for diagnosing stream issues are ffprobe and ffplay.

ffplay

A quick and simple way to verify if an RTSP feed is functional.

ffplay rtsp://...

ffprobe

This tool inspects multimedia streams, revealing details like metadata, codecs, and more.

ffprobe rtsp://...

Example output:

Input #0, rtsp, from 'rtsp://...':
  Metadata:
    title           : ...
  Duration: ...
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 90k tbn, 50 tbc

From the above, we can ascertain that the video stream is H.264 encoded with a resolution of 1920x1080 at 25 fps. This can inform potential optimizations or adjustments in our GStreamer pipeline.

Refining the CVEDIA-RT Pipeline with Insights from ffprobe and Real-World Scenarios¶

Using the default GStreamer pipeline provided by CVEDIA-RT:

gstreamer:///urisourcebin uri=rtsp://... ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

ffprobe, combined with real-world user insights, can be invaluable in fine-tuning your GStreamer pipeline. Let's explore these adjustments based on various scenarios:

H.264 Encoded Video: If ffprobe indicates your video stream is encoded with H.264, optimize with a specialized decoder. Adapt the pipeline to use avdec_h264 for decoding:
```
gstreamer:///urisourcebin uri=rtsp://... ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink
```

H.265 (HEVC) Encoded Video: For streams encoded with H.265 (HEVC), leverage the avdec_h265 decoder for better performance:

gstreamer:///urisourcebin uri=rtsp://... ! avdec_h265 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=true name=cvdsink

Explicit Protocol Setting: When faced with connection issues, explicitly setting your RTSP protocol might bypass certain network-related challenges. For a more stable connection, especially on stricter networks, use protocols=tcp:
```
gstreamer:///rtspsrc protocols=tcp location=rtsp://...
```
RTSP Header Adjustments: Some streams may require "short start codes". If ffprobe or other diagnostics suggest this, integrate short-header=true in the pipeline.
Retransmission Control: If your stream stutters or lags, especially on congested networks, consider setting do-retransmission=false to achieve smoother playback.
Explicit Depacketization and Decoding: Providing explicit steps for depacketization, parsing, and decoding ensures the stream undergoes proper processing. This approach is especially beneficial for H.264 streams:
```
... ! rtph264depay ! h264parse ! avdec_h264 ...
```
Frame Rate Consistency: To combat variable frame rates or ensure a steady delivery, include videorate in your pipeline.
Resolution Adjustments: If ffprobe reveals a notably high stream resolution, and you don’t need such high fidelity, consider downscaling to enhance processing speed:
```
... ! videoconvert ! videoscale ! video/x-raw, width=1920, height=1080, format=BGR ...
```

Harnessing the insights from ffprobe and the experiences of other users ensures your pipeline is not only compatible but also optimized for the specific intricacies of your RTSP feed.

Conclusion¶

In the multifaceted realm of multimedia streaming, tools such as GST_DEBUG, ffprobe, and ffplay are not just handy, but often indispensable. They offer profound insights into the streaming mechanism, enabling you to tweak and refine for the best results. Always be ready to delve, diagnose, and adjust!

Happy streaming!