GStreamerReader Plugin ¶

Description ¶

The GStreamerReader plugin is a multimedia input plugin that leverages the GStreamer framework to provide comprehensive video and audio input capabilities for CVEDIA-RT. It offers advanced streaming support, pipeline management, and integration with various multimedia sources and formats.

GStreamerReader provides a powerful and flexible video input solution by utilizing the entire GStreamer multimedia framework ecosystem. This plugin enables CVEDIA-RT to access a wide range of multimedia sources and formats through GStreamer's extensive plugin architecture.

Key capabilities include:

GStreamer Pipeline Management: Advanced pipeline construction and management with full control over the multimedia processing chain
Multi-source Support: Seamless handling of cameras, files, network streams, and custom multimedia sources
Real-time Streaming: Optimized for live video streaming and processing with low-latency capabilities
Format Negotiation: Automatic format detection and capability negotiation between pipeline elements
Buffer Management: Efficient buffer handling and memory management for optimal performance
Error Recovery: Robust error handling and stream recovery mechanisms to maintain connection stability
Plugin Ecosystem: Access to the entire GStreamer plugin ecosystem for advanced multimedia processing

Requirements ¶

Software Requirements ¶

GStreamer Framework: Complete GStreamer multimedia framework (1.0 or later)
GStreamer Plugins: Base, good, bad, and ugly plugin sets for comprehensive format support
GLIB2: GLib library for object management and utilities
GObject: GObject library for object-oriented programming in C

Hardware Requirements ¶

Standard video processing capabilities
Network connectivity for streaming sources
Hardware acceleration support (optional, when available)

Configuration ¶

Basic Configuration ¶

{
  "input": {
    "handlers": {
      "gstreamer-input": {
        "uri": "rtsp://camera.example.com/stream",
        "plugin": "GStreamerReader",
        "enabled": true
      }
    }
  }
}

Advanced Configuration ¶

{
  "input": {
    "handlers": {
      "gstreamer-input": {
        "uri": "rtsp://camera.example.com/stream",
        "plugin": "GStreamerReader",
        "enabled": true,
        "config": {
          "real_time": false,
          "sampling_rate": 30,
          "scale_width": 1920,
          "scale_height": 1080,
          "retry_backoff": 10,
          "connection_timeout": 30,
          "pull_timeout": 60,
          "pull_retries": 3,
          "pull_backoff": 100
        }
      }
    }
  }
}

Configuration Parameters ¶

Parameter	Type	Default	Description
`real_time`	boolean	`false`	Slow down disk-based video reading to simulate actual FPS
`sampling_rate`	integer	`0`	If set and real_time is false, sets FPS to this value instead of video default
`scale_width`	integer	`0`	Target width for video scaling (0 = no scaling)
`scale_height`	integer	`0`	Target height for video scaling (0 = no scaling)
`retry_backoff`	integer	`10`	Wait time in seconds before reconnecting after failure
`connection_timeout`	integer	`30`	Initial connection timeout in seconds
`pull_timeout`	integer	`60`	Pull frame timeout in seconds (must be a keyframe to count)
`pull_retries`	integer	`3`	Number of retries to pull a frame before reconnection
`pull_backoff`	integer	`100`	Sleep time between retries in milliseconds

Supported Sources and Formats ¶

Input Sources ¶

Local Video Files: MP4, AVI, MOV, MKV, and other common video formats
Network Streams: RTSP, RTMP, HTTP/HTTPS live streams
IP Cameras: Network cameras supporting standard streaming protocols
USB Cameras: Local USB-connected cameras and webcams
Custom Pipelines: User-defined GStreamer pipeline descriptions

Video Formats ¶

The plugin supports all video formats available through the GStreamer ecosystem, including:

Codecs: H.264, H.265/HEVC, VP8, VP9, MJPEG, and more
Containers: MP4, AVI, MOV, MKV, WebM, FLV, and others
Streaming Protocols: RTSP, RTMP, HLS, DASH, and custom protocols

API Reference ¶

Core Classes ¶

GStreamerReaderCore ¶

The main GStreamer-based reader implementation that handles pipeline management and frame processing.

Key Methods:

openUri(const std::string& uri): Opens a multimedia URI for reading
readNextFrame(): Reads the next available frame from the stream
readFrame(cmap frameSettings): Reads a frame with specific settings
getNextFrame(): Gets the next frame without advancing the stream
setFps(float fps): Sets the target frame rate
getFps(FPSType fpsType): Gets the current frame rate
getCurrentTimestamp(): Returns the current timestamp
getCurrentFrame(): Returns the current frame index
isEnded(): Checks if the stream has ended
close(): Closes the current stream

GStreamerReaderHandler ¶

Handles GStreamer pipeline events and manages the multimedia processing workflow.

GStreamerUtils ¶

Utility functions for GStreamer operations, including codec detection and pipeline management.

Configuration Structure ¶

struct config {
    bool real_time = false;
    int sampling_rate = 0;
    int scale_width = 0;
    int scale_height = 0;
    int retry_backoff = 10;
    int connection_timeout = 30;
    int pull_timeout = 60;
    int pull_retries = 3;
    int pull_backoff = 100;
};

Statistics Structure ¶

struct stats {
    int connections = 0;
    int input_width = 0;
    int input_height = 0;
    int input_fps = 0;
    int target_fps = 0;
    int current_frame = 0;
    float current_frame_skip = 0;
    int total_frames = 0;
    std::string current_uri = "";
};

Examples ¶

Camera Input ¶

{
  "input": {
    "handlers": {
      "camera-stream": {
        "uri": "rtsp://192.168.1.100:554/stream",
        "plugin": "GStreamerReader",
        "enabled": true,
        "config": {
          "connection_timeout": 30,
          "pull_timeout": 60,
          "retry_backoff": 5
        }
      }
    }
  }
}

Network Stream with Scaling ¶

{
  "input": {
    "handlers": {
      "network-stream": {
        "uri": "http://example.com/live/stream.m3u8",
        "plugin": "GStreamerReader",
        "enabled": true,
        "config": {
          "scale_width": 1280,
          "scale_height": 720,
          "real_time": true
        }
      }
    }
  }
}

Custom GStreamer Pipeline ¶

{
  "input": {
    "handlers": {
      "custom-pipeline": {
        "uri": "gst-pipeline://videotestsrc pattern=ball ! video/x-raw,width=640,height=480 ! videoconvert",
        "plugin": "GStreamerReader",
        "enabled": true,
        "config": {
          "sampling_rate": 25
        }
      }
    }
  }
}

File Playback with Frame Rate Control ¶

{
  "input": {
    "handlers": {
      "video-file": {
        "uri": "file:///path/to/video.mp4",
        "plugin": "GStreamerReader",
        "enabled": true,
        "config": {
          "real_time": false,
          "sampling_rate": 15
        }
      }
    }
  }
}

Performance Considerations ¶

Pipeline Optimization ¶

Hardware Acceleration: Utilize hardware-accelerated decoders when available (e.g., nvdec for NVIDIA GPUs)
Buffer Management: Configure appropriate buffer sizes for your use case
Format Negotiation: Allow GStreamer to negotiate optimal formats automatically
Memory Usage: Monitor memory consumption for long-running streams

Network Streaming ¶

Connection Timeouts: Adjust timeout values based on network conditions
Retry Logic: Configure retry parameters for unstable network connections
Buffering: Use appropriate buffering strategies for live vs. on-demand content

Recommended Settings ¶

{
  "config": {
    "connection_timeout": 30,
    "pull_timeout": 60,
    "retry_backoff": 10,
    "pull_retries": 3,
    "pull_backoff": 100
  }
}

export GST_DEBUG=3
# On Windows: set GST_DEBUG=3

Use GStreamer tools for pipeline testing:

gst-launch-1.0 rtsp://camera.example.com/stream ! autovideosink

Known Limitations ¶

Windows-specific plugin paths may require manual configuration
Some advanced GStreamer features may require additional plugin installations
Hardware acceleration availability depends on system configuration