Skip to content

GStreamerWriter Plugin

Description

The GStreamerWriter plugin provides comprehensive multimedia output capabilities using the GStreamer framework. It supports various output formats including RTSP streaming, file recording, network protocols, and custom GStreamer pipelines. The plugin offers both RTSP server functionality for live streaming and flexible GStreamer pipeline support for file outputs and advanced streaming scenarios.

Key Features

  • RTSP Server: Built-in RTSP server with multi-stream support
  • Custom GStreamer Pipelines: Full GStreamer pipeline flexibility
  • Multiple Output Formats: Video files, streams, network protocols
  • Hardware Acceleration: Support for hardware-accelerated encoding
  • Multi-Protocol Support: RTSP, HTTP, UDP, TCP, file formats
  • Session Management: Automatic RTSP session cleanup and management
  • Real-time Streaming: Low-latency live video streaming
  • Configurable Encoding: Adjustable quality, bitrate, and codec settings

When to Use

  • RTSP live streaming to clients
  • Recording video files in various formats
  • Network streaming protocols (UDP, TCP)
  • Hardware-accelerated video encoding
  • Custom video processing pipelines
  • Multi-client video distribution
  • Integration with existing GStreamer workflows

Requirements

Software Dependencies

  • GStreamer framework (≥1.16)
  • GStreamer RTSP server library
  • GStreamer plugins (base, good, bad, ugly)
  • Video codec libraries (x264, x265, etc.)
  • Optional: Hardware acceleration libraries (NVENC, VAAPI, etc.)

Hardware Requirements

  • Sufficient CPU for software encoding (or GPU for hardware acceleration)
  • Network bandwidth for streaming applications
  • Storage space for file-based outputs
  • Compatible hardware encoders (optional)

Configuration

Basic Configuration

{
  "output": {
    "handlers": {
      "rtsp-stream": {
        "uri": "rtsp://0.0.0.0:8554/live",
        "sink": "output",
        "enabled": true,
        "fps": 30,
        "monotonic": true
      }
    }
  }
}

Advanced RTSP Configuration

{
  "output": {
    "handlers": {
      "hd-rtsp": {
        "uri": "rtsp://0.0.0.0:8554/hd-stream",
        "sink": "video",
        "enabled": true,
        "fps": 30,
        "monotonic": true,
        "pipeline": "( appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=ultrafast tune=zerolatency bitrate=4000 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
      },
      "mobile-rtsp": {
        "uri": "rtsp://0.0.0.0:8555/mobile-stream", 
        "sink": "video",
        "enabled": true,
        "fps": 15,
        "monotonic": true,
        "pipeline": "( appsrc name=cvedia-rt ! videoscale ! video/x-raw,width=1280,height=720 ! videoconvert ! x264enc speed-preset=fast bitrate=1000 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
      }
    }
  }
}

File Output Configuration

{
  "output": {
    "handlers": {
      "mp4-recording": {
        "uri": "gstreamer:///recordings/output.mp4",
        "sink": "output",
        "enabled": true,
        "fps": 25,
        "monotonic": true,
        "pipeline": "appsrc name=cvedia-rt ! videoconvert ! x264enc ! mp4mux ! filesink location=/recordings/output.mp4",
        "debug": "0"
      },
      "mkv-recording": {
        "uri": "gstreamer:///recordings/output.mkv",
        "sink": "output", 
        "enabled": true,
        "fps": 30,
        "monotonic": false,
        "pipeline": "appsrc name=cvedia-rt ! videoconvert ! x265enc speed-preset=medium ! matroskamux ! filesink location=/recordings/output.mkv"
      }
    }
  }
}

Hardware Acceleration Configuration

{
  "output": {
    "handlers": {
      "nvenc-rtsp": {
        "uri": "rtsp://0.0.0.0:8554/nvenc-stream",
        "sink": "output",
        "enabled": true,
        "fps": 30,
        "pipeline": "( appsrc name=cvedia-rt ! videoconvert ! nvh264enc bitrate=5000 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
      },
      "vaapi-rtsp": {
        "uri": "rtsp://0.0.0.0:8554/vaapi-stream",
        "sink": "output",
        "enabled": true,
        "fps": 30,
        "pipeline": "( appsrc name=cvedia-rt ! videoconvert ! vaapipostproc ! vaapih264enc bitrate=3000 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
      }
    }
  }
}

Configuration Schema

Parameter Type Default Description
uri string required Output URI (rtsp:// or gstreamer://)
sink string "output" Data sink to connect to
enabled boolean true Enable/disable output
fps integer 30 Target frames per second
monotonic boolean true Use monotonic timestamps
pipeline string auto Custom GStreamer pipeline
debug string "0" GStreamer debug level (0-5)

URI Schemes

The GStreamerWriter plugin supports two URI schemes:

RTSP Scheme (rtsp://)

Format: rtsp://[host]:[port]/path

  • host: Server bind address (default: 0.0.0.0)
  • port: RTSP server port (default: 8554)
  • path: Stream path identifier

Examples: - rtsp://0.0.0.0:8554/live - Default RTSP stream - rtsp://192.168.1.100:8555/camera1 - Specific host and port - rtsp://localhost:9554/analytics - Local analytics stream

GStreamer Scheme (gstreamer://)

Format: gstreamer://path (path used as dynamic parameter)

  • Used for file outputs and custom pipelines
  • Path parameter passed to GStreamer pipeline
  • Requires explicit pipeline configuration

API Reference

C++ API

The GStreamerWriter plugin implements the iface::OutputHandler interface:

class GStreamerWriterOutput : public iface::OutputHandler {
public:
    // Constructor
    GStreamerWriterOutput(const std::string& moduleName,
                         const std::string& schema, 
                         const std::string& sink,
                         const std::string& uri, 
                         pCValue config);

    // Factory method
    static std::shared_ptr<iface::OutputHandler> create(
        const std::string& moduleName,
        const std::string& schema,
        const std::string& sink, 
        const std::string& path,
        pCValue config);

    // Stream control
    expected<bool> write(pCValue sinkData = VAL(), 
                        std::string dynamicPath = "") override;
    void stop() override;
    void close() override;

    // Stream information
    std::string getSink() override;
};

Core Classes

// RTSP Server Management
class RtspServer {
public:
    expected<void> startServer(const std::string& host, int port);
    expected<bool> createStream(const std::string& path, 
                               const std::string& pipeline,
                               int fps, bool monotonic);
    expected<void> removeStream(const std::string& path);
    expected<bool> write(const std::string& path, cbuffer frame);
    bool hasStream(const std::string& path);
    int streamCount() const;
};

// GStreamer Core Processing
class GStreamerWriterCore {
public:
    struct config {
        bool real_time = false;
        int sampling_rate = 0;
        int scale_width = 0;
        int scale_height = 0;
    };

    bool startWriter(const std::string& path, 
                    std::shared_ptr<iface::Buffer> frame,
                    const std::string& debugLevel = "0",
                    int fps = 30, bool monotonic = true);
    expected<bool> write(cbuffer const* frame);
    void close();
    void stop();
};

Lua API

The GStreamerWriter plugin is integrated through the Output plugin system and doesn't expose direct Lua bindings:

-- GStreamer outputs are created through the Output plugin
local output = api.factory.output.create(instance, "GStreamerOutput")

-- Add RTSP handler
local rtspHandler = output:addHandler(
    "rtsp-stream",
    "rtsp://0.0.0.0:8554/live",
    "output",
    {
        fps = 30,
        monotonic = true,
        pipeline = "( appsrc name=cvedia-rt ! videoconvert ! x264enc ! rtph264pay name=pay0 pt=96 )"
    }
)

-- Add file output handler
local fileHandler = output:addHandler(
    "mp4-output",
    "gstreamer:///recordings/video.mp4",
    "output",
    {
        fps = 25,
        pipeline = "appsrc name=cvedia-rt ! videoconvert ! x264enc ! mp4mux ! filesink location=/recordings/video.mp4"
    }
)

Note: GStreamerWriter operates as an output handler registered with URI schemes "rtsp://" and "gstreamer://".

Examples

Basic RTSP Streaming

-- Create output instance
local instance = api.thread.getCurrentInstance()
local output = api.factory.output.create(instance, "RTSPStreamer")

-- Configure basic RTSP stream
local rtspConfig = {
    fps = 30,
    monotonic = true
}

-- Add RTSP handler (uses default pipeline)
local stream = output:addHandler(
    "live-stream",
    "rtsp://0.0.0.0:8554/live",
    "output",
    rtspConfig
)

if stream then
    api.logging.LogInfo("RTSP stream available at: rtsp://localhost:8554/live")
else
    api.logging.LogError("Failed to create RTSP stream")
end

Multi-Quality RTSP Streaming

local output = api.factory.output.create(instance, "MultiQualityRTSP")

-- High quality stream
local hdConfig = {
    fps = 30,
    monotonic = true,
    pipeline = "( appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=medium bitrate=5000 ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 )"
}

local hdStream = output:addHandler(
    "hd-stream",
    "rtsp://0.0.0.0:8554/hd",
    "output",
    hdConfig
)

-- Mobile quality stream
local mobileConfig = {
    fps = 15,
    monotonic = true,
    pipeline = "( appsrc name=cvedia-rt ! videoscale ! video/x-raw,width=854,height=480 ! videoconvert ! x264enc speed-preset=fast bitrate=800 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
}

local mobileStream = output:addHandler(
    "mobile-stream",
    "rtsp://0.0.0.0:8555/mobile",
    "output", 
    mobileConfig
)

api.logging.LogInfo("HD RTSP: rtsp://localhost:8554/hd")
api.logging.LogInfo("Mobile RTSP: rtsp://localhost:8555/mobile")

File Recording

local output = api.factory.output.create(instance, "VideoRecorder")

-- MP4 recording with timestamp
local timestamp = os.date("%Y%m%d_%H%M%S")
local filename = string.format("/recordings/video_%s.mp4", timestamp)

local mp4Config = {
    fps = 25,
    monotonic = false,
    pipeline = string.format("appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=medium bitrate=3000 ! mp4mux ! filesink location=%s", filename)
}

local recorder = output:addHandler(
    "mp4-recorder",
    "gstreamer://" .. filename,
    "output",
    mp4Config
)

if recorder then
    api.logging.LogInfo("Recording to: " .. filename)

    -- Function to stop recording
    function stopRecording()
        output:removeHandler("mp4-recorder")
        api.logging.LogInfo("Recording stopped")
    end
else
    api.logging.LogError("Failed to start recording")
end

Hardware Accelerated Streaming

local output = api.factory.output.create(instance, "HWAccelStreamer")

-- Check for NVIDIA hardware encoding
local nvencConfig = {
    fps = 30,
    monotonic = true,
    pipeline = "( appsrc name=cvedia-rt ! videoconvert ! nvh264enc preset=low-latency-hq bitrate=4000 ! video/x-h264,profile=main ! rtph264pay name=pay0 pt=96 )"
}

local nvencStream = output:addHandler(
    "nvenc-stream",
    "rtsp://0.0.0.0:8554/nvenc",
    "output",
    nvencConfig
)

-- Fallback to software encoding if hardware fails
if not nvencStream then
    api.logging.LogWarning("NVENC not available, falling back to software encoding")

    local softwareConfig = {
        fps = 30,
        monotonic = true,
        pipeline = "( appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=ultrafast tune=zerolatency bitrate=3000 ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )"
    }

    local softwareStream = output:addHandler(
        "software-stream",
        "rtsp://0.0.0.0:8554/software",
        "output",
        softwareConfig
    )

    if softwareStream then
        api.logging.LogInfo("Software RTSP: rtsp://localhost:8554/software")
    end
else
    api.logging.LogInfo("Hardware RTSP: rtsp://localhost:8554/nvenc")
end

Multi-Format Recording

local output = api.factory.output.create(instance, "MultiFormatRecorder")

local timestamp = os.date("%Y%m%d_%H%M%S")

-- Multiple format configurations
local formats = {
    {
        name = "mp4-h264",
        uri = string.format("gstreamer:///recordings/video_%s.mp4", timestamp),
        pipeline = string.format("appsrc name=cvedia-rt ! videoconvert ! x264enc speed-preset=medium bitrate=3000 ! mp4mux ! filesink location=/recordings/video_%s.mp4", timestamp)
    },
    {
        name = "mkv-h265",
        uri = string.format("gstreamer:///recordings/video_%s.mkv", timestamp),
        pipeline = string.format("appsrc name=cvedia-rt ! videoconvert ! x265enc speed-preset=medium bitrate=2000 ! matroskamux ! filesink location=/recordings/video_%s.mkv", timestamp)
    },
    {
        name = "webm-vp9",
        uri = string.format("gstreamer:///recordings/video_%s.webm", timestamp),
        pipeline = string.format("appsrc name=cvedia-rt ! videoconvert ! vp9enc ! webmmux ! filesink location=/recordings/video_%s.webm", timestamp)
    }
}

-- Create all recorders
local activeRecorders = {}
for _, format in ipairs(formats) do
    local config = {
        fps = 25,
        monotonic = false,
        pipeline = format.pipeline
    }

    local recorder = output:addHandler(format.name, format.uri, "output", config)
    if recorder then
        table.insert(activeRecorders, format.name)
        api.logging.LogInfo("Started recording: " .. format.name)
    else
        api.logging.LogError("Failed to start: " .. format.name)
    end
end

-- Function to stop all recordings
function stopAllRecordings()
    for _, name in ipairs(activeRecorders) do
        output:removeHandler(name)
        api.logging.LogInfo("Stopped: " .. name)
    end
    activeRecorders = {}
end

Dynamic Pipeline Configuration

local output = api.factory.output.create(instance, "DynamicStreamer")

-- Function to create pipeline based on quality setting
function createPipeline(quality, resolution, bitrate)
    local width, height = resolution.width, resolution.height
    local preset = quality == "high" and "medium" or quality == "medium" and "fast" or "ultrafast"

    return string.format(
        "( appsrc name=cvedia-rt ! videoscale ! video/x-raw,width=%d,height=%d ! videoconvert ! x264enc speed-preset=%s bitrate=%d ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96 )",
        width, height, preset, bitrate
    )
end

-- Quality profiles
local profiles = {
    high = { resolution = {width = 1920, height = 1080}, bitrate = 5000 },
    medium = { resolution = {width = 1280, height = 720}, bitrate = 2500 },
    low = { resolution = {width = 854, height = 480}, bitrate = 1000 }
}

-- Function to switch quality dynamically
function switchQuality(qualityLevel)
    local profile = profiles[qualityLevel]
    if not profile then
        api.logging.LogError("Unknown quality level: " .. qualityLevel)
        return false
    end

    -- Remove existing stream
    output:removeHandler("dynamic-stream")

    -- Create new stream with different quality
    local config = {
        fps = 30,
        monotonic = true,
        pipeline = createPipeline(qualityLevel, profile.resolution, profile.bitrate)
    }

    local stream = output:addHandler(
        "dynamic-stream",
        "rtsp://0.0.0.0:8554/dynamic",
        "output",
        config
    )

    if stream then
        api.logging.LogInfo(string.format("Switched to %s quality (%dx%d, %d kbps)", 
            qualityLevel, profile.resolution.width, profile.resolution.height, profile.bitrate))
        return true
    else
        api.logging.LogError("Failed to switch quality")
        return false
    end
end

-- Start with medium quality
switchQuality("medium")

Best Practices

Pipeline Design

  1. Use appropriate presets:

    • ultrafast for real-time streaming (high CPU, lower quality)
    • fast for balanced performance
    • medium for better quality (higher CPU usage)

  2. Configure bitrate properly:

    • Match bitrate to network capabilities
    • Use rate control for consistent quality
    • Monitor encoding performance

  3. Choose appropriate profiles:

    • baseline for maximum compatibility
    • main for better compression
    • high for best quality (newer clients)

Performance Optimization

  1. Hardware Acceleration:

    • Use NVENC on NVIDIA GPUs: nvh264enc
    • Use VAAPI on Intel/AMD: vaapih264enc
    • Use Video Toolbox on macOS: vtenc_h264

  2. Memory Management:

    • Monitor buffer sizes in pipelines
    • Implement proper cleanup for stopped streams
    • Use appropriate queue sizes

  3. Network Optimization:

    • Configure MTU for network efficiency
    • Use appropriate RTP payload sizes
    • Monitor network congestion

RTSP Server Management

  1. Session Management:

    • Automatic session cleanup (2-second intervals)
    • Monitor concurrent client connections
    • Implement connection limits if needed

  2. Security Considerations:

    • Use authentication for sensitive streams
    • Implement IP-based access control
    • Consider RTSP over TLS (RTSPS)

  3. Monitoring:

    • Track active streams and clients
    • Monitor server resource usage
    • Log connection events

Troubleshooting

Stream Creation Issues

  1. "Failed to start RTSP server"

    • Check if port is already in use
    • Verify network interface availability
    • Ensure sufficient system resources
    • Check firewall settings

  2. "Pipeline parsing failed"

    • Validate GStreamer pipeline syntax
    • Check if required plugins are installed
    • Verify element names and properties
    • Test pipeline with gst-launch-1.0

  3. "No data on stream"

    • Verify input data format
    • Check sink data availability
    • Monitor appsrc buffer status
    • Validate frame dimensions and format

Encoding Issues

  1. "Hardware encoder not available"

    • Check hardware encoder installation
    • Verify driver compatibility
    • Fall back to software encoding
    • Test with gst-inspect-1.0

  2. "High CPU usage"

    • Use hardware acceleration when available
    • Adjust encoding presets (faster presets)
    • Reduce resolution or frame rate
    • Monitor system resources

  3. "Poor video quality"

    • Increase bitrate settings
    • Use slower encoding presets
    • Check input video quality
    • Adjust rate control settings

Network Issues

  1. "Clients cannot connect"

    • Verify RTSP URL accessibility
    • Check network firewall rules
    • Test with RTSP client tools
    • Monitor server logs

  2. "Streaming stutters or drops"

    • Check network bandwidth vs. bitrate
    • Monitor packet loss
    • Adjust buffer sizes
    • Use appropriate transport protocol

  3. "Session timeout errors"

    • Verify client keep-alive settings
    • Check session timeout configuration
    • Monitor session pool cleanup
    • Implement proper session management

File Output Issues

  1. "File not created"

    • Check file path permissions
    • Verify directory exists
    • Check disk space availability
    • Validate file format support

  2. "Recording stops unexpectedly"

    • Monitor disk space during recording
    • Check file system limits
    • Verify pipeline stability
    • Implement error handling

Integration Examples

VLC Media Player

# Connect to RTSP stream
vlc rtsp://localhost:8554/live

# With authentication
vlc rtsp://username:password@localhost:8554/secure

FFmpeg Client

# Stream to another RTSP server
ffmpeg -i rtsp://localhost:8554/live -c copy -f rtsp rtsp://remote-server/stream

# Convert to HLS
ffmpeg -i rtsp://localhost:8554/live -c copy -hls_time 4 -hls_playlist_type vod output.m3u8

# Record to file
ffmpeg -i rtsp://localhost:8554/live -c copy output.mp4

Python Client

import cv2

# OpenCV RTSP client
cap = cv2.VideoCapture('rtsp://localhost:8554/live')

while True:
    ret, frame = cap.read()
    if ret:
        cv2.imshow('RTSP Stream', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break

cap.release()
cv2.destroyAllWindows()

Web Browser Integration

<!DOCTYPE html>
<html>
<head>
    <title>RTSP Stream Viewer</title>
</head>
<body>
    <video id="video" controls autoplay width="800" height="600">
        <source src="rtsp://localhost:8554/live" type="application/x-rtsp">
        Your browser doesn't support RTSP directly.
    </video>

    <!-- Alternative: Convert RTSP to HLS for web browsers -->
    <script>
        // Note: Most browsers don't support RTSP directly
        // Consider using HLS plugin or WebRTC for browser streaming
        console.log('For web browsers, consider converting RTSP to HLS or WebRTC');
    </script>
</body>
</html>

See Also