HLS Plugin

Description

The HLS (HTTP Live Streaming) plugin provides HTTP Live Streaming capabilities for CVEDIA-RT, enabling real-time video streaming over HTTP networks. It implements the HLS protocol with adaptive bitrate streaming, automatic segmentation, and M3U8 playlist generation for live video distribution to web browsers, mobile devices, and media players.

Key Features

• HTTP Live Streaming: Standards-compliant HLS implementation
• Automatic Segmentation: Real-time video segmentation into .ts files
• M3U8 Playlist Generation: Dynamic playlist creation and management
• Web Server Integration: Built-in HTTP server for stream delivery
• Configurable Segment Duration: Adjustable segment length for latency optimization
• Memory-based Streaming: In-memory segment storage for low-latency delivery
• Multi-stream Support: Handle multiple concurrent HLS streams

When to Use

• Live video streaming to web browsers
• Mobile device video streaming
• Real-time surveillance video distribution
• Video analytics result streaming with overlays
• Multi-client video broadcasting
• Low-latency video delivery over HTTP

Requirements

Software Dependencies

• FFmpeg libraries (libavformat, libavcodec)
• OpenCV for frame processing
• HTTP server capability (built-in webserver)
• Network connectivity for client access

Hardware Requirements

• Sufficient CPU for H.264 encoding
• Adequate memory for segment buffering
• Network bandwidth for concurrent clients
• Storage space for temporary segments (if file-based)

Configuration

Basic Configuration

{
  "output": {
    "handlers": {
      "hls-stream": {
        "uri": "hls://live-stream",
        "sink": "output",
        "enabled": true,
        "segment_duration": 2
      }
    }
  }
}

Advanced Configuration

{
  "output": {
    "handlers": {
      "primary-hls": {
        "uri": "hls://main-stream",
        "sink": "video",
        "enabled": true,
        "segment_duration": 2,
        "max_segments": 10,
        "codec": "h264",
        "bitrate": 2000000,
        "fps": 25,
        "resolution": {
          "width": 1920,
          "height": 1080
        }
      },
      "mobile-hls": {
        "uri": "hls://mobile-stream", 
        "sink": "video",
        "enabled": true,
        "segment_duration": 4,
        "max_segments": 5,
        "codec": "h264",
        "bitrate": 800000,
        "fps": 15,
        "resolution": {
          "width": 1280,
          "height": 720
        }
      }
    },
    "webserver": {
      "port": 8080,
      "cors_enabled": true,
      "cors_origins": ["*"],
      "max_connections": 100
    }
  }
}

Configuration Schema

Parameter	Type	Default	Description
uri	string	required	HLS stream identifier (hls://stream-name)
sink	string	"output"	Data sink to connect to
enabled	boolean	true	Enable/disable stream
segment_duration	integer	2	Segment duration in seconds
max_segments	integer	10	Maximum segments in playlist
codec	string	"h264"	Video codec (h264, hevc)
bitrate	integer	2000000	Target bitrate in bits/sec
fps	integer	25	Target frames per second
resolution.width	integer	auto	Video width in pixels
resolution.height	integer	auto	Video height in pixels

Stream Access

Stream URLs

HLS streams are automatically served through the built-in web server:

• M3U8 Playlist: http://localhost:8080/hls/{stream-name}.m3u8
• Video Segments: http://localhost:8080/hls/{stream-name}/{segment-id}.ts

Where {stream-name} corresponds to the instance ID or configured stream name.

Client Access

<!-- HTML5 Video Player -->
<video controls>
  <source src="http://localhost:8080/hls/stream-name.m3u8" type="application/vnd.apple.mpegurl">
  Your browser does not support HLS.
</source>

<!-- With HLS.js for broader browser support -->
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<video id="video" controls></video>
<script>
  const video = document.getElementById('video');
  if (Hls.isSupported()) {
    const hls = new Hls();
    hls.loadSource('http://localhost:8080/hls/stream-name.m3u8');
    hls.attachMedia(video);
  }
</script>

API Reference

C++ API

The HLS plugin implements the iface::OutputHandler interface:

class HLSOutput : public iface::OutputHandler {
public:
    // Constructor
    HLSOutput(const std::string& moduleName, 
              const std::string& schema, 
              const std::string& sink, 
              const std::string& path, 
              pCValue config);

    // Factory method
    static std::shared_ptr<iface::OutputHandler> create(
        const std::string& moduleName,
        const std::string& schema, 
        const std::string& sink, 
        const std::string& path, 
        pCValue config);

    // Stream control
    expected<bool> write(pCValue sinkData = VAL(), 
                        std::string dynamicPath = "") override;
    void stop() override;
    void close() override;

    // Stream information
    std::string getSink() override;
};

Core HLS Classes

class HlsCore {
public:
    struct config {
        int segmentDuration = 2;  // Segment duration in seconds
    };

    // Stream management
    expected<bool> write(cbuffer frame, const std::string& feedName);
    void close(const std::string& feedName);
};

class HlsStream {
public:
    // Stream control
    HlsStream(const std::string& name, int segmentDuration = 2);
    void updateFrame(cbuffer frame);

    // Stream information
    std::string getName() const;
    uint64_t getSegmentIndex() const;
    std::string getM3u8() const;

    // Segment access
    expected<std::shared_ptr<Segment>> getVideoSegment(const std::string& key) const;
};

Lua API

The HLS plugin is integrated through the Output plugin system and doesn't expose direct Lua bindings. It's configured and used through the Output plugin:

-- HLS streams are created through the Output plugin
local output = api.factory.output.create(instance, "HLSStreamer")

-- Add HLS handler through Output plugin
local hlsHandler = output:addHandler(
    "hls-stream",
    "hls://live-feed",
    "video",
    {
        segment_duration = 2,
        max_segments = 10
    }
)

Note: HLS plugin operates as an output handler registered with the URI scheme "hls://".

Examples

Basic Live Streaming

-- Create output instance
local instance = api.thread.getCurrentInstance()
local output = api.factory.output.create(instance, "LiveStreamer")

-- Configure HLS streaming
local hlsConfig = {
    segment_duration = 2,
    max_segments = 8,
    fps = 25
}

-- Add HLS handler
local stream = output:addHandler(
    "live-stream",
    "hls://main-feed",
    "output",
    hlsConfig
)

if stream then
    print("HLS stream available at: http://localhost:8080/hls/main-feed.m3u8")
else
    print("Failed to create HLS stream")
end

Multi-Quality Streaming

local output = api.factory.output.create(instance, "AdaptiveStreamer")

-- High quality stream
local hdConfig = {
    segment_duration = 2,
    bitrate = 3000000,
    resolution = { width = 1920, height = 1080 },
    fps = 30
}

local hdStream = output:addHandler(
    "hd-stream",
    "hls://hd-feed",
    "output",
    hdConfig
)

-- Mobile quality stream  
local mobileConfig = {
    segment_duration = 4,
    bitrate = 800000,
    resolution = { width = 1280, height = 720 },
    fps = 15
}

local mobileStream = output:addHandler(
    "mobile-stream", 
    "hls://mobile-feed",
    "output",
    mobileConfig
)

-- Print stream URLs
if hdStream then
    print("HD Stream: http://localhost:8080/hls/hd-feed.m3u8")
end

if mobileStream then
    print("Mobile Stream: http://localhost:8080/hls/mobile-feed.m3u8")
end

Analytics Overlay Streaming

-- Create output for analytics results with overlays
local output = api.factory.output.create(instance, "AnalyticsStreamer")

-- Configure for analytics streaming
local analyticsConfig = {
    segment_duration = 1,  -- Low latency for real-time analytics
    max_segments = 5,
    fps = 15,
    include_overlays = true
}

local analyticsStream = output:addHandler(
    "analytics-stream",
    "hls://analytics-feed", 
    "rendered",  -- Use rendered sink with overlays
    analyticsConfig
)

-- Function to toggle overlay visibility
function toggleOverlays(enabled)
    local config = output:getConfig()
    config.handlers["analytics-stream"].include_overlays = enabled
    output:saveConfig(config)
end

print("Analytics stream: http://localhost:8080/hls/analytics-feed.m3u8")

Stream Health Monitoring

local output = api.factory.output.create(instance, "MonitoredStreamer")

-- Create stream with monitoring
local streamConfig = {
    segment_duration = 2,
    max_segments = 10,
    health_check_interval = 30  -- seconds
}

local stream = output:addHandler(
    "monitored-stream",
    "hls://health-monitored",
    "output", 
    streamConfig
)

-- Monitor stream health
function checkStreamHealth()
    -- Check if stream is still active
    local config = output:getConfig()
    local streamName = "monitored-stream"

    if config.handlers[streamName] then
        print("Stream is active: " .. streamName)
        return true
    else
        print("Stream is inactive: " .. streamName)
        return false
    end
end

-- Health check should be implemented in your main processing loop
-- Example: Check every N frames or use a frame counter
-- No built-in setInterval exists in the Lua API

Dynamic Stream Management

local output = api.factory.output.create(instance, "DynamicStreamer")

-- Function to create stream dynamically
function createStream(streamName, quality)
    local configs = {
        high = {
            segment_duration = 2,
            bitrate = 4000000,
            resolution = { width = 1920, height = 1080 },
            fps = 30
        },
        medium = {
            segment_duration = 3,
            bitrate = 2000000,
            resolution = { width = 1280, height = 720 },
            fps = 25
        },
        low = {
            segment_duration = 4,
            bitrate = 800000,
            resolution = { width = 854, height = 480 },
            fps = 15
        }
    }

    local config = configs[quality] or configs.medium

    local stream = output:addHandler(
        streamName,
        "hls://" .. streamName,
        "output",
        config
    )

    if stream then
        local url = string.format("http://localhost:8080/hls/%s.m3u8", streamName)
        print(string.format("Created %s quality stream: %s", quality, url))
        return url
    end

    return nil
end

-- Function to remove stream
function removeStream(streamName)
    local success = output:removeHandler(streamName)
    if success then
        print("Removed stream: " .. streamName)
    else
        print("Failed to remove stream: " .. streamName)
    end
    return success
end

-- Dynamic stream creation
local hdUrl = createStream("camera-01-hd", "high")
local mobileUrl = createStream("camera-01-mobile", "low")

-- Later, remove streams
-- removeStream("camera-01-hd")

Best Practices

Stream Configuration

1. Segment Duration:

   • Use 2-6 seconds for good latency/quality balance
   • Shorter segments = lower latency, higher overhead
   • Longer segments = higher latency, better compression

2. Playlist Length:

   • Keep 3-10 segments in playlist
   • More segments = better seeking, higher memory usage
   • Fewer segments = lower memory, limited seeking

3. Bitrate Selection:

   • Match bitrate to content complexity and network capacity
   • Use adaptive streaming for varied client capabilities
   • Monitor bandwidth usage vs. quality trade-offs

Performance Optimization

1. Memory Management:

   • Monitor segment buffer usage
   • Implement proper cleanup for stopped streams
   • Use appropriate max_segments limits

2. CPU Usage:

   • Balance encoding quality vs. CPU load
   • Consider hardware acceleration when available
   • Monitor frame processing rates

3. Network Optimization:

   • Use CDN for large-scale distribution
   • Implement client-side buffering
   • Monitor concurrent connection limits

Security Considerations

1. Access Control:

   • Implement authentication for sensitive streams
   • Use HTTPS for secure transmission
   • Restrict access by IP or user credentials

2. Content Protection:

   • Consider HLS encryption for sensitive content
   • Implement token-based access control
   • Monitor for unauthorized access attempts

Troubleshooting

Stream Creation Issues

1. "Failed to create HLS stream"

   • Check if webserver port is available
   • Verify FFmpeg libraries are installed
   • Ensure sufficient system resources
   • Check for conflicting stream names

2. "Segment generation failed"

   • Verify video codec support
   • Check input frame format compatibility
   • Monitor CPU usage during encoding
   • Validate segment duration settings

Playback Issues

1. "Stream not loading in browser"

   • Verify M3U8 URL accessibility
   • Check CORS settings for cross-origin requests
   • Ensure HLS.js is loaded for unsupported browsers
   • Validate playlist format

2. "Stuttering or buffering"

   • Check network bandwidth vs. bitrate
   • Reduce segment duration for lower latency
   • Monitor server CPU usage
   • Verify client-side buffer settings

3. "Old content playing"

   • Check segment cleanup process
   • Verify playlist updates
   • Clear browser cache
   • Check segment naming conflicts

Performance Issues

1. "High CPU usage"

   • Reduce encoding quality settings
   • Lower frame rate or resolution
   • Check for multiple concurrent encodings
   • Consider hardware acceleration

2. "Memory usage growing"

   • Verify segment cleanup on stream stop
   • Check max_segments configuration
   • Monitor for memory leaks
   • Implement periodic garbage collection

3. "Network congestion"

   • Reduce bitrate or resolution
   • Implement client-side adaptive streaming
   • Use content delivery network (CDN)
   • Monitor concurrent client connections

Integration Examples

Web Player Integration

<!DOCTYPE html>
<html>
<head>
    <title>CVEDIA-RT HLS Stream</title>
    <script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
</head>
<body>
    <video id="video" controls width="800" height="600"></video>

    <script>
        const video = document.getElementById('video');
        const streamUrl = 'http://localhost:8080/hls/main-feed.m3u8';

        if (Hls.isSupported()) {
            const hls = new Hls({
                debug: false,
                enableWorker: true,
                lowLatencyMode: true,
                backBufferLength: 90
            });

            hls.loadSource(streamUrl);
            hls.attachMedia(video);

            hls.on(Hls.Events.MANIFEST_PARSED, function() {
                console.log('Stream loaded successfully');
                video.play();
            });

            hls.on(Hls.Events.ERROR, function(event, data) {
                console.error('HLS Error:', data);
            });
        } else if (video.canPlayType('application/vnd.apple.mpegurl')) {
            // Native HLS support (Safari)
            video.src = streamUrl;
        } else {
            console.error('HLS not supported');
        }
    </script>
</body>
</html>

Mobile App Integration

// React Native example
import Video from 'react-native-video';

const HLSPlayer = () => {
  return (
    <Video
      source={{ uri: 'http://192.168.1.100:8080/hls/mobile-feed.m3u8' }}
      style={{ width: '100%', height: 200 }}
      controls={true}
      resizeMode="contain"
      onError={(error) => console.log('Video Error:', error)}
      onLoad={() => console.log('Video loaded')}
    />
  );
};

RTMP to HLS Bridge

-- Bridge RTMP input to HLS output
local input = api.factory.gstreamerreader.create(instance, "RTMPInput")
local output = api.factory.output.create(instance, "HLSOutput")

-- Configure RTMP input
input:setConfig({
    uri = "rtmp://source.example.com/live/stream",
    format = "h264"
})

-- Configure HLS output
local hlsStream = output:addHandler(
    "rtmp-bridge",
    "hls://rtmp-bridge",
    "video",
    {
        segment_duration = 3,
        bitrate = 2000000
    }
)

-- Start processing
input:start()
print("RTMP to HLS bridge: http://localhost:8080/hls/rtmp-bridge.m3u8")

See Also

• Output Plugins Overview
• GStreamerWriter Plugin - Advanced video output
• Output Plugin - Main output system
• Input Plugins - Video input sources
• Webserver Configuration
• FFmpeg Integration Guide