FFmpegReader Plugin ¶

Description ¶

FFmpegReader is a comprehensive video input plugin that leverages the FFmpeg multimedia framework to provide support for a wide variety of video formats and codecs. It serves as the primary video reading solution for CVEDIA-RT, offering robust video decoding, frame seeking, and playback control capabilities.

The plugin provides universal video format support through FFmpeg integration, hardware-accelerated decoding when available, frame-accurate seeking, real-time playback control, automatic stream analysis, and built-in format conversion capabilities.

Requirements ¶

Software Dependencies ¶

FFmpeg Libraries (complete multimedia framework):
libavcodec - Video/audio codec libraries
libavformat - Container format libraries
libavutil - Utility functions and data structures
libswscale - Image scaling and color conversion
CVEDIA-RT Core - Platform core functionality

Platform Requirements ¶

Windows: FFmpeg Windows packages
Linux: FFmpeg development packages (/opt/ffmpeg or system packages)

Hardware Support ¶

CPU-based decoding (all platforms)
Hardware-accelerated decoding when supported decoders are available
GPU acceleration support varies by platform and available drivers

Configuration ¶

Basic Configuration ¶

{
  "start_frame": 0,
  "end_frame": 0,
  "real_time": true,
  "file_sampling_rate": 0
}

Advanced Configuration ¶

{
  "start_frame": 100,
  "end_frame": 5000,
  "real_time": false,
  "file_sampling_rate": 15
}

Configuration Parameters ¶

Parameter	Type	Default	Description
`start_frame`	integer	`0`	Starting frame for disk-based video sources
`end_frame`	integer	`0`	End frame for disk-based video sources (0 = end of video)
`real_time`	boolean	`true`	Slow down disk-based video reading to simulate actual FPS
`file_sampling_rate`	integer	`0`	If set and real_time is false, sets FPS to this value instead of video default

Supported Formats and Codecs ¶

Video Codecs ¶

H.264/AVC - Most common modern codec
H.265/HEVC - High Efficiency Video Coding
MJPEG - Motion JPEG for high-quality applications
MPEG-2 - Legacy broadcast and DVD format
MPEG-4 - Classic internet video format
VP8 - Google WebM codec
VP9 - Advanced WebM codec
VC-1 - Microsoft codec

Container Formats ¶

MP4 - Most common container format
AVI - Audio Video Interleave
MOV - QuickTime movie format
MKV/WebM - Matroska container
FLV - Flash Video format
Raw video streams
Various broadcast and professional formats

API Reference ¶

C++ API - FFmpegReaderCore Class ¶

Frame Operations ¶

expected<cvec> readFrame();                    // Read and decode next video frame
void setCurrentFrame(int frameNum);           // Seek to specific frame number
int getCurrentFrame() const;                  // Get current frame position
int getFrameCount() const;                    // Get total number of frames

Playback Control ¶

float getFps(iface::FPSType fpsType) const;  // Get frame rate information
double getCurrentTimestamp() const;           // Get current playback timestamp
void setRepeatMode(InputRepeatMode mode);     // Configure playback looping
InputRepeatMode getRepeatMode() const;        // Get current repeat mode

Source Management ¶

expected<void> openUri(const std::string& uri);  // Open video source
void close();                                     // Close current source
std::string getSourceURI();                      // Get current source URI
std::string getSourceDesc();                     // Get source description

Stream Information ¶

bool canRead() const;                         // Check if source can be read
bool canSeek() const;                         // Check if source supports seeking
bool isEnded() const;                         // Check if playback ended
bool fileReachedEnd() const;                  // Check if file reached end
std::optional<Codec> getSourceCodec();       // Get detected codec

Format Control ¶

void setSourceDefaultFormat(InputFormat fmt); // Set default input format
InputFormat getSourceDefaultFormat();         // Get current input format

Configuration Structure ¶

struct config {
    int start_frame = 0;          // Starting frame for playback
    int end_frame = 0;            // End frame (0 = end of video)
    bool real_time = true;        // Real-time playback mode
    int file_sampling_rate = 0;   // Custom sampling rate
};

Statistics Structure ¶

struct stats {
    int input_width = 0;          // Video width in pixels
    int input_height = 0;         // Video height in pixels
    float input_fps = 0;          // Original video frame rate
    int target_fps = 0;           // Target playback frame rate
    int current_frame = 0;        // Current frame number
    int total_frames = 0;         // Total frames in video
    std::string current_uri = ""; // Current source URI
};

Examples ¶

Basic File Reading ¶

// Create FFmpeg reader instance
FFmpegReaderCore reader;

// Open video file
auto result = reader.openUri("/path/to/video.mp4");
if (!result) {
    // Handle error
    return;
}

// Read frames in sequence
while (reader.canRead() && !reader.isEnded()) {
    auto frame = reader.readFrame();
    if (frame) {
        // Process frame data
        processFrame(*frame);
    }
}

// Clean up
reader.close();

Frame-Accurate Seeking ¶

FFmpegReaderCore reader;
reader.openUri("/path/to/video.mp4");

// Get video information
int totalFrames = reader.getFrameCount();
float fps = reader.getFps();

// Seek to specific frame
reader.setCurrentFrame(1000);

// Read frame at that position
auto frame = reader.readFrame();

Custom Playback Configuration ¶

FFmpegReaderCore reader;

// Configure playback settings
reader.pluginConf.start_frame = 100;
reader.pluginConf.end_frame = 2000;
reader.pluginConf.real_time = false;
reader.pluginConf.file_sampling_rate = 30;

// Open and process video segment
reader.openUri("/path/to/video.mp4");

Hardware-Accelerated Streaming ¶

FFmpegReaderCore reader;

// Open network stream
auto result = reader.openUri("rtsp://camera.example.com:554/stream");

if (result) {
    // Check codec for hardware acceleration compatibility
    auto codec = reader.getSourceCodec();
    if (codec && (*codec == Codec::H264 || *codec == Codec::HEVC)) {
        // Hardware acceleration available for H.264/H.265
        std::cout << "Hardware decoding available\n";
    }

    // Stream processing loop
    while (reader.canRead()) {
        auto frame = reader.readFrame();
        if (frame) {
            processStreamFrame(*frame);
        }
    }
}

Performance Considerations ¶

Hardware Acceleration ¶

Hardware acceleration is automatically utilized when available decoders are detected
Supported hardware varies by platform:
Windows: DirectX Video Acceleration (DXVA), NVDEC, Intel Quick Sync
Linux: VAAPI, NVDEC, Intel Quick Sync
Hardware acceleration provides significant performance improvements for H.264 and H.265 content

Memory Management ¶

The plugin uses efficient memory management with automatic cleanup
Frame buffers are reused to minimize memory allocation overhead
Large video files are streamed rather than loaded entirely into memory

Threading Considerations ¶

The plugin is thread-safe with internal mutex protection
Multiple reader instances can be used simultaneously
Frame processing should be done on separate threads for optimal performance

Troubleshooting ¶

Common Issues ¶

Video fails to open - Verify FFmpeg libraries are properly installed - Check that the video file path is accessible - Ensure the video format is supported by your FFmpeg build

Poor playback performance - Enable hardware acceleration if available - Check if the video resolution exceeds system capabilities - Consider reducing frame rate or resolution for real-time processing

Seeking inaccuracies - Some video formats may not support frame-accurate seeking - Use keyframe-based seeking for better compatibility - Consider re-encoding problematic videos with regular keyframes

Memory usage issues - Monitor memory usage with large video files - Implement frame buffering limits for streaming applications - Use appropriate end_frame settings to limit processing scope

Error Codes ¶

RTErrc::FileCannotBeOpened - File access or format issues
RTErrc::UnsupportedCodec - Codec not supported by current build
AVERROR(EAGAIN) - Temporary read failure, retry recommended

FFmpeg Version Compatibility ¶

The plugin supports multiple FFmpeg versions: - FFmpeg 5.x and earlier (legacy support) - FFmpeg 6.x and later (recommended)

Version-specific functionality is handled automatically through compile-time detection.