Datasets Plugin

Description

The Datasets plugin collection provides unified interfaces for loading and processing various computer vision datasets within CVEDIA-RT. It supports popular annotation formats including COCO, CVAT, and folder-based organization, enabling seamless integration of training and evaluation data into computer vision workflows.

The plugin architecture implements a common DatasetHandler interface, allowing consistent access to different dataset formats through standardized APIs. This enables format-agnostic dataset processing, validation, and conversion capabilities across the platform.

Requirements

Hardware Requirements

• RAM: Minimum 4GB, 8GB+ recommended for large datasets
• Storage: SSD recommended for optimal I/O performance with large image datasets
• CPU: Multi-core processor recommended for batch processing operations

Software Requirements

• CVEDIA-RT Core: Latest compatible version
• JSON Libraries: For COCO format support
• XML Libraries: For CVAT format support (pugixml)
• File System: Support for large file operations and directory traversal

Supported Dataset Formats

• COCO: Microsoft Common Objects in Context JSON format
• CVAT Images: Computer Vision Annotation Tool XML format for images
• CVAT Videos: CVAT XML format with temporal annotations for videos
• Class Folders: Directory-based organization with automatic class discovery

Configuration

Basic COCO Dataset Configuration

{
  "dataset_type": "coco",
  "dataset_path": "/path/to/annotations.json",
  "image_root": "/path/to/images/",
  "validation": {
    "check_images": true,
    "verify_annotations": true
  }
}

CVAT Images Configuration

{
  "dataset_type": "cvat4images",
  "dataset_path": "/path/to/annotations.xml",
  "image_root": "/path/to/images/",
  "class_mapping": {
    "car": 0,
    "person": 1,
    "bicycle": 2
  }
}

Class Folders Configuration

{
  "dataset_type": "classfolders",
  "dataset_path": "/path/to/dataset/",
  "recursive": true,
  "supported_formats": [".jpg", ".png", ".jpeg", ".bmp"],
  "auto_class_discovery": true
}

Advanced Configuration

{
  "dataset_type": "coco",
  "dataset_path": "/path/to/annotations.json",
  "image_root": "/path/to/images/",
  "preprocessing": {
    "resize": {
      "width": 640,
      "height": 640,
      "maintain_aspect_ratio": true
    },
    "normalization": {
      "mean": [0.485, 0.456, 0.406],
      "std": [0.229, 0.224, 0.225]
    }
  },
  "filtering": {
    "min_area": 100,
    "max_objects_per_image": 50,
    "categories": ["person", "car", "bicycle"]
  },
  "augmentation": {
    "horizontal_flip": 0.5,
    "rotation_range": 15,
    "brightness_range": [0.8, 1.2]
  }
}

Configuration Parameters

Parameter	Type	Default	Description
dataset_type	string	-	Dataset format type (coco, cvat4images, cvat4videos, classfolders)
dataset_path	string	-	Path to dataset file or directory
image_root	string	-	Root directory for image files (relative paths in annotations)
validation.check_images	boolean	true	Verify image files exist and are readable
validation.verify_annotations	boolean	true	Validate annotation format and consistency
class_mapping	object	{}	Custom class name to ID mapping
recursive	boolean	false	Enable recursive directory traversal for class folders
supported_formats	array	[".jpg",".png"]	Supported image file extensions
preprocessing.resize	object	-	Image resizing configuration
filtering.min_area	number	0	Minimum bounding box area filter
filtering.categories	array	-	Include only specified categories
augmentation	object	-	Data augmentation parameters
batch_size	number	32	Batch size for dataset iteration
shuffle	boolean	false	Shuffle dataset order
limit	number	-1	Maximum number of samples to load (-1 for all)

Supported Dataset Formats

COCO (Common Objects in Context)

• Format: JSON annotations with image metadata
• Features: Object detection, instance segmentation, keypoint detection
• Structure: Images, annotations, categories with hierarchical relationships
• Use Cases: Object detection training, evaluation benchmarks

Example COCO structure:

{
  "images": [
    {
      "id": 1,
      "file_name": "image001.jpg",
      "width": 640,
      "height": 480
    }
  ],
  "annotations": [
    {
      "id": 1,
      "image_id": 1,
      "category_id": 1,
      "bbox": [100, 100, 200, 150],
      "area": 30000,
      "segmentation": [[...]],
      "iscrowd": 0
    }
  ],
  "categories": [
    {
      "id": 1,
      "name": "person",
      "supercategory": "person"
    }
  ]
}

CVAT (Computer Vision Annotation Tool)

CVAT for Images: - Format: XML annotations with image references - Features: Bounding boxes, polygons, points, attributes - Structure: Task-based organization with tracks and shapes

CVAT for Videos: - Format: XML with temporal annotations - Features: Object tracking, interpolation, frame-based annotations - Structure: Temporal tracks with keyframe interpolation

Class Folders

• Format: Directory structure with class names as folder names
• Features: Automatic class discovery, recursive traversal
• Structure: /dataset/class1/image1.jpg, /dataset/class2/image2.jpg
• Use Cases: Image classification, simple dataset organization

API Reference

C++ API

The Datasets plugin implements the iface::DatasetHandler interface:

namespace cvedia {
  namespace rt {
    namespace iface {
      class DatasetHandler {
      public:
        virtual expected<std::vector<pCValue>> readDataset(int limit = -1) = 0;
        virtual expected<void> writeDataset(std::vector<pCValue> data) = 0;
      };
    }
  }
}

Core Classes

COCO Dataset Handler

class Coco final : public iface::DatasetHandler {
public:
  static std::unique_ptr<iface::DatasetHandler> create(std::string const& datasetPath);
  expected<std::vector<pCValue>> readDataset(int limit = -1) override;
  expected<void> writeDataset(std::vector<pCValue> data) override;
};

CVAT Images Handler

class CVAT4Images final : public iface::DatasetHandler {
public:
  static std::unique_ptr<iface::DatasetHandler> create(std::string const& datasetPath);
  expected<std::vector<pCValue>> readDataset(int limit = -1) override;
  expected<void> writeDataset(std::vector<pCValue> data) override;
};

Class Folders Handler

class ClassFolders final : public iface::DatasetHandler {
public:
  static std::unique_ptr<iface::DatasetHandler> create(std::string const& datasetPath);
  expected<std::vector<pCValue>> readDataset(int limit = -1) override;
  expected<void> writeDataset(std::vector<pCValue> data) override;
};

Factory Registration

Dataset handlers are registered with the factory system:

// Registration
api::factory::DatasetHandler::registerImplementation("coco", &Coco::create);
api::factory::DatasetHandler::registerImplementation("cvat4images", &CVAT4Images::create);
api::factory::DatasetHandler::registerImplementation("classfolders", &ClassFolders::create);

// Usage
auto handler = api::factory::DatasetHandler::create("coco", "/path/to/dataset.json");

Lua API

-- Create dataset handler using factory
local instance = api.thread.getCurrentInstance()
local dataset = api.factory.datasethandler.create(instance, "COCODataset")
dataset:setConfigValue("dataset_type", "coco")
dataset:setConfigValue("dataset_path", "/path/to/annotations.json")

-- Read dataset with limit
local samples = dataset:readDataset(100)

-- Process samples
for i, sample in ipairs(samples) do
  local image_path = sample:getNode("image_path"):asString()
  local annotations = sample:getNode("annotations"):asVector()

  -- Process annotations
  for j, annotation in ipairs(annotations) do
    local bbox = annotation:getNode("bbox"):asVector()
    local category = annotation:getNode("category_id"):asInt()
    -- Handle annotation data
  end
end

Examples

Loading a COCO Dataset

#include "api/factory.h"
#include "interface/datasethandler.h"

using namespace cvedia::rt;

// Create COCO dataset handler
auto handler = api::factory::DatasetHandler::create("coco", "/path/to/coco/annotations.json");
if (!handler) {
    // Handle error
    return;
}

// Read all samples
auto samples = handler.value()->readDataset();
if (!samples) {
    // Handle error
    return;
}

// Process samples
for (auto& sample : samples.value()) {
    auto imagePath = sample->getNode("image_path");
    auto annotations = sample->getNode("annotations");

    // Process image and annotations
    processImage(imagePath->asString(), annotations->asVector());
}

Processing CVAT Image Annotations

-- Load CVAT dataset
local instance = api.thread.getCurrentInstance()
local dataset = api.factory.datasethandler.create(instance, "CVATDataset")
dataset:setConfigValue("dataset_type", "cvat4images")
dataset:setConfigValue("dataset_path", "/path/to/annotations.xml")
local samples = dataset:readDataset()

-- Statistics collection
local class_counts = {}
local total_objects = 0

for i, sample in ipairs(samples) do
  local annotations = sample:getNode("annotations"):asVector()

  for j, annotation in ipairs(annotations) do
    local label = annotation:getNode("label"):asString()
    class_counts[label] = (class_counts[label] or 0) + 1
    total_objects = total_objects + 1
  end
end

-- Log statistics
api.logging.LogInfo("Dataset Statistics:")
api.logging.LogInfo("Total objects: " .. total_objects)
for class, count in pairs(class_counts) do
  api.logging.LogInfo(string.format("%s: %d objects", class, count))
end

Class Folders Dataset Processing

// Create class folders handler
auto handler = api::factory::DatasetHandler::create("classfolders", "/path/to/dataset/");

// Read samples in batches
int batchSize = 32;
int offset = 0;
std::vector<pCValue> allSamples;

while (true) {
    auto batch = handler.value()->readDataset(batchSize);
    if (!batch || batch.value().empty()) {
        break;
    }

    // Process batch
    for (auto& sample : batch.value()) {
        auto imagePath = sample->getNode("image_path")->asString();
        auto classLabel = sample->getNode("class_id")->asInt();
        auto className = sample->getNode("class_name")->asString();

        // Add to training data
        trainingData.push_back({imagePath, classLabel});
    }

    offset += batchSize;
}

Dataset Format Conversion

-- Convert CVAT to COCO format
local instance = api.thread.getCurrentInstance()

-- Create CVAT reader
local cvat_dataset = api.factory.datasethandler.create(instance, "CVATReader")
cvat_dataset:setConfigValue("dataset_type", "cvat4images")
cvat_dataset:setConfigValue("dataset_path", "/path/to/cvat.xml")

-- Create COCO writer
local coco_dataset = api.factory.datasethandler.create(instance, "COCOWriter")
coco_dataset:setConfigValue("dataset_type", "coco")
coco_dataset:setConfigValue("dataset_path", "/path/to/output.json")

-- Read CVAT data
local samples = cvat_dataset:readDataset()

-- Convert format and write
local converted_samples = {}
for i, sample in ipairs(samples) do
  -- Convert CVAT format to COCO format
  local coco_sample = convertCvatToCoco(sample)
  table.insert(converted_samples, coco_sample)
end

-- Write converted data
api.logging.LogInfo("Converting " .. #converted_samples .. " samples from CVAT to COCO format")
coco_dataset:writeDataset(converted_samples)
api.logging.LogInfo("Dataset conversion completed")

Dataset Validation and Statistics

#include <unordered_map>
#include <iostream>

void validateAndAnalyzeDataset(const std::string& datasetType, const std::string& datasetPath) {
    auto handler = api::factory::DatasetHandler::create(datasetType, datasetPath);
    if (!handler) {
        std::cerr << "Failed to create dataset handler" << std::endl;
        return;
    }

    auto samples = handler.value()->readDataset();
    if (!samples) {
        std::cerr << "Failed to read dataset" << std::endl;
        return;
    }

    // Statistics
    std::unordered_map<std::string, int> classCounts;
    std::unordered_map<std::string, std::vector<int>> imageSizes;
    int totalObjects = 0;
    int validImages = 0;

    for (const auto& sample : samples.value()) {
        // Check image existence
        auto imagePath = sample->getNode("image_path")->asString();
        if (fs::exists(imagePath)) {
            validImages++;

            // Collect image dimensions
            auto width = sample->getNode("width")->asInt();
            auto height = sample->getNode("height")->asInt();
            imageSizes["widths"].push_back(width);
            imageSizes["heights"].push_back(height);
        }

        // Process annotations
        if (auto annotations = sample->getNode("annotations")) {
            for (const auto& annotation : annotations->asVector()) {
                auto className = annotation->getNode("class_name")->asString();
                classCounts[className]++;
                totalObjects++;
            }
        }
    }

    // Print statistics
    std::cout << "Dataset Analysis Results:" << std::endl;
    std::cout << "Total samples: " << samples.value().size() << std::endl;
    std::cout << "Valid images: " << validImages << std::endl;
    std::cout << "Total objects: " << totalObjects << std::endl;

    std::cout << "\nClass distribution:" << std::endl;
    for (const auto& [className, count] : classCounts) {
        std::cout << "  " << className << ": " << count << std::endl;
    }
}

Performance Considerations

Memory Management

• Batch Processing: Use batched loading for large datasets to manage memory usage
• Lazy Loading: Images are loaded on-demand to minimize memory footprint
• Cache Management: Implement appropriate caching strategies for frequently accessed data

I/O Optimization

• SSD Storage: Store datasets on SSD for optimal read performance
• Parallel Loading: Utilize multi-threading for concurrent image loading
• Network Storage: Consider data locality when using network-attached storage

Large Dataset Handling

// Efficient large dataset processing
void processLargeDataset(const std::string& datasetPath) {
    auto handler = api::factory::DatasetHandler::create("coco", datasetPath);

    const int batchSize = 64;  // Adjust based on available memory
    int processedCount = 0;

    while (true) {
        auto batch = handler.value()->readDataset(batchSize);
        if (!batch || batch.value().empty()) {
            break;
        }

        // Process batch in parallel
        #pragma omp parallel for
        for (size_t i = 0; i < batch.value().size(); ++i) {
            processSample(batch.value()[i]);
        }

        processedCount += batch.value().size();
        std::cout << "Processed " << processedCount << " samples" << std::endl;
    }
}

Troubleshooting

Common Issues

Dataset Not Found

Error: Dataset file not found at specified path

- Verify the dataset path exists and is accessible - Check file permissions for read access - Ensure the path uses correct directory separators

Invalid Format

Error: Invalid dataset format or corrupted annotations

- Validate JSON/XML syntax using external tools - Check annotation schema compliance - Verify image paths in annotations are correct

Memory Issues

Error: Out of memory during dataset loading

- Reduce batch size in configuration - Use streaming/lazy loading approaches - Monitor system memory usage during processing

Performance Issues

Slow dataset loading and processing

- Move dataset to SSD storage - Increase batch size (within memory limits) - Enable parallel processing where possible - Use appropriate image formats (avoid excessive compression)

Validation Commands

# Validate COCO dataset format
python -c "import json; json.load(open('annotations.json'))"

# Check CVAT XML format
xmllint --noout annotations.xml

# Verify class folders structure
find /path/to/dataset -name "*.jpg" | wc -l

Debug Configuration

{
  "dataset_type": "coco",
  "dataset_path": "/path/to/annotations.json",
  "debug": {
    "log_level": "debug",
    "validate_on_load": true,
    "show_statistics": true,
    "performance_metrics": true
  }
}

See Also

• Plugin Overview
• Data Plugins Category
• COCO Evaluation Plugin
• Image Reader Plugin
• Training Pipeline Documentation