REAL WORK

Real work, not staged demos

Operators record the tasks they already perform — kitchens, workshops, warehouses, clinics, streets. Task taxonomies and scenario briefs are defined before a single frame is recorded.

Artifacts

• Scenario brief
• Task taxonomy
• Operator consent

CAPTURE

Head-mounted multimodal capture

Egocentric RGB at 30–60 fps, spatial audio, IMU, GPS and, where available, depth and hand tracking — all hardware-synchronised on a shared clock.

Artifacts

• RGB / depth
• IMU + GPS
• Spatial audio

INGEST

Chain-of-custody ingest

Encrypted upload, checksum verification, device and session registration. Every asset gets an immutable ID that follows it through delivery.

Artifacts

• SHA-256 manifest
• Session registry
• Encrypted transfer

CLEAN

Signal cleaning & privacy

Corrupt frames, dropouts and drift are removed; timestamps are re-aligned. Faces, plates and screens are blurred, PII in audio is bleeped.

Artifacts

• Face/plate blur
• Audio redaction
• Clock re-sync

SEGMENT

Temporal segmentation

Continuous sessions are cut into atomic action clips with clear start and end boundaries, so each sample maps to one intent.

Artifacts

• Action clips
• Boundary timestamps
• Take quality flags

ANNOTATE

Human + model-assisted labels

Bounding boxes, hand and object keypoints, grasp types, natural-language task descriptions and step-level instructions — model pre-labels, human verification.

Artifacts

• 2D/3D boxes
• Hand keypoints
• Language instructions

ENRICH

Context enrichment

Scene, lighting, object inventory, tool usage, success/failure outcome, difficulty and operator metadata are attached to every clip.

Artifacts

• Scene metadata
• Outcome labels
• Embeddings

QC

Multi-pass quality control

Automated validators plus a blind human review pass. Inter-annotator agreement is tracked; batches below threshold are returned to annotation.

Artifacts

• IAA score
• Auto validators
• Reject / rework loop

PACKAGE

Training-ready packaging

Standard schemas (LeRobot, RLDS, WebDataset, COCO-style JSON), deterministic train/val/test splits, and a datasheet documenting provenance and bias.

Artifacts

• LeRobot / RLDS
• Datasheet
• Reproducible splits

DELIVER

Delivery & feedback loop

Handover via your cloud bucket or ours, with versioned releases. Model performance feedback flows back in to reweight the next collection round.

Artifacts

• Versioned releases
• S3 / GCS handover
• Impact scoring