Sample project
A training-ready robot dataset: 3 episodes, 2,700 frames, 31 checks passed
Data processingDepthActions & Captions
- Data
- LeRobot-style dataset · RGB, depth, parquet
- Volume
- 3 episodes · 2,700 frames at 29.97 fps
- Status
- Sample project
The problem
Labels on their own don't train a policy. Robot-learning code expects episodes, per-frame observations, task text, sub-goals and train/validation splits in a predictable layout.
What we did
- Packaged each clip as an episode with a per-frame parquet table (timestamps, hand positions, sub-goal index, keyframe flags)
- Added RGB video, 16-bit depth frames, 2D tracks, 3D boxes, sub-goals and natural-language task instructions
- Split by episode, so no frame leaks between train and validation
- Wrote dataset statistics and 476 SHA-256 checksums; ran 31 validation checks with 0 errors
- Recorded a QA note per episode: every track checked frame by frame, 69, 35, 59 hand-placed keyframes
Result
3 episodes and 2,700 frames that load with one script, with 31/31 validation checks passing. The dataset states plainly that it is human demonstration video, with no robot joint or force streams.
Labeled video · 30 seconds
- Caption track
- 00:00Pumpkin purée is scraped out of the food-processor bowl with a red spatula.
- 00:02The purée on the spatula is scraped into the purée bowl.
- 00:04The food-processor bowl and spatula are cleared from the board.
- 00:05The white onion is taken from its glass bowl and brought to the board.
- 00:06The onion is peeled and cut with the knife.
- 00:13The onion is diced.
- 00:18The diced onion is gathered into the glass bowl.
- 00:20The bowl of onion is set beside the purée and the knife is put down.
- 00:21The bowl of oil is picked up and carried to the saucepan.
- 00:23The oil is poured into the saucepan on the hob.
- 00:25The bowl of garlic is picked up and carried to the saucepan.
- 00:28The garlic is added to the saucepan.
Frames, depth and review sheet
Sample output
An excerpt from the delivered files.
meta/splits.json + annotations/quality_control.jsonexcerpt
{
"split_by": "episode (no episode appears in two splits, so no frame leaks)",
"train": [
0,
1
],
"val": [
2
],
"quality_control": {
"episode": "episode_000000",
"outcome": "success",
"objects": 20,
"hand_placed_keyframes": 69,
"failures": []
}
}Source footage: “Pumpkin Curry Soup”, US Department of Agriculture (Agricultural Marketing Service), via Wikimedia Commons (Public domain). Labels, captions, tracks and depth by Annoroid.