Skip to content
Annoroid
Request a free pilot
All work

Sample project

Pixel-accurate segmentation: 18 traced masks on a single frame

SegmentationTracking
Pixel-accurate segmentation: 18 traced masks on a single frame — raw frame and labeled output
RAWLABELED
Data
Video · 1920×1080
Volume
18 masks on frame 54 · 150 frames tracked
Status
Sample project

The problem

Boxes tell a model roughly where an object is. Manipulation needs more: the exact edge of a knife blade, the rim of a bowl, where a hand ends and a tomato begins.

What we did

  • Objects outlined with SAM 2.1 video segmentation from reviewer-placed anchors
  • Every outline reviewed; drifting or leaking masks corrected
  • Outlines kept in COCO segmentation format and rasterised to class masks
  • Small objects (garlic cloves, herbs) drawn on top so they are never hidden by larger ones

Result

18 traced masks on this frame, 18 and 24 on the other two samples, delivered as COCO polygons and PNG class masks.

Labeled video · 30 seconds

  1. Caption track
  2. 00:00The last tomato half is sliced and the slices are added to the glass bowl.
  3. 00:01The board is cleared and a garlic bulb is brought over.
  4. 00:04Garlic cloves are broken off the bulb and peeled by hand.
  5. 00:12The peeled garlic is minced with the knife.
  6. 00:18The minced garlic is gathered by hand and sprinkled over the tomatoes.
  7. 00:22Salt is pinched from the salt bowl and sprinkled over the tomatoes.
  8. 00:26Pepper is pinched from the pepper bowl and sprinkled over the tomatoes.
  9. 00:28Serving tongs are brought in and the bowl is drawn to the centre of the board.
  10. 00:29The salad is tossed with the tongs.

Frames, depth and review sheet

Class mask · the training target
Class mask · the training target
Watermelon salad · 18 masks
Watermelon salad · 18 masks
Pumpkin soup · 24 masks
Pumpkin soup · 24 masks
Labeled frame 18
Frame 18 · labeled
Labeled frame 90
Frame 90 · labeled
Labeled frame 126
Frame 126 · labeled
Estimated depth for the same scene
Depth · frame 52
Contact sheet of 15 labeled frames used to review every track
Review contact sheet · every 10th frame

Sample output

An excerpt from the delivered files. "source": "ai" marks boxes propagated by the segmentation model; every one was reviewed by a person before export.

tomato.jsonexcerpt
{
  "image_id": 55,
  "category": "bowl stack",
  "bbox": [13, 85, 217, 299],
  "segmentation": [[102, 86, 83, 92, 60, 107, 35, 137, "…"]],
  "area": 52605.16
}

Source footage: “Farmers Market Food Demo - Spanish Tomato Salad”, US Department of Agriculture (Agricultural Marketing Service), via Wikimedia Commons (Public domain). Labels, captions, tracks and depth by Annoroid.

Request a free pilot for your own data.

Send 10 to 20 frames or a short clip. We label them to your guidelines and send a quality report within 48 hours.

Request a free pilotBook a 20-minute call