Drive Loop

Self-driving fleets record petabytes of video, and the rare dangerous moments are what the next model must learn from. Drive Loop finds them, checks them and turns them into training data. How it works: we use the Toronto dashcam footage already stored in the VAST VSS archive (clips in S3 on VAST, detections and search vectors in VastDB). Stored YOLO11 boxes give 558 candidates where a person is in the car's lane, out of 1,080 clips. NVIDIA Cosmos3-Reason watches each closest clip and says where the person is and what the car should do. NVIDIA Nemotron VL, served by W&B Inference, double-checks the frame and removes cyclists and duplicates: 20 events, 8 of 8 correct when checked by eye. Optical flow measures what the driver really did, so each event gets a 0-100 danger score and a driver-versus-model verdict. Engineers review events in a web app: clip with a perception overlay, speed chart synced to the video, and a "who was right?" label that becomes training data. Anyone can write a new detector in plain words, test it on one clip on the shared NVIDIA GPU in about 5 seconds, then run it over the footage. Each event becomes a scenario package for NVIDIA Cosmos Transfer or Omniverse; the simulator studio previews up to 1,080 variants (rain, snow, night, a child). Sending to a simulator is a demo. The app runs on CoreWeave Kubernetes, judge calls are traced in W&B Weave, and we built on the event's Cursor agent skills.