PharOS

NYC DOT’s traffic cameras watch the city’s most dangerous corners but record nothing, so every near miss is lost. PharOS is a 3D map of Manhattan with a heatmap of injury crashes from the last 12 months of NYC Open Data, and every node is a real DOT camera. Open an intersection to see its camera image, crash history, reported causes, crashes by hour, and next steps. For continuous video, PharOS monitors live 511NY feeds and returns a verdict on each 8-second clip in about 14 seconds. Plain-English search finds matching moments in recorded footage. How we used the stack: VAST AI OS: The workshop’s VAST pipeline had already indexed recorded New York footage into VastDB. PharOS queries it with hybrid semantic search and streams matching clips for seekable playback. NVIDIA: On each live clip, YOLO11 boxes every car, person and bike across 32 frames, while Cosmos Reason reads the same video and describes what happened. Neither can claim motion from a single frame. CoreWeave: Its GPUs serve the NVIDIA models, which return results in under 3 seconds. Weights & Biases: W&B Inference runs Llama 3.3 70B. It turns each clip’s findings into a structured verdict, turns questions into search plans, and writes next steps linked to official guidance. Next: live video at every DOT camera, and a citywide ranking of intersections by near misses for planners.