LargeScaleRobotDataCollection

Collecting humanoid data is hard and expensive. We created real-time data collection pipeline, it corrects the collector if they make a mistake in real time (visual "unit tests"). A voice guide walks collectors through a task, vision checks each placement We use LLMs and Vision LLMs hosted on Weights&Biases as part of this. All videos are then uploaded via API to VAST platform for navigation, exploration and quality assurance (see https://real-sim-physical.vercel.app/library which uses VAST platform). The videos are then used to train NVidia GR00T model and we demonstrate that GR-1 model works with trained model in NVidia's simulator. The model runs on GPUs on Modal.com.