NVIDIA released four open weight AI checkpoints built on its Cosmos 3 Nano base. Each model contains 16 billion trainable parameters and is designed to simulate how physical environments react to robotic movements and driving commands.

Two checkpoints focus on autonomous vehicles. Transfer Auto creates multi camera video and aligned LiDAR data under varying weather and lighting conditions. Sim Auto is a streaming variant that generates synchronized camera and LiDAR observations in real time using driving control inputs.

The remaining two checkpoints focus on robotics and physical simulation. Sim Transfer converts simulator controls and text prompts into photorealistic video using depth and segmentation maps. Sim Bimanual predicts visual updates directly from two armed robot action sequences.

Why it matters

Developing self driving vehicles and physical robots requires massive amounts of training data across unpredictable conditions. Gathering this data in the real world is expensive and unsafe. These models let developers generate realistic synthetic video and 3D LiDAR data straight from physics engines.

Because NVIDIA released these checkpoints under the OpenMDW1.1 license, creators, researchers, and startups in India and worldwide can use them commercially without licensing fees. Anyone with supported NVIDIA hardware can deploy these tools to build and test automated agents locally.

Try it yourself

Use this prompt to design a synthetic training data strategy for your own computer vision or robotics project.

I am building an AI vision model for [insert application, e.g., warehouse robots or autonomous delivery vehicles]. List five critical environmental edge cases I should generate in a simulator, including variations in [insert conditions, e.g., lighting, weather, or surface reflections]. For each scenario, explain the exact camera and depth sensor observations required to train the model effectively.