Back to Blog

NVIDIA: Cosmos World Foundation Model Platform for Physical AI

Jeremy WeaverJanuary 8, 2025
Premium

NVIDIA's Cosmos World Foundation Model platform for Physical AI uses a dual-stage training approach with diffusion and autoregressive models on a massive curated video dataset to create versatile foundation models that are fine-tuned for robotic manipulation, autonomous driving, and other tasks, featuring a novel video tokenizer and integrated safety measures.

NVIDIA: Cosmos World Foundation Model Platform for Physical AI



Summary of https://d1qx31qr3h6wln.cloudfront.net/publications/NVIDIA%20Cosmos_3.pdf

Introduces NVIDIA's Cosmos World Foundation Model (WFM) platform for Physical AI. Cosmos uses a pre-training and post-training paradigm, employing both diffusion and autoregressive models trained on a massive, curated video dataset (20M hours) to create generalist WFMs.

These are then fine-tuned for specialized Physical AI tasks like robotic manipulation and autonomous driving. The platform includes a novel video tokenizer for efficient processing and a guardrail system for safety.

Results demonstrate state-of-the-art performance across various benchmarks and applications.