Cosmos-Predict2
Robotics modelArchived NVIDIA Cosmos world foundation model family (0.6B/2B/14B Text2Image and 2B/14B Video2World, plus Bridge action-conditioned and GR00T Dreams samples) with Apache-2.0 code and gated NVIDIA Open Model License weights; superseded by Cosmos-Predict2.5. [1] [2] [4]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Physical AI and world-model researchers; Robotics simulation and synthetic-data engineers; Post-training teams with datacenter NVIDIA GPUs [2] [3] [5]
- Embodiments
- General-purpose Physical AI visual world simulator (embodiment-agnostic Video2World), GR1 humanoid robot (GR00T… [2] [5]
- Source rights
- Named licenses are documented by part. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
Overview
Cosmos-Predict2 is NVIDIA's second-generation world foundation model family for Physical AI, spanning Text2Image (0.6B, 2B, 14B) and Video2World (2B, 14B) Diffusion Transformer checkpoints plus robotics post-trained samples for GR00T Dreams (GR1 and DROID) and Bridge action-conditioned video prediction. The GitHub repository (nvidia-cosmos/cosmos-predict2) was archived when checked on 2026-11-21, its reviewed HEAD commit 661da4774b0c is dated 2025-10-07, and its README explicitly advises migrating to nvidia-cosmos/cosmos-predict2.5 for continued support. [1] [2] [4]
Model card summary
- Architecture
Diffusion Transformer (DiT) world foundation model family for Text2Image (0.6B, 2B, 14B) and Video2World (2B, 14B) visual world prediction, supporting multi-frame conditioning, optional NATTEN sparse attention for 720p Hopper/Blackwell inference, and action-conditioned Video2World post-training.
- Base model
Cosmos-Predict2 DiT (0.6B / 2B / 14B base checkpoints)
- Parameter count
0.6B, 2B, and 14B variants (Text2Image: 0.6B/2B/14B; Video2World: 2B/14B; Action-Conditioned sample: 2B; GR00T Dreams GR1/DROID samples: 14B)
- Input modalities
- rgblanguageproprioception
RGB image/video and text prompts across Video2World/Text2Image; Bridge action-conditioned variant adds end-effector pose/gripper state and 7-DoF action displacement conditioning.
- Output modalities
- video-frames
Video2World outputs future RGB video frames (480p and 720p at 10-16 FPS); Text2Image variants output single RGB images.
- Action space
Text/visual prompt conditioning in base Video2World; in the Bridge action-conditioned variant, 7-DoF end-effector displacement (x, y, z, roll, pitch, yaw in gripper frame plus binary open/close gripper command) paired with end-effector state and gripper width, conditioning generated future video frames rather than directly driving hardware.
- Control frequency
10-16 Hz (10 FPS and 16 FPS video generation documented in performance guide)
Refers to generated video frame rate, not real-time closed-loop robot control frequency.
- Trained / supported embodiments
- General-purpose Physical AI visual world simulator (embodiment-agnostic Video2World)
- GR1 humanoid robot (GR00T Dreams GR1 post-trained sample)
- DROID robot manipulation platform (GR00T Dreams DROID post-trained sample)
- Bridge robot arm dataset embodiment (2B action-conditioned sample)
- Training datasets
- Pretraining corpus not publicly disclosed or released
- Bridge dataset (IRASim split) for action-conditioned post-training sample
- GR00T Dreams GR1 and DROID datasets for post-trained sample checkpoints
- AgiBotWorld-Alpha (fisheye) and Cosmos-NeMo-Assets documented as post-training case studies
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| nvidia/Cosmos-Predict2-0.6B-Text2Image | pytorch | 4.32 GB | gated | dd55b6858b22 |
| nvidia/Cosmos-Predict2-2B-Text2Image | safetensors | 18.57 GB | gated | acdb5fde992a |
| nvidia/Cosmos-Predict2-14B-Text2Image | safetensors | 67.81 GB | gated | 015332720f70 |
| nvidia/Cosmos-Predict2-2B-Video2World | safetensors | 38.14 GB | gated | f50c09f5d8ab |
| nvidia/Cosmos-Predict2-14B-Video2World | safetensors | 210.46 GB | gated | 03b03a377fed |
| nvidia/Cosmos-Predict2-14B-Sample-GR00T-Dreams-GR1 | pytorch | 29.04 GB | gated | 926f34dd8386 |
| nvidia/Cosmos-Predict2-14B-Sample-GR00T-Dreams-DROID | pytorch | 29.04 GB | gated | ebc29d30a3fa |
| nvidia/Cosmos-Predict2-2B-Sample-Action-Conditioned | pytorch | 4.56 GB | gated | 1d2b0a920e76 |
Sizes are decimal GB sums of sibling files at the pinned HF revision (repos marked safetensors also contain .pt/.pth checkpoints). All 8 repos are gated='auto' under NVIDIA Open Model License.
Is this for you?
- Use cases
- Video-and-text conditioned future visual world prediction (Video2World)
- Action-conditioned robotic manipulation video prediction (Bridge post-training)
- Text-to-image Physical AI scene generation (0.6B, 2B, 14B)
- Domain-specific world-model post-training (AgiBotWorld-Alpha fisheye, GR00T Dreams GR1/DROID, Cosmos-NeMo-Assets)
- Known limitations
- Upstream GitHub repository is archived (HEAD 2025-10-07) and explicitly recommends migrating to Cosmos-Predict2.5 for continued support
- Model weights are gated on Hugging Face under the NVIDIA Open Model License rather than Apache-2.0, and default safety guardrails rely on Llama Guard 3 under its own license
- High GPU VRAM and storage requirements (26.02-32.54 GB VRAM for 2B models, 48.93-56.38 GB VRAM for 14B models; 14B-Video2World repo is 210.46 GB at HEAD)
- Pretraining datasets are not released; NATTEN sparse attention speedup is restricted to 720p inference on Hopper/Blackwell GPUs (compute capability 9.0 or 10.0)
- Safety and limitations
- Outputs simulated images or video frames rather than closed-loop physical robot motor commands; supplied documentation does not establish physical-dynamics accuracy for real robot control.
- Repository is archived (2025-10-07) and superseded by Cosmos-Predict2.5; users adopting Predict2 weights accept discontinued upstream maintenance.
- Weights are gated under the NVIDIA Open Model License (not OSI-approved Apache-2.0), and default content moderation uses Llama Guard 3 under its own license.
- Pretraining datasets are not released; 14B-Text2Image reports OOM on NVIDIA L40S in upstream benchmarks.
Compute & fine-tuning
- Inference compute
- Minimum VRAM: 26 GB
- Documented hardware: NVIDIA GB200, B200, H200 SXM/NVL, H100 PCIe/NVL/SXM, H20, L40S, RTX PRO 6000, RTX 6000 Ada, DGX Spark
- Precision:
bf16 - Reported latency: 25610 ms
- Edge / consumer capable: No
Required VRAM in performance.md: 26.02 GB (2B-Text2Image), 32.54 GB (2B-Video2World), 48.93 GB (14B-Text2Image), 56.38 GB (14B-Video2World). Latency 25,610 ms (25.61 s) is fastest upstream-reported 480p 16 FPS 2B-Video2World generation on GB200 (79.87 s on H100 PCIe; 14B-Video2World 85.26 s on GB200, 286.46 s on H100 PCIe).
- Fine-tuning
- Supported: Yes
- Methods: Full post-training (FSDP); Action-conditioned Video2World post-training
- Recipe: Upstream fine-tuning documentation
Action-conditioned Bridge post-training example uses torchrun --nproc_per_node=2 with FSDP; exact minimum post-training VRAM is not specified in that guide.
- Host computer
Linux x86-64 with NVIDIA Ampere (RTX 30 Series, A100) or newer GPU; upstream recommends at least 32 GB VRAM for 2B models (26.02-32.54 GB measured) and at least 64 GB VRAM for 14B models (48.93-56.38 GB measured).
- Operating system
- Linux x86-64 (glibc >=2.31, e.g., Ubuntu >=22.04)
- Dependencies
- Python 3.10
- PyTorch 2.6.0 (cu126)
- CUDA 12.6
- diffusers>=0.34.0 (optional HF pipeline)
- NATTEN (optional sparse attention on Hopper/Blackwell)
- Llama Guard 3 (content input filter)
Setup & usage
Open the upstream getting-started guide [2] · Model card / documentation [1] [2]
- Prepare Linux x86-64 (glibc >=2.31) with NVIDIA Ampere or newer GPU, CUDA 12.6 compatible driver, and Python 3.10
- Create a Python 3.10 virtual environment with uv and install cosmos-predict2[cu126] from PyPI with the cosmos-dependencies extra index
- Accept NVIDIA Open Model License gating on Hugging Face and download checkpoints or run via diffusers>=0.34.0
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Cosmos-Predict2 480p 16 FPS Video2World inference latency (documentations/performance.md) | 2B-Video2World: 25.61 s (GB200), 30.7 s (B200), 50.2 s (H200 SXM), 79.87 s (H100 PCIe), 127.49 s (L40S); 14B-Video2World: 85.26 s (GB200), 176.19 s (H200 SXM), 286.46 s (H100 PCIe), 1036.24 s (L40S). | Upstream-reported generation runtime benchmarks at 480p 16 FPS; not independently verified by OSR. |
| Cosmos-Predict2 720p 16 FPS Video2World NATTEN sparse attention speedup (documentations/performance.md) | 2B-Video2World: 123.9 s -> 54.0 s (2.3x on B200), 221.7 s -> 89.4 s (2.5x on H200 SXM), 378.5 s -> 149.6 s (2.5x on H100 PCIe); 14B-Video2World: 439.4 s -> 223.1 s (2.0x on B200), 836.9 s -> 412.9 s (2.0x on H200 SXM). | Upstream-reported 720p Hopper/Blackwell sparse attention benchmarks; requires compute capability 9.0 or 10.0; not independently verified by OSR. |
Project notes
Capabilities, action conditioning, and robotics post-training
Base Video2World checkpoints generate future visual frames from an initial image or video and a text prompt, with Python (cosmos-predict2 on PyPI) and diffusers>=0.34.0 inference interfaces. For robotics, documentations/post-training_video2world_action.md documents an action-conditioned Bridge recipe (using the IRASim Bridge split) that conditions the 2B Video2World model on end-effector pose, gripper width, and 7-DoF gripper displacement (x, y, z, roll, pitch, yaw plus binary open/close) via 2-GPU torchrun FSDP training; minimum per-GPU VRAM for this post-training recipe is unspecified in the supplied documentation, and the inference memory figures below should not be treated as training requirements. Like other video world models, Cosmos-Predict2 outputs simulated visual frames rather than closed-loop physical robot motor commands, and the reviewed documentation does not establish physical-dynamics fidelity for direct hardware control. [2] [3] [5]
Hardware footprint and upstream-reported performance
Upstream's performance guide specifies Linux x86-64 with NVIDIA Ampere or newer GPUs and CUDA 12.6, with measured VRAM requirements of 26.02 GB (2B-Text2Image), 32.54 GB (2B-Video2World), 48.93 GB (14B-Text2Image), and 56.38 GB (14B-Video2World). Although upstream recommends at least 32 GB VRAM for 2B models and 64 GB for 14B models (14B-Text2Image reports OOM on L40S), the reported 32.54 GB requirement for 2B-Video2World exceeds that general 32 GB recommendation, so the recommendation does not establish that 2B-Video2World fits on a 32 GB GPU. Upstream reports 480p 16 FPS Video2World runtimes ranging from 25.61 s (2B on GB200) and 79.87 s (2B on H100 PCIe) to 85.26 s (14B on GB200) and 286.46 s (14B on H100 PCIe), plus 2.0x–2.6x speedups at 720p using NATTEN sparse attention on Hopper/Blackwell GPUs (compute capability 9.0 or 10.0). Total repository file sizes at the reviewed Hugging Face revisions span 4.32 GB (0.6B-Text2Image) to 210.46 GB (14B-Video2World) across multiple bundled file formats, so the download size for a selected inference configuration may differ. All runtime and VRAM figures are upstream-reported; Open Source Robotics has not run training, fine-tuning, or inference on Cosmos-Predict2. [2] [3] [4]
What we checked
None. Prepared from public upstream sources without hands-on training, fine-tuning, or inference.
Updates
- Upstream last updated
- Oct 7, 2025 [1]
- Latest release
- Sep 1, 2025 [4]
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- nvidia-cosmos/cosmos-predict2@661da4774b0ca41d082a0ecbeb47550bcf07e03f; HF Cosmos-Predict2-2B-Video2World@f50c09f5d8ab133a90cac3f4886a6471e9ba3f18; HF Cosmos-Predict2-14B-Video2World@03b03a377fede782647afac998f674d9f358e319 · software repository
Separate license matrix
Major Physical AI world foundation model family with Apache-2.0 inference/post-training code and 8 public checkpoints, marked include_partial because weights are gated under the NVIDIA Open Model License, pretraining datasets are unreleased, and the repo is archived in favor of Cosmos-Predict2.5.
| Layer | Availability / license |
|---|---|
| Code | available · Apache-2.0 Verified Apache-2.0 in root LICENSE file and README. [1] [2] |
| Model weights | available · NVIDIA Open Model License (gated) All 8 checkpoints are gated='auto' on Hugging Face under the NVIDIA Open Model License; Llama Guard 3 filter carries its own license. [2] [4] |
| Training datasets | unavailable · License not fully assessed Pretraining datasets are not released; post-training tutorials reference external datasets such as Bridge (IRASim), AgiBotWorld-Alpha, and GR00T Dreams. [2] [5] |
| Documentation | available · Apache-2.0 [1] [2] [3] |
Licensing boundaries and archived status
Licensing splits between code, weights, and auxiliary guardrails. The repository source code is Apache-2.0 (LICENSE), whereas all eight Hugging Face checkpoints are gated (gated: auto) under the NVIDIA Open Model License, and default input content moderation relies on Llama Guard 3 under its own license. Pretraining datasets are not released. Teams evaluating Cosmos-Predict2 should account for both the non-OSI weight license and the archived status of the repository. [1] [2] [4]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 nvidia-cosmos/cosmos-predict2 GitHub metadata, HEAD commit, and LICENSE
Default branch main HEAD is 661da4774b0ca41d082a0ecbeb47550bcf07e03f (2025-10-07T04:55:31Z); archived=true; root LICENSE is Apache-2.0.Revision, date & where cited
nvidia-cosmos (GitHub) · github.com · revision
661da47· GitHub API repos/nvidia-cosmos/cosmos-predict2 & root LICENSE at 661da4774b0c · Accessed Nov 21, 2026
Used for: Overview, Is this for you?, Setup & usage, What we checked and 1 more - S002 nvidia-cosmos/cosmos-predict2 README.md at pinned commit
Lists 8 Hugging Face models (0.6B/2B/14B Text2Image, 2B/14B Video2World, GR00T Dreams GR1/DROID, Bridge Action-Conditioned), PyPI/Diffusers >=0.34.0 setup, Apache-2.0 code vs NVIDIA Open Model License weights, Llama Guard 3 filter, and migration notice to Cosmos-Predict2.5.Revision, date & where cited
nvidia-cosmos (GitHub) · github.com · revision
661da47· README.md at commit 661da4774b0c · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 5 more - S003 Cosmos-Predict2 Performance Guide (documentations/performance.md)
Reports GPU VRAM requirements (26.02-56.38 GB), 480p 16 FPS generation latencies across 11 NVIDIA GPUs (including L40S OOM on 14B-Text2Image), and 720p NATTEN sparse attention speedups (up to 2.6x on Hopper/Blackwell).Revision, date & where cited
nvidia-cosmos (GitHub) · github.com · revision
661da47· documentations/performance.md at commit 661da4774b0c · Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Upstream-reported benchmarks and 3 more - S004 Hugging Face API metadata for 8 nvidia/Cosmos-Predict2-* checkpoints
Verified pinned SHAs, sibling file sizes (4.32 GB to 210.46 GB at HEAD), gated='auto', and license_name='nvidia-open-model-license' across all 8 checkpoints.Revision, date & where cited
nvidia (Hugging Face) · huggingface.co · HF API /api/models/nvidia/Cosmos-Predict2-* (8 repos) · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 4 more - S005 Cosmos-Predict2 Video2World Action-Conditioned Post-Training Guide
Documents Bridge dataset (IRASim split) 7-DoF gripper displacement + binary gripper action conditioning and 2-GPU torchrun FSDP post-training recipe.Revision, date & where cited
nvidia-cosmos (GitHub) · github.com · revision
661da47· documentations/post-training_video2world_action.md at commit 661da4774b0c · Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Capabilities, action conditioning, and robotics post-training and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile prepared from archived GitHub commit 661da4774b0c and 8 verified Hugging Face checkpoints.