V-JEPA 2-AC
Robotics modelAction-conditioned video world-model implementation and DROID post-training configuration for robotics research. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Robotics researchers evaluating action-conditioned latent prediction [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
- Parameters
- 1B ViT-g/16 encoder (1,034,555,264 F32 params in base HF checkpoint) + 24-layer d=1024 action-conditioned predictor… [1] [6] [12] [14]
- Embodiments
- Franka Emika Panda (DROID post-training dataset and zero-shot two-lab evaluations reported in arXiv:2506.09985) [6] [13]
- Source rights
- Some licenses are unresolved. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- MIT majority code; selected preprocessing Apache-2.0 · license not fully assessed for model weights · Details by part
Overview
V-JEPA 2-AC is Meta FAIR's action-conditioned latent world model inside the facebookresearch/vjepa2 repository (which also hosts generic V-JEPA 2 and V-JEPA 2.1 video encoders). Rather than decoding robot actions directly, V-JEPA 2-AC pairs a frozen 1B-parameter V-JEPA 2 ViT-g/16 video encoder (1,034,555,264 F32 parameters in the base facebook/vjepa2-vitg-fpc64-256 Hugging Face checkpoint) with a 24-layer action-conditioned transformer predictor (pred_embed_dim: 1024, pred_num_heads: 16, RoPE) to predict future latent video representations given robot actions. [1] [5] [6] [13] [14]
Model card summary
- Architecture
Frozen V-JEPA 2 ViT-g/16 video encoder (1B / 1,034,555,264 params in base HF ViT-g) paired with a 24-layer action-conditioned transformer predictor (pred_embed_dim: 1024, pred_num_heads: 16, RoPE) operating on 256x256 8-frame video clips.
- Parameter count
1B ViT-g/16 encoder (1,034,555,264 F32 params in base HF checkpoint) + 24-layer d=1024 action-conditioned predictor (exact combined AC parameter count not stated in README; vjepa2-ac-vitg.pt is 11.76 GB)
- Input modalities
- rgbproprioception
256x256 RGB video clips (8 frames/clip in droid-256px-8f.yaml) conditioned on recorded robot end-effector state and action sequences.
- Output modalities
- video-frames
Predicts future latent video patch representations conditioned on robot actions (mapped to closest schema enum video-frames; outputs latent embeddings rather than decoded RGB pixels, used with planning for robot control).
- Action space
Action-conditioned latent dynamics prediction over 256x256 8-frame clips conditioned on DROID robot trajectory actions; downstream manipulation relies on external image-goal planning rather than direct end-to-end policy action decoding.
- Trained / supported embodiments
- Franka Emika Panda (DROID post-training dataset and zero-shot two-lab evaluations reported in arXiv:2506.09985)
- Training datasets
- V-JEPA 2 base encoder pretrained on >1M hours of internet video; V-JEPA 2-AC post-trained on <62 hours of unlabeled robot videos from the DROID dataset (per arXiv:2506.09985)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| vjepa2-ac-vitg.pt | pytorch | 11.76 GB | open | etag:12c945a28cc4f447e728fd2ba810607c-1402 (2025-05-30) |
Public HTTP 200 download (11,760,743,310 bytes) verified via HEAD on 2026-11-21 without downloading binary payload.
Is this for you?
- Use cases
- Latent action-conditioned video dynamics prediction for robotics planning
- Zero-shot image-goal manipulation planning research on Franka arms
- Post-training action-conditioned world models on DROID robot trajectories
- Known limitations
- Code license is separate from checkpoint/data terms; binary payloads and normalization/action contract not independently validated.
- Training config resource requests are not measured inference requirements or demonstrated robot planning.
Editorial assessment of cited source; no runtime or hardware test.
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] · Editorial assessment
- Safety and limitations
- Inspected src/hub/backbones.py sets the checkpoint base URL to a local HTTP server on port 8300 with the public URL commented out; reconcile before using torch.hub.load.
- Code is MIT (with Apache-2.0 preprocessing), and the separate base ViT-g HF card declares Apache-2.0, but an artifact-specific license grant for vjepa2-ac-vitg.pt remains unestablished.
- Training config requests (4 nodes, 8 tasks/node, 220G mem_per_gpu) are workload requests, not measured inference minimums.
- No weight deserialization, planning rollout, or physical robot testing was performed by Open Source Robotics.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU strongly recommended by README (training config requests 4 nodes x 8 tasks, mem_per_gpu 220G)
- Edge / consumer capable: No
11.76 GB ViT-g + 24-layer predictor checkpoint; minimum inference VRAM and latency_ms are not established.
- Fine-tuning
- Supported: Yes
- Methods: DROID post-training via app/vjepa_droid/train.py and configs/train/vitg16/droid-256px-8f.yaml
- Recipe: Upstream fine-tuning documentation
- Host computer
CUDA-enabled PyTorch strongly recommended by README; droid-256px-8f.yaml training config requests 4 nodes x 8 tasks with mem_per_gpu 220G (workload config, not a measured inference minimum).
- Operating system
- Linux
Inferred from CUDA/PyTorch and distributed training config.
Setup & usage
Open the upstream getting-started guide [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
- Inspect vjepa2_ac_vit_giant Hub entry and the action-conditioned predictor before selecting a checkpoint.
- For adaptation inspect droid-256px-8f.yaml and replace dataset/pretrain paths; keep video and robot action conventions matched.
Editorial assessment of cited source; no runtime or hardware test.
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] · Editorial assessment
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Zero-shot Franka manipulation with image goals in two labs (arXiv:2506.09985) | Upstream paper reports zero-shot pick-and-place on Franka arms in two labs without environment-specific data collection, task-specific training, or reward | Upstream-reported results in arXiv:2506.09985; OSR has not downloaded weights, run planning, or evaluated physical robots. |
Project notes
Checkpoint, loader discrepancy, and post-training config
The README lists the public PyTorch checkpoint vjepa2-ac-vitg.pt (11,760,743,310 bytes / 11.76 GB, HTTP 200 verified via HEAD on 2026-11-21 without downloading the binary; ETag 12c945a28cc4f447e728fd2ba810607c-1402) and PyTorch Hub entry vjepa2_ac_vit_giant. At pinned commit 204698b45b3712590f06245fbfba32d3be539812, hubconf.py exports that entry, but src/hub/backbones.py points its checkpoint base URL to a local HTTP server on port 8300 with the public URL commented out—so users must reconcile that loader configuration before relying on torch.hub.load. Full-source setup uses Python 3.12 with PyTorch, timm, einops, and strongly recommended CUDA support. [1] [7] [8] [11] [12]
For post-training, configs/train/vitg16/droid-256px-8f.yaml configures app/vjepa_droid/train.py with 256×256 crops, 8-frame clips, and placeholder paths for DROID trajectory CSVs and a pretrained vitg.pt checkpoint. Its Slurm-style workload config requests four nodes, eight tasks per node, and mem_per_gpu: 220G; those are training workload requests rather than measured inference minimums. Minimum inference VRAM and latency remain unestablished in this review, and adapting new robot trajectories requires checking action/state conventions and normalization directly against src/models/ac_predictor.py and app/vjepa_droid/train.py—a contract that has not been independently validated here. [1] [4] [5] [6] [12]
Upstream-reported evaluations and unverified execution
According to arXiv:2506.09985, the base V-JEPA 2 encoder was pretrained on over 1 million hours of internet video, and V-JEPA 2-AC was post-trained on fewer than 62 hours of unlabeled DROID robot videos, then deployed zero-shot on Franka arms in two labs for pick-and-place via image-goal planning without task-specific training or reward. All evaluation results are strictly upstream-reported; Open Source Robotics has not downloaded weight binaries, run post-training or inference, or tested robot planning. [1] [13]
Separate code, checkpoint, and dataset terms
The facebookresearch/vjepa2 repository is licensed under MIT, with selected preprocessing utilities under Apache-2.0 (APACHE-LICENSE), and was last pushed 2026-10-29. While Meta's release blog describes code and checkpoints as available for commercial and research applications and the separate base facebook/vjepa2-vitg-fpc64-256 Hugging Face card declares Apache-2.0, an artifact-specific license notice for vjepa2-ac-vitg.pt itself remains unestablished in the inspected files, and DROID trajectory data must be obtained separately under its own terms. [2] [3] [9] [10] [14]
What we checked
Pinned code/document inspection only; no weight/data download, environment, post-training, robot planning or physical test.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Oct 29, 2026 [14]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- GitHub facebookresearch/vjepa2@204698b45b3712590f06245fbfba32d3be539812; checkpoint vjepa2-ac-vitg.pt (ETag 12c945a28cc4f447e728fd2ba810607c-1402) · software repository
Separate license matrix
Narrow robotics-facing2-AC implementation and source research; generic V-JEPA2/2.1 video features alone are not the selected contribution.
| Layer | Availability / license |
|---|---|
| Code | available · MIT majority code; selected preprocessing Apache-2.0 Code notices do not automatically license downloaded model weights, DROID trajectories or unrelated assets. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] |
| Model weights | available · License not fully assessed Official checkpoint vjepa2-ac-vitg.pt verified via HTTP HEAD (200, 11.76 GB, ETag 12c945a28cc4f447e728fd2ba810607c-1402) without binary download. Meta blog describes code and checkpoints as available for commercial/research use, and base ViT-g HF card declares Apache-2.0, but an artifact-specific license notice for the AC checkpoint itself remains unestablished. [1] [9] [10] [11] [12] |
| Training datasets | unavailable · License not fully assessed Base encoder pretrained on >1M hours of internet video (not redistributed); AC post-training uses <62 hours of DROID robot trajectories supplied externally via placeholder CSV paths in droid-256px-8f.yaml. [6] [13] |
| Documentation | available · MIT majority code; selected preprocessing Apache-2.0 Code notices do not automatically license downloaded model weights, DROID trajectories or unrelated assets. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] |
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 README.md
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· README.md at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S002 LICENSE
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· LICENSE at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Separate code, checkpoint, and dataset terms and 1 more - S003 APACHE-LICENSE
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· APACHE-LICENSE at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Separate code, checkpoint, and dataset terms and 1 more - S004 app/vjepa_droid/train.py
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· app/vjepa_droid/train.py at · Accessed Nov 1, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Setup & usage and 2 more - S005 src/models/ac_predictor.py
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· src/models/ac_predictor.py at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Is this for you?, Compute & fine-tuning and 3 more - S006 configs/train/vitg16/droid-256px-8f.yaml
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· configs/train/vitg16/droid-256px-8f.yaml at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Is this for you?, Compute & fine-tuning and 3 more - S007 hubconf.py
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· hubconf.py at · Accessed Nov 1, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Setup & usage and 2 more - S008 src/hub/backbones.py
Revision, date & where cited
facebookresearch/vjepa2 contributors · github.com · revision
204698b· src/hub/backbones.py at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Checkpoint, loader discrepancy, and post-training config and 1 more - S009 Official V-JEPA2 release announcement
Revision, date & where cited
Meta FAIR · ai.meta.com · Official source checked 2026-11-01 · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Separate code, checkpoint, and dataset terms and 1 more - S010 Official base ViT-g model card (not action-conditioned)
Revision, date & where cited
Meta FAIR / facebook · huggingface.co · Official source checked 2026-11-01 · Accessed Nov 1, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Setup & usage and 2 more - S011 Official2-AC checkpoint download availability
Revision, date & where cited
Meta FAIR · dl.fbaipublicfiles.com · Official source checked 2026-11-01 · Accessed Nov 1, 2026
Used for: Checkpoints, Is this for you?, Compute & fine-tuning, Setup & usage and 2 more - S012 HTTP HEAD verification of vjepa2-ac-vitg.pt checkpoint
HTTP 200, content-length 11,760,743,310 bytes (11.76 GB), last-modified 2025-05-30, ETag 12c945a28cc4f447e728fd2ba810607c-1402.Revision, date & where cited
Meta FAIR · dl.fbaipublicfiles.com · HTTP HEAD response on 2026-11-21 · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Compute & fine-tuning, Checkpoint, loader discrepancy, and post-training config and 1 more - S013 V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (arXiv:2506.09985)
V-JEPA 2 pretrained on >1M hours internet video; V-JEPA 2-AC post-trained on <62 hours unlabeled DROID robot videos and evaluated zero-shot on Franka arms in two labs for pick-and-place with image goals.Revision, date & where cited
Meta FAIR (arXiv) · arxiv.org · arXiv:2506.09985v1 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Upstream-reported benchmarks and 2 more - S014 GitHub API metadata for facebookresearch/vjepa2 and HF API metadata for facebook/vjepa2-vitg-fpc64-256
GitHub pushed_at 2026-10-29T05:39:42Z, license MIT; base HF ViT-g encoder has 1,034,555,264 F32 parameters (sha 875c192b7b704b87d1e1d99345769632dd5f739a, license apache-2.0).Revision, date & where cited
Meta FAIR (GitHub / Hugging Face) · huggingface.co · GitHub /repos/facebookresearch/vjepa2 and HF /api/models/facebook/vjepa2-vitg-fpc64-256 · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Separate code, checkpoint, and dataset terms, What we checked
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Migrated from 0.7.0-draft.5 project to 0.8.0-draft.1 model; added verified 11.76 GB checkpoint metadata, ViT-g + 24-layer AC predictor architecture, and upstream-reported Franka benchmarks.