Open Source RoboticsRobots, software & learning

VGGT (Visual Geometry Grounded Transformer)

Robotics model

1B-parameter feed-forward 3D transformer that directly infers camera extrinsics/intrinsics, depth maps, 3D point maps, and 3D point tracks from one, a few, or hundreds of RGB views. [1] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
3D vision and robotics perception researchers building multi-view reconstruction, mapping, or Gaussian Splatting pipelines; Commercial teams evaluating VGGT-1B-Commercial within Meta's Acceptable Use Policy boundaries [1] [2]
Parameters
1B parameters (VGGT-1B and VGGT-1B-Commercial: 5.026 GB safetensors / 5.027 GB .pt [1] [4]
Embodiments
Embodiment-agnostic 3D visual perception module (single-view or multi-view RGB cameras) [1] [5]
Source rights
Named licenses are documented by part. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware

Requirements · Research details and dates

On this page

Overview

VGGT (facebookresearch/vggt, CVPR 2025 Best Paper Award, arXiv:2503.11651) is a 1B-parameter Visual Geometry Grounded Transformer from the University of Oxford VGG and Meta AI that infers camera extrinsics and intrinsics, dense depth maps, 3D point maps, and 3D point tracks in a single feed-forward pass from one, a few, or hundreds of RGB views. At pinned commit a288dd0f14786c93483e45524328726ab7b1b4ce (updated May 2026 with a memory-efficiency fix enabling ~2–3× more input frames), the repository includes inference code, COLMAP/Gaussian Splatting export, Gradio/Viser demos, and PyTorch DDP fine-tuning scripts. [1] [3] [5] [6]

Model card summary

Architecture

1B-parameter feed-forward Visual Geometry Grounded Transformer combining a multi-view token aggregator (model.aggregator) with specialized prediction heads for camera extrinsics/intrinsics (camera_head, OpenCV convention), depth maps (depth_head), 3D point maps (point_head), and 3D point tracks (track_head).

[1] [5]

Parameter count

1B parameters (VGGT-1B and VGGT-1B-Commercial: 5.026 GB safetensors / 5.027 GB .pt; VGGT-500M and VGGT-200M remain unreleased)

[1] [4]

Input modalities
rgb

One, a few, or hundreds of RGB images (plus optional 2D query pixel coordinates for track_head).

[1]

Output modalities
6d-pose

Outputs OpenCV-convention 6D camera extrinsics and intrinsics alongside dense depth maps, 3D point maps, and 3D point tracks (exportable to COLMAP format).

[1] [5]

Action space

Perception-only 3D geometry model (no robot action output); produces OpenCV camera extrinsics/intrinsics, dense depth maps, 3D point maps, and 3D point tracks.

[1] [5]

Trained / supported embodiments
  • Embodiment-agnostic 3D visual perception module (single-view or multi-view RGB cameras)

[1] [5]

Training datasets
  • Multi-view 3D training datasets described in CVPR 2025 paper (exact full pre-training dataset mix not enumerated in README; training/README.md documents Co3D fine-tuning with JianyuanWang/co3d_anno)

[5] [6]

Interfaces
  • Python API (VGGT.from_pretrained)
  • COLMAP sparse exporter (demo_colmap.py)
  • Gradio and Viser interactive 3D viewers
  • PyTorch DDP training CLI

[1] [6]

Checkpoints

CheckpointFormatSizeAccessPinned revision
facebook/VGGT-1B (original, CC-BY-NC-4.0)safetensors5.026 GBopen860abec7937d
facebook/VGGT-1B-Commercial (gated, VGGT License v1)safetensors5.026 GBgatedebb29a532abe

Both 1B checkpoints verified via Hugging Face API (each offers both 5.026 GB model.safetensors and 5.027 GB .pt weights); VGGT-1B is open/CC-BY-NC-4.0 while VGGT-1B-Commercial is manually gated under VGGT License v1.

[1] [4]

Is this for you?

Use cases
  • Multi-view or zero-shot single-view 3D scene reconstruction (camera extrinsics/intrinsics, depth maps, 3D point maps)
  • 3D point tracking from user-supplied 2D query points
  • Feed-forward COLMAP sparse export (with optional bundle adjustment) for 3D Gaussian Splatting (gsplat)
  • Fine-tuning on custom multi-view datasets such as Co3D using PyTorch DDP

[1] [6]

Known limitations
  • License split between checkpoints: ungated facebook/VGGT-1B remains CC-BY-NC-4.0 (non-commercial), while commercial use requires manual approval for gated facebook/VGGT-1B-Commercial under VGGT License v1.
  • VGGT License v1 Acceptable Use Policy prohibits operation of critical infrastructure, transportation technologies, heavy machinery, and military/ITAR applications—a major constraint for many physical robotics deployments.
  • Smaller VGGT-500M and VGGT-200M checkpoints remain unchecked on the README checklist; only the 1B-parameter (5.026 GB) model is released.
  • Open Source Robotics has not installed VGGT, downloaded weights, run inference, or reproduced Co3D benchmarks.

[1] [2] [4]

Safety and limitations
  • VGGT License v1 Acceptable Use Policy (Section 2) explicitly bars use in military/warfare/ITAR, critical infrastructure, transportation technologies, and heavy machinery—teams deploying on autonomous vehicles or industrial robots must perform legal review.
  • Original ungated facebook/VGGT-1B weights are CC-BY-NC-4.0 (non-commercial); only gated facebook/VGGT-1B-Commercial permits commercial use.
  • Perception-only 3D geometry model outputting camera poses and point/depth maps; does not output robot control commands or metric-guaranteed safety bounds.
  • No independent runtime, memory, or Co3D accuracy verification was performed by Open Source Robotics.

[1] [2] [4]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA GPU recommended (bfloat16 on Ampere CC 8.0+, float16 on older CUDA GPUs; CPU fallback in code; upstream reports <1 s scene reconstruction)
  • Precision: bf16 / fp16
  • Edge / consumer capable: No

Quick Start uses torch.bfloat16 on CC >= 8.0 and torch.float16 otherwise; May 2026 commit reduced memory to allow ~2–3x more input frames, but exact minimum VRAM is not stated.

[1] [5]

Fine-tuning

[1] [6]

Host computer

CUDA-capable NVIDIA GPU (Ampere CC >= 8.0 for bf16, else fp16; 5.026 GB checkpoint); 4-GPU PyTorch DDP example documented for Co3D fine-tuning.

[1] [4] [6]

Operating system
  • Linux

PyTorch CUDA and DDP training workflow targets Linux.

[1] [6] · Editorial assessment

Dependencies
  • PyTorch and torchvision
  • numpy, Pillow, huggingface_hub
  • Optional demo/export deps (requirements_demo.txt, Gradio, Viser, gsplat==1.3.0)

[1]

Skills
  • Python and PyTorch (AMP bf16/fp16)
  • Multi-view geometry / COLMAP conventions

Inferred from Quick Start and COLMAP export workflow.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone facebookresearch/vggt and install requirements.txt (torch, torchvision, numpy, Pillow, huggingface_hub) or install as a package (pip install -e .).
  • Choose facebook/VGGT-1B (ungated, non-commercial) or apply for gated access to facebook/VGGT-1B-Commercial.
  • Load via VGGT.from_pretrained(...) in bfloat16/float16 autocast and pass preprocessed multi-view images to model(images).
  • Optionally run demo_colmap.py (with --use_ba) to export COLMAP sparse reconstructions for gsplat, or demo_gradio.py / demo_viser.py for 3D viewing.

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
Co3D Camera Pose Estimation (AUC@30)VGGT-1B-Commercial: 90.37 vs original VGGT-1B: 89.98 (CVPR 2025 Best Paper Award, arXiv:2503.11651).Upstream-reported in README License section; OSR has not run the evaluation branch or reproduced Co3D metrics.

[1] [5]

Project notes

Architecture, checkpoints, and workflows

VGGT aggregates multi-view image tokens (model.aggregator) and routes them to four task heads: camera_head (OpenCV-convention extrinsics and intrinsics), depth_head, point_head, and track_head (for user-supplied 2D query points). Inference runs in bfloat16 on Ampere+ GPUs (or float16 on older CUDA devices) and can export sparse COLMAP reconstructions (demo_colmap.py, with optional bundle adjustment) directly into gsplat. [1]

Two 1B-parameter checkpoints are published on Hugging Face (each providing 5.026 GB safetensors and 5.027 GB .pt weights): ungated facebook/VGGT-1B and gated facebook/VGGT-1B-Commercial. Access to VGGT-1B-Commercial requires an application; Hugging Face API metadata labels the gate manual, whereas the README describes an automated LLaMA-style approval workflow (the actual approval process was not tested in this review). Smaller VGGT-500M and VGGT-200M checkpoints remain unchecked on the README checklist. A re-implemented PyTorch DDP training and fine-tuning pipeline (training/README.md) demonstrates fine-tuning on Co3D with 4 GPUs. [1] [4] [6]

Upstream-reported benchmarks

The README reports Co3D camera pose estimation scores of AUC@30 = 90.37 for VGGT-1B-Commercial versus 89.98 for the original VGGT-1B, and the CVPR 2025 abstract reports sub-second scene reconstruction and state-of-the-art results across camera estimation, multi-view depth, dense point reconstruction, and 3D tracking. All performance figures are upstream-reported; Open Source Robotics has not installed VGGT, downloaded weights, run training, fine-tuning, or inference, or reproduced Co3D evaluations. [1] [5]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
May 19, 2026 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / a288dd0f14786c93483e45524328726ab7b1b4ce · software repository

Separate license matrix

Include VGGT as a widely used 3D visual geometry model under source-available terms; explicitly distinguish CC-BY-NC-4.0 ungated weights from gated VGGT-1B-Commercial weights and flag the AUP prohibition on transportation/heavy machinery.

LayerAvailability / license
Codeavailable · VGGT License v1 (with Acceptable Use Policy)
Custom Meta license (updated July 29, 2025) permitting commercial use subject to an Acceptable Use Policy prohibiting military/ITAR, critical infrastructure, transportation technologies, and heavy machinery. [2] [3]
Model weightsavailable · CC-BY-NC-4.0 (VGGT-1B) / VGGT License v1 (VGGT-1B-Commercial, gated)
Split checkpoint licensing: ungated facebook/VGGT-1B is CC-BY-NC-4.0 (non-commercial); manually gated facebook/VGGT-1B-Commercial is under VGGT License v1 with AUP restrictions. [1] [2] [4]
Training datasetsnot found · License not fully assessed
Pre-training datasets are not bundled in the repository; training/README.md links external Co3D and Hugging Face Co3D annotations for fine-tuning. [6]
Documentationavailable · VGGT License v1 (with Acceptable Use Policy)
README, training guide, package docs, and demo scripts included. [1] [2]

Licensing split and robotics AUP restrictions

Licensing requires close attention before deployment. While the repository code and gated facebook/VGGT-1B-Commercial checkpoint moved to VGGT License v1 (July 29, 2025) to permit commercial use, the original ungated facebook/VGGT-1B checkpoint remains under CC-BY-NC-4.0 (non-commercial). Moreover, VGGT License v1 incorporates an Acceptable Use Policy that explicitly prohibits military/ITAR applications as well as the "operation of critical infrastructure, transportation technologies, or heavy machinery"—clauses that directly affect autonomous vehicles and industrial robotics deployments. [1] [2] [4]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for VGGT (2 verified 1B checkpoints, split license and AUP analysis).