DUSt3R (Dense and Unconstrained Stereo 3D Reconstruction)
Robotics modelPairwise 3D pointmap regression transformer from Naver Labs Europe that reconstructs dense 3D scene geometry, depth maps, pixel matches, and camera poses from uncalibrated RGB images without prior camera intrinsics. [1] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- 3D computer vision and robotics perception researchers building uncalibrated reconstruction, visual localization, or scene-mapping pipelines; Academic and non-commercial teams fine-tuning pointmap transformers on custom multi-view datasets [1] [2]
- Embodiments
- Embodiment-agnostic 3D visual perception module (monocular, stereo, or multi-view uncalibrated RGB cameras) [1] [5]
- Source rights
- Named licenses are documented by part. Includes noncommercial terms. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- CC-BY-NC-SA-4.0 (noncommercial), CC-BY-NC-SA-4.0 plus upstream training dataset licenses (noncommercial), Mixed upstream dataset licenses (predominantly non-commercial: CC-BY-NC-4.0, CC-BY-NC-SA-4.0, custom NC; plus InternalUnreleasedDataset) (noncommercial) · Details by part
Overview
DUSt3R (naver/dust3r, CVPR 2024, arXiv:2312.14132) is a geometric 3D vision model from NAVER Labs Europe that reformulates dense multi-view stereo reconstruction as pairwise 3D pointmap regression without requiring prior camera intrinsics or poses. At pinned commit 4c24a6ebf04809f2cfe59915e51779c8984aaa40 (last updated July 2025), the repository provides inference code, an interactive Gradio/Docker demo, visual localization (visloc.py), dataset preprocessing scripts, and 3-stage PyTorch DDP training commands. [1] [3] [5]
Model card summary
- Architecture
AsymmetricCroCo3DStereo Siamese-encoder / cross-attending-decoder Vision Transformer (ViT-Large encoder: dim 1024, depth 24, 16 heads; ViT-Base decoder: dim 768, depth 12, 12 heads; RoPE100 positional embeddings) initialized from CroCo v2, paired with either a linear or DPT regression head outputting dense 3D pointmaps and per-pixel confidence maps.
- Base model
CroCo v2 (CroCo_V2_ViTLarge_BaseDecoder)
Training step 1 initializes AsymmetricCroCo3DStereo from CroCo_V2_ViTLarge_BaseDecoder.pth.
- Parameter count
Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder; safetensors checkpoints are 2.285 GB for 512_dpt and 2.129 GB for 512_linear / 224_linear)
Verified exact Hugging Face model.safetensors sizes: 512_dpt (2,284,790,056 bytes / 2.285 GB), 512_linear (2,129,459,344 bytes / 2.129 GB), 224_linear (2,129,459,344 bytes / 2.129 GB).
- Input modalities
- rgb
One, two, or more uncalibrated RGB images resized to 224 or 512 px max side.
- Output modalities
- 6d-pose
Outputs dense pairwise 3D pointmaps and confidence maps, from which global alignment recovers 6D camera poses, focal lengths, depth maps, and 2D-2D pixel matches.
- Action space
Perception-only 3D geometry model (no robot action output); produces dense 3D pointmaps, confidence masks, focal lengths, 6D camera poses, and reciprocal 2D-2D matches.
- Trained / supported embodiments
- Embodiment-agnostic 3D visual perception module (monocular, stereo, or multi-view uncalibrated RGB cameras)
- Training datasets
- Habitat-Sim, BlendedMVS, MegaDepth, ARKitScenes, CO3Dv2, StaticThings3D, ScanNet++, Waymo Open Dataset, WildRGB-D (preprocessing scripts and pair lists provided)
- InternalUnreleasedDataset (referenced in upstream hyperparameter commands; not publicly released)
README notes provided dataset pair lists are close to, though not strictly equivalent to, the original training mix.
- Interfaces
- Python API (AsymmetricCroCo3DStereo.from_pretrained, global_aligner)
- Interactive web UI (demo.py, port 7860) and Docker Compose launchers
- Visual localization CLI (visloc.py) and PyTorch DDP training CLI (train.py)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt (512px, DPT head) | safetensors | 2.285 GB | open | 61c57447d7b0 |
| naver/DUSt3R_ViTLarge_BaseDecoder_512_linear (512px, Linear head) | safetensors | 2.129 GB | open | 2e8a5a23f219 |
| naver/DUSt3R_ViTLarge_BaseDecoder_224_linear (224px, Linear head) | safetensors | 2.129 GB | open | fd3c21d15d56 |
All three checkpoints verified via Hugging Face API (ungated model.safetensors); equivalent .pth weights are also hosted on download.europe.naverlabs.com. Subject to CC-BY-NC-SA-4.0 plus upstream training dataset licenses.
Is this for you?
- Use cases
- Uncalibrated pairwise and multi-view 3D point cloud and depth map reconstruction
- Camera focal length and relative/absolute 6D pose estimation via global alignment (PointCloudOptimizer)
- Reciprocal 2D-2D pixel correspondence matching from aligned 3D pointmaps
- Visual localization (visloc.py) and serving as a pretrained backbone for downstream models such as MASt3R
- Known limitations
- Code, croco submodule, and checkpoints are licensed under CC-BY-NC-SA-4.0 (Non-Commercial, ShareAlike), prohibiting commercial deployment; checkpoint users must also comply with all upstream training dataset licenses.
- Pre-training included an InternalUnreleasedDataset alongside public datasets (CO3Dv2, ARKitScenes, ScanNet++, BlendedMVS, MegaDepth, Waymo, Habitat, StaticThings3D), so exact full pre-training reproduction is not possible.
- Multi-view reconstruction of more than two images requires iterative global alignment optimization (e.g., 300 iterations in README example), which scales poorly to very large image collections compared to later MASt3R-SfM.
- Open Source Robotics has not installed DUSt3R, downloaded checkpoints, run inference or training, or reproduced paper benchmarks.
- Safety and limitations
- CC-BY-NC-SA-4.0 non-commercial share-alike license plus upstream dataset terms (including non-commercial CO3Dv2, ARKitScenes, ScanNet++, and Waymo Open Dataset) preclude commercial robotics deployment.
- Perception-only 3D geometry model outputting scale-ambiguous pointmaps and camera poses; does not output robot control actions or metric-guaranteed safety bounds.
- No independent runtime, VRAM, or 3D accuracy verification was performed by Open Source Robotics.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU recommended (default device='cuda'; Docker CPU fallback supported; optional compiled RoPE CUDA kernels for faster runtime)
- Precision:
fp32 - Edge / consumer capable: No
Exact minimum VRAM and inference latency are not specified in the README.
- Fine-tuning
- Supported: Yes
- Methods: 3-stage PyTorch DDP full training/fine-tuning (224 linear -> 512 linear -> 512 DPT via torchrun train.py)
- Recipe: Upstream fine-tuning documentation
README documents a 4-GPU CO3Dv2 subset demo and 8-GPU full hyperparameter training commands.
- Host computer
NVIDIA CUDA GPU recommended (~2.0–2.1 GB checkpoint; CPU Docker mode available); 4-GPU demo and 8-GPU full training commands documented via torchrun.
- Operating system
- Linux
Conda/CUDA, Docker, and torchrun workflows target Linux.
[1] · Editorial assessment
- Dependencies
- Python 3.11, CMake, PyTorch, TorchVision (CUDA 12.1 example)
- croco git submodule (optional RoPE CUDA extension)
- requirements.txt and optional requirements_optional.txt / Docker + NVIDIA Container Toolkit
- Skills
- Python and PyTorch (CUDA extension compilation for optional RoPE kernels)
- Multi-view 3D geometry and point-cloud alignment
Inferred from installation, global alignment API, and training scripts.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Clone naver/dust3r recursively (including the croco submodule) and create a Python 3.11 environment with PyTorch, TorchVision, and CUDA.
- Install requirements.txt (and optionally requirements_optional.txt for visloc.py, HEIC, or pyrender) and optionally compile the RoPE CUDA kernels in croco/models/curope/.
- Load a checkpoint automatically via AsymmetricCroCo3DStereo.from_pretrained('naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt') or launch demo.py (locally or via Docker).
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| CVPR 2024 monocular/multi-view depth & relative pose estimation (arXiv:2312.14132) | Upstream abstract reports state-of-the-art results on monocular/multi-view depth estimation and relative camera pose estimation (numeric table is in the CVPR 2024 paper rather than the README). | Upstream-reported in CVPR 2024 / arXiv:2312.14132 abstract; OSR has not run inference, visual localization, or benchmark evaluations. |
Project notes
Architecture, checkpoints, and workflows
DUSt3R uses an asymmetric Vision Transformer (AsymmetricCroCo3DStereo) initialized from CroCo v2 (CroCo_V2_ViTLarge_BaseDecoder), combining a ViT-Large encoder (dim=1024, depth=24, 16 heads) with a cross-attending ViT-Base decoder (dim=768, depth=12, 12 heads) and RoPE100 positional embeddings (with optional compiled CUDA kernels in croco/models/curope/). Given an image pair, it predicts dense 3D pointmaps expressed in the first view's coordinate frame along with per-pixel confidence maps; for collections of more than two images, global_aligner (PointCloudOptimizer) aligns all pairwise pointmaps into a common frame to recover focal lengths, 6D camera poses, depth maps, and reciprocal 2D-2D matches. Because DUSt3R is evaluated with a scale-shift-invariant criterion (Regr3D_ScaleShiftInv)—whereas the README notes that successor MASt3R adds metric pointmaps—recovered geometry should not be treated as guaranteed metric physical distances without external scale calibration. [1] [5]
Three ungated safetensors checkpoints are published on Hugging Face (with .pth mirrors on download.europe.naverlabs.com): naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt (2.285 GB, DPT head), naver/DUSt3R_ViTLarge_BaseDecoder_512_linear (2.129 GB, linear head), and naver/DUSt3R_ViTLarge_BaseDecoder_224_linear (2.129 GB, 224px linear head). The README documents a 3-stage training schedule (224 linear → 512 linear → 512 DPT) using torchrun on 8 GPUs (plus a 4-GPU CO3Dv2 subset demo). Minimum inference VRAM and workstation global-alignment runtime are not specified in the README or established by this review, and checkpoint file sizes do not establish runtime GPU memory requirements. [1] [4]
Upstream-reported benchmarks
The CVPR 2024 abstract (arXiv:2312.14132) reports that DUSt3R unifies monocular and binocular reconstruction and sets new state-of-the-art results on monocular/multi-view depth estimation and relative camera pose estimation (detailed numeric tables are in the paper rather than the README). All claims are upstream-reported; Open Source Robotics has not installed DUSt3R, downloaded weights, run training, fine-tuning, or inference, or reproduced any benchmark evaluations. [1] [5]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Jul 1, 2025 [3]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / 4c24a6ebf04809f2cfe59915e51779c8984aaa40 · software repository
Separate license matrix
Include DUSt3R as a foundational 3D scene-reconstruction model with public weights and training scripts, classified as source-available due to its CC-BY-NC-SA-4.0 non-commercial license and upstream non-commercial dataset terms.
| Layer | Availability / license |
|---|---|
| Code | available · CC-BY-NC-SA-4.0 Licensed by Naver Corporation under CC-BY-NC-SA-4.0 (non-commercial, share-alike); NOTICE confirms croco submodule is also CC-BY-NC-SA-4.0. [1] [2] [3] [6] |
| Model weights | available · CC-BY-NC-SA-4.0 plus upstream training dataset licenses Three ungated safetensors checkpoints on Hugging Face (no license tag in HF card metadata; governed by repo CC-BY-NC-SA-4.0 and upstream dataset licenses per README). [1] [2] [4] |
| Training datasets | available · Mixed upstream dataset licenses (predominantly non-commercial: CC-BY-NC-4.0, CC-BY-NC-SA-4.0, custom NC; plus InternalUnreleasedDataset) Preprocessing scripts and pair-list archives are provided for 9 public datasets, but users must download datasets under their respective licenses, and InternalUnreleasedDataset is not released. [1] |
| Documentation | available · CC-BY-NC-SA-4.0 README covers installation, Docker setup, Python API, training hyperparameters, and dataset preprocessing. [1] [2] |
Licensing and dataset constraints
DUSt3R's code (LICENSE), croco submodule (NOTICE), and released checkpoints are licensed under CC-BY-NC-SA-4.0 (non-commercial, share-alike). In addition, the README explicitly requires checkpoint users to agree to the licenses of all upstream public training datasets and base checkpoints—several of which are themselves non-commercial (CO3Dv2, ARKitScenes, ScanNet++, Waymo Open Dataset)—and notes that training also included an InternalUnreleasedDataset that is not redistributed. Teams requiring commercial robotics deployment cannot use the released DUSt3R code or weights under these terms. [1] [2] [4] [6]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 DUSt3R README
Installation, Docker demo, three checkpoints, Python global alignment API, 9 public datasets + InternalUnreleasedDataset, and 3-stage training hyperparameters.Revision, date & where cited
NAVER Labs Europe · github.com · revision
4c24a6e· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 5 more - S002 DUSt3R LICENSE (CC-BY-NC-SA-4.0)
DUSt3R is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0.Revision, date & where cited
Naver Corporation · github.com · revision
4c24a6e· Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S003 GitHub repository metadata and pinned commit
Last commit 2025-07-01; no GitHub releases; not archived.Revision, date & where cited
naver/dust3r / GitHub · github.com · revision
4c24a6e· main / · Accessed Nov 21, 2026
Used for: Overview, What we checked, Separate license matrix - S004 Hugging Face DUSt3R checkpoint metadata (512_dpt, 512_linear, 224_linear)
Verified three ungated safetensors checkpoints: 512_dpt (2.128 GB, sha 61c57447d7b0adc8a1a30b2b0adec7a8935aa2a3), 512_linear (1.983 GB, sha 2e8a5a23f219560c8cf810c0079275305e55bfea), 224_linear (1.983 GB, sha fd3c21d15d5653493b7d8ae71e268fa60532540c).Revision, date & where cited
NAVER Labs Europe / Hugging Face · huggingface.co · API metadata 2026-11-21 · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Is this for you?, Compute & fine-tuning and 2 more - S005 DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024 / arXiv:2312.14132)
Introduces uncalibrated pairwise pointmap regression with global alignment, unifying monocular/multi-view depth, pixel matching, and relative/absolute camera pose estimation.Revision, date & where cited
NAVER Labs Europe (CVPR 2024 / arXiv) · arxiv.org · arXiv:2312.14132v3 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Upstream-reported benchmarks and 1 more - S006 DUSt3R NOTICE (subcomponent licenses)
Documents naver/croco subcomponent under Creative Commons Attribution-NonCommercial-ShareAlike 4.0.Revision, date & where cited
Naver Corporation · github.com · revision
4c24a6e· Accessed Nov 21, 2026
Used for: Separate license matrix
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for DUSt3R (3 verified HF safetensors checkpoints, CC-BY-NC-SA-4.0 non-commercial terms).