Open Source RoboticsRobots, software & learning

DUSt3R (Dense and Unconstrained Stereo 3D Reconstruction)

Robotics model

Pairwise 3D pointmap regression transformer from Naver Labs Europe that reconstructs dense 3D scene geometry, depth maps, pixel matches, and camera poses from uncalibrated RGB images without prior camera intrinsics. [1] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
3D computer vision and robotics perception researchers building uncalibrated reconstruction, visual localization, or scene-mapping pipelines; Academic and non-commercial teams fine-tuning pointmap transformers on custom multi-view datasets [1] [2]
Parameters
Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder [1] [4]
Embodiments
Embodiment-agnostic 3D visual perception module (monocular, stereo, or multi-view uncalibrated RGB cameras) [1] [5]
Source rights
Named licenses are documented by part. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware

Requirements · Research details and dates

On this page

Overview

DUSt3R (naver/dust3r, CVPR 2024, arXiv:2312.14132) is a geometric 3D vision model from NAVER Labs Europe that reformulates dense multi-view stereo reconstruction as pairwise 3D pointmap regression without requiring prior camera intrinsics or poses. At pinned commit 4c24a6ebf04809f2cfe59915e51779c8984aaa40 (last updated July 2025), the repository provides inference code, an interactive Gradio/Docker demo, visual localization (visloc.py), dataset preprocessing scripts, and 3-stage PyTorch DDP training commands. [1] [3] [5]

Model card summary

Architecture

AsymmetricCroCo3DStereo Siamese-encoder / cross-attending-decoder Vision Transformer (ViT-Large encoder: dim 1024, depth 24, 16 heads; ViT-Base decoder: dim 768, depth 12, 12 heads; RoPE100 positional embeddings) initialized from CroCo v2, paired with either a linear or DPT regression head outputting dense 3D pointmaps and per-pixel confidence maps.

[1] [5]

Base model

CroCo v2 (CroCo_V2_ViTLarge_BaseDecoder)

Training step 1 initializes AsymmetricCroCo3DStereo from CroCo_V2_ViTLarge_BaseDecoder.pth.

[1]

Parameter count

Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder; safetensors checkpoints are 2.285 GB for 512_dpt and 2.129 GB for 512_linear / 224_linear)

Verified exact Hugging Face model.safetensors sizes: 512_dpt (2,284,790,056 bytes / 2.285 GB), 512_linear (2,129,459,344 bytes / 2.129 GB), 224_linear (2,129,459,344 bytes / 2.129 GB).

[1] [4]

Input modalities
rgb

One, two, or more uncalibrated RGB images resized to 224 or 512 px max side.

[1]

Output modalities
6d-pose

Outputs dense pairwise 3D pointmaps and confidence maps, from which global alignment recovers 6D camera poses, focal lengths, depth maps, and 2D-2D pixel matches.

[1] [5]

Action space

Perception-only 3D geometry model (no robot action output); produces dense 3D pointmaps, confidence masks, focal lengths, 6D camera poses, and reciprocal 2D-2D matches.

[1] [5]

Trained / supported embodiments
  • Embodiment-agnostic 3D visual perception module (monocular, stereo, or multi-view uncalibrated RGB cameras)

[1] [5]

Training datasets
  • Habitat-Sim, BlendedMVS, MegaDepth, ARKitScenes, CO3Dv2, StaticThings3D, ScanNet++, Waymo Open Dataset, WildRGB-D (preprocessing scripts and pair lists provided)
  • InternalUnreleasedDataset (referenced in upstream hyperparameter commands; not publicly released)

README notes provided dataset pair lists are close to, though not strictly equivalent to, the original training mix.

[1]

Interfaces
  • Python API (AsymmetricCroCo3DStereo.from_pretrained, global_aligner)
  • Interactive web UI (demo.py, port 7860) and Docker Compose launchers
  • Visual localization CLI (visloc.py) and PyTorch DDP training CLI (train.py)

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt (512px, DPT head)safetensors2.285 GBopen61c57447d7b0
naver/DUSt3R_ViTLarge_BaseDecoder_512_linear (512px, Linear head)safetensors2.129 GBopen2e8a5a23f219
naver/DUSt3R_ViTLarge_BaseDecoder_224_linear (224px, Linear head)safetensors2.129 GBopenfd3c21d15d56

All three checkpoints verified via Hugging Face API (ungated model.safetensors); equivalent .pth weights are also hosted on download.europe.naverlabs.com. Subject to CC-BY-NC-SA-4.0 plus upstream training dataset licenses.

[1] [4]

Is this for you?

Use cases
  • Uncalibrated pairwise and multi-view 3D point cloud and depth map reconstruction
  • Camera focal length and relative/absolute 6D pose estimation via global alignment (PointCloudOptimizer)
  • Reciprocal 2D-2D pixel correspondence matching from aligned 3D pointmaps
  • Visual localization (visloc.py) and serving as a pretrained backbone for downstream models such as MASt3R

[1] [5]

Known limitations
  • Code, croco submodule, and checkpoints are licensed under CC-BY-NC-SA-4.0 (Non-Commercial, ShareAlike), prohibiting commercial deployment; checkpoint users must also comply with all upstream training dataset licenses.
  • Pre-training included an InternalUnreleasedDataset alongside public datasets (CO3Dv2, ARKitScenes, ScanNet++, BlendedMVS, MegaDepth, Waymo, Habitat, StaticThings3D), so exact full pre-training reproduction is not possible.
  • Multi-view reconstruction of more than two images requires iterative global alignment optimization (e.g., 300 iterations in README example), which scales poorly to very large image collections compared to later MASt3R-SfM.
  • Open Source Robotics has not installed DUSt3R, downloaded checkpoints, run inference or training, or reproduced paper benchmarks.

[1] [2] [4]

Safety and limitations
  • CC-BY-NC-SA-4.0 non-commercial share-alike license plus upstream dataset terms (including non-commercial CO3Dv2, ARKitScenes, ScanNet++, and Waymo Open Dataset) preclude commercial robotics deployment.
  • Perception-only 3D geometry model outputting scale-ambiguous pointmaps and camera poses; does not output robot control actions or metric-guaranteed safety bounds.
  • No independent runtime, VRAM, or 3D accuracy verification was performed by Open Source Robotics.

[1] [2] [4]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA GPU recommended (default device='cuda'; Docker CPU fallback supported; optional compiled RoPE CUDA kernels for faster runtime)
  • Precision: fp32
  • Edge / consumer capable: No

Exact minimum VRAM and inference latency are not specified in the README.

[1]

Fine-tuning
  • Supported: Yes
  • Methods: 3-stage PyTorch DDP full training/fine-tuning (224 linear -> 512 linear -> 512 DPT via torchrun train.py)
  • Recipe: Upstream fine-tuning documentation

README documents a 4-GPU CO3Dv2 subset demo and 8-GPU full hyperparameter training commands.

[1]

Host computer

NVIDIA CUDA GPU recommended (~2.0–2.1 GB checkpoint; CPU Docker mode available); 4-GPU demo and 8-GPU full training commands documented via torchrun.

[1] [4]

Operating system
  • Linux

Conda/CUDA, Docker, and torchrun workflows target Linux.

[1] · Editorial assessment

Dependencies
  • Python 3.11, CMake, PyTorch, TorchVision (CUDA 12.1 example)
  • croco git submodule (optional RoPE CUDA extension)
  • requirements.txt and optional requirements_optional.txt / Docker + NVIDIA Container Toolkit

[1]

Skills
  • Python and PyTorch (CUDA extension compilation for optional RoPE kernels)
  • Multi-view 3D geometry and point-cloud alignment

Inferred from installation, global alignment API, and training scripts.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone naver/dust3r recursively (including the croco submodule) and create a Python 3.11 environment with PyTorch, TorchVision, and CUDA.
  • Install requirements.txt (and optionally requirements_optional.txt for visloc.py, HEIC, or pyrender) and optionally compile the RoPE CUDA kernels in croco/models/curope/.
  • Load a checkpoint automatically via AsymmetricCroCo3DStereo.from_pretrained('naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt') or launch demo.py (locally or via Docker).

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
CVPR 2024 monocular/multi-view depth & relative pose estimation (arXiv:2312.14132)Upstream abstract reports state-of-the-art results on monocular/multi-view depth estimation and relative camera pose estimation (numeric table is in the CVPR 2024 paper rather than the README).Upstream-reported in CVPR 2024 / arXiv:2312.14132 abstract; OSR has not run inference, visual localization, or benchmark evaluations.

[5]

Project notes

Architecture, checkpoints, and workflows

DUSt3R uses an asymmetric Vision Transformer (AsymmetricCroCo3DStereo) initialized from CroCo v2 (CroCo_V2_ViTLarge_BaseDecoder), combining a ViT-Large encoder (dim=1024, depth=24, 16 heads) with a cross-attending ViT-Base decoder (dim=768, depth=12, 12 heads) and RoPE100 positional embeddings (with optional compiled CUDA kernels in croco/models/curope/). Given an image pair, it predicts dense 3D pointmaps expressed in the first view's coordinate frame along with per-pixel confidence maps; for collections of more than two images, global_aligner (PointCloudOptimizer) aligns all pairwise pointmaps into a common frame to recover focal lengths, 6D camera poses, depth maps, and reciprocal 2D-2D matches. Because DUSt3R is evaluated with a scale-shift-invariant criterion (Regr3D_ScaleShiftInv)—whereas the README notes that successor MASt3R adds metric pointmaps—recovered geometry should not be treated as guaranteed metric physical distances without external scale calibration. [1] [5]

Three ungated safetensors checkpoints are published on Hugging Face (with .pth mirrors on download.europe.naverlabs.com): naver/DUSt3R_ViTLarge_BaseDecoder_512_dpt (2.285 GB, DPT head), naver/DUSt3R_ViTLarge_BaseDecoder_512_linear (2.129 GB, linear head), and naver/DUSt3R_ViTLarge_BaseDecoder_224_linear (2.129 GB, 224px linear head). The README documents a 3-stage training schedule (224 linear → 512 linear → 512 DPT) using torchrun on 8 GPUs (plus a 4-GPU CO3Dv2 subset demo). Minimum inference VRAM and workstation global-alignment runtime are not specified in the README or established by this review, and checkpoint file sizes do not establish runtime GPU memory requirements. [1] [4]

Upstream-reported benchmarks

The CVPR 2024 abstract (arXiv:2312.14132) reports that DUSt3R unifies monocular and binocular reconstruction and sets new state-of-the-art results on monocular/multi-view depth estimation and relative camera pose estimation (detailed numeric tables are in the paper rather than the README). All claims are upstream-reported; Open Source Robotics has not installed DUSt3R, downloaded weights, run training, fine-tuning, or inference, or reproduced any benchmark evaluations. [1] [5]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Jul 1, 2025 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / 4c24a6ebf04809f2cfe59915e51779c8984aaa40 · software repository

Separate license matrix

Include DUSt3R as a foundational 3D scene-reconstruction model with public weights and training scripts, classified as source-available due to its CC-BY-NC-SA-4.0 non-commercial license and upstream non-commercial dataset terms.

LayerAvailability / license
Codeavailable · CC-BY-NC-SA-4.0
Licensed by Naver Corporation under CC-BY-NC-SA-4.0 (non-commercial, share-alike); NOTICE confirms croco submodule is also CC-BY-NC-SA-4.0. [1] [2] [3] [6]
Model weightsavailable · CC-BY-NC-SA-4.0 plus upstream training dataset licenses
Three ungated safetensors checkpoints on Hugging Face (no license tag in HF card metadata; governed by repo CC-BY-NC-SA-4.0 and upstream dataset licenses per README). [1] [2] [4]
Training datasetsavailable · Mixed upstream dataset licenses (predominantly non-commercial: CC-BY-NC-4.0, CC-BY-NC-SA-4.0, custom NC; plus InternalUnreleasedDataset)
Preprocessing scripts and pair-list archives are provided for 9 public datasets, but users must download datasets under their respective licenses, and InternalUnreleasedDataset is not released. [1]
Documentationavailable · CC-BY-NC-SA-4.0
README covers installation, Docker setup, Python API, training hyperparameters, and dataset preprocessing. [1] [2]

Licensing and dataset constraints

DUSt3R's code (LICENSE), croco submodule (NOTICE), and released checkpoints are licensed under CC-BY-NC-SA-4.0 (non-commercial, share-alike). In addition, the README explicitly requires checkpoint users to agree to the licenses of all upstream public training datasets and base checkpoints—several of which are themselves non-commercial (CO3Dv2, ARKitScenes, ScanNet++, Waymo Open Dataset)—and notes that training also included an InternalUnreleasedDataset that is not redistributed. Teams requiring commercial robotics deployment cannot use the released DUSt3R code or weights under these terms. [1] [2] [4] [6]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for DUSt3R (3 verified HF safetensors checkpoints, CC-BY-NC-SA-4.0 non-commercial terms).