MASt3R (Matching And Stereo 3D Reconstruction)
Robotics model3D-grounded dense image matching and metric stereo reconstruction transformer from NAVER Labs Europe extending DUSt3R with a 24D local feature head, fast reciprocal matching, visual localization, and MASt3R-SfM sparse global alignment. [1] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- 3D vision, SLAM, and robotics localization researchers building SfM, relocalization, or wide-baseline matching pipelines; Academic and non-commercial teams evaluating metric pointmap and local-descriptor models [1] [2] [6]
- Parameters
- Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder + CatMLP+DPT head [1] [4]
- Embodiments
- Embodiment-agnostic 3D visual perception and localization module (monocular, stereo, or multi-view RGB cameras) [1] [5]
- Source rights
- Named licenses are documented by part. Includes noncommercial terms. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- CC-BY-NC-SA-4.0 (with BSD/MIT third-party notices) (noncommercial), CC-BY-NC-SA-4.0 plus restrictive CHECKPOINTS_NOTICE dataset licenses (noncommercial), Mixed restrictive upstream dataset licenses (including non-commercial and no-derivatives terms; InternalUnreleasedDataset not released) (noncommercial), CC-BY-NC-SA-4.0 (noncommercial) · Details by part
Overview
MASt3R (naver/mast3r, ECCV 2024, arXiv:2406.09756) is a 3D matching and metric stereo reconstruction model from NAVER Labs Europe that builds on DUSt3R by adding a dense local feature head, metric-scale pointmap regression, fast reciprocal matching, and MASt3R-SfM (3DV 2025) sparse global alignment. At pinned commit f5209afc300cec36239a7ac992263f36847bbba0 (last updated June 2025), the repository includes inference demos, ASMK-based image retrieval, visual localization scripts (visloc.py), and PyTorch DDP training code. [1] [3] [5]
Model card summary
- Architecture
AsymmetricMASt3R transformer initialized from DUSt3R_ViTLarge_BaseDecoder_512_dpt (ViT-Large encoder: dim 1024, depth 24, 16 heads; ViT-Base decoder: dim 768, depth 12, 12 heads; RoPE100) augmented with a CatMLP+DPT head outputting metric-scale 3D pointmaps, dense 24D local matching descriptors (pts3d+desc24), and dual confidence maps, paired with fast reciprocal nearest-neighbor matching and MASt3R-SfM sparse global alignment.
- Base model
DUSt3R (DUSt3R_ViTLarge_BaseDecoder_512_dpt)
Upstream training initializes AsymmetricMASt3R from DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth.
- Parameter count
Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder + CatMLP+DPT head; model.safetensors is 2.755 GB)
Verified exact Hugging Face model.safetensors size: 2,754,661,648 bytes (2.755 GB).
- Input modalities
- rgb
Uncalibrated RGB image pairs or collections (512px max side for standard MASt3R; 518px for optional DUNE+MASt3R).
- Output modalities
- 6d-pose
Outputs metric 3D pointmaps, dense 24D local matching descriptors, 2D-2D correspondences, and 6D camera poses via sparse global alignment (MASt3R-SfM) or PnP visual localization (visloc.py).
- Action space
Perception-only 3D matching and reconstruction model (no robot action output); produces metric 3D pointmaps, 24D local descriptors, 2D-2D correspondences, and 6D camera poses.
- Trained / supported embodiments
- Embodiment-agnostic 3D visual perception and localization module (monocular, stereo, or multi-view RGB cameras)
- Training datasets
- Habitat, BlendedMVS, MegaDepth, ARKitScenes, CO3Dv2, StaticThings3D, ScanNet++, TartanAir, UnrealStereo4K, VirtualKITTI, WildRGB-D, Niantic Map-free, DL3DV
- InternalUnreleasedDataset (referenced in upstream hyperparameter commands; not publicly released)
Training mix combines 13 named datasets plus InternalUnreleasedDataset; CHECKPOINTS_NOTICE documents inherited CroCo/DUSt3R and MASt3R dataset terms.
- Interfaces
- Python API (AsymmetricMASt3R.from_pretrained, fast_reciprocal_NNs)
- Interactive MASt3R-SfM web UI (demo.py, port 7860) and Docker Compose launchers
- Visual localization CLI (visloc.py) and PyTorch DDP training CLI (train.py)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric (512px, CatMLP+DPT metric) | safetensors | 2.755 GB | open | 06e7259f34c3 |
Primary MASt3R checkpoint verified via Hugging Face API (ungated model.safetensors, 2.565 GB); equivalent .pth, optional retrieval model (.pth + codebook .pkl), and partial-compatibility DUNE+MASt3R checkpoints (separate Project NLE DUNE LICENSE.txt) are hosted on download.europe.naverlabs.com.
Is this for you?
- Use cases
- Wide-baseline 2D-2D correspondence matching via fast_reciprocal_NNs over dense 24D local descriptors
- Metric-scale stereo 3D pointmap reconstruction and multi-view MASt3R-SfM sparse global alignment
- 6D visual localization on Aachen-Day-Night, InLoc, Cambridge Landmarks, and 7-Scenes (visloc.py)
- Optional image-pair retrieval using ASMK + training-free retrieval checkpoint
- Known limitations
- Code is CC-BY-NC-SA-4.0 (non-commercial, share-alike), and CHECKPOINTS_NOTICE binds checkpoint use to numerous restrictive upstream dataset licenses—explicitly flagging the Map-free dataset license as very restrictive alongside IndoorVL CC-BY-NC-ND-4.0.
- Training used 14 datasets including InternalUnreleasedDataset and NianticMapFree; upstream states not all training datasets were released.
- Optional DUNE+MASt3R encoder checkpoints are under a separate Project NLE DUNE LICENSE.txt and have limited compatibility with the rest of the codebase; kapture_mast3r_mapping.py and demo_glomap.py are explicitly labeled untested 'toys'.
- Open Source Robotics has not installed MASt3R, downloaded checkpoints, run inference or visual localization, or reproduced paper benchmarks.
- Safety and limitations
- Layered non-commercial and restrictive dataset licensing (CC-BY-NC-SA-4.0 plus CHECKPOINTS_NOTICE covering Niantic Map-free, IndoorVL CC-BY-NC-ND-4.0, and others) bars commercial robotics deployment.
- Perception-only matching and 3D reconstruction module without robot control outputs or metric-guaranteed safety bounds.
- No independent runtime, VRAM, or visual-localization accuracy verification was performed by Open Source Robotics.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU recommended (default device='cuda'; Docker CPU fallback supported; optional compiled RoPE CUDA kernels and Cython ASMK for retrieval)
- Precision:
fp32 - Edge / consumer capable: No
Exact minimum VRAM and inference latency are not specified in the README.
- Fine-tuning
- Supported: Yes
- Methods: PyTorch DDP full training/fine-tuning from DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth via torchrun train.py
- Recipe: Upstream fine-tuning documentation
README documents a 4-GPU CO3Dv2 subset demo and 8-GPU full metric+matching training command.
- Host computer
NVIDIA CUDA GPU recommended (2.755 GB checkpoint file; Docker CPU fallback supported); 4-GPU demo and 8-GPU full training commands documented via torchrun.
- Operating system
- Linux
Conda/CUDA, Cython ASMK, Docker, and torchrun workflows target Linux.
[1] · Editorial assessment
- Dependencies
- Python 3.11, CMake, PyTorch, TorchVision (CUDA 12.1 example)
- dust3r and croco git submodules
- ASMK (Cython build from github.com/jenicek/asmk) for retrieval-based pairing
- requirements.txt, dust3r/requirements.txt, optional dust3r/requirements_optional.txt (poselib/visloc)
- Skills
- Python, PyTorch, and Cython/CUDA extension builds (ASMK, optional RoPE kernels)
- 3D matching, SfM, and PnP visual localization workflows
Inferred from installation, ASMK compilation, and visloc.py instructions.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Clone naver/mast3r recursively (including dust3r and croco submodules) and create a Python 3.11 environment with PyTorch, TorchVision, and CUDA.
- Install requirements.txt and dust3r/requirements.txt, compile and install ASMK (Cython) for retrieval pairing, and optionally compile RoPE CUDA kernels in dust3r/croco/models/curope/.
- Load via AsymmetricMASt3R.from_pretrained('naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric') or launch demo.py (MASt3R-SfM sparse global alignment) or visloc.py.
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Map-free Visual Relocalization (VCRE AUC, arXiv:2406.09756) | Upstream abstract reports +30% absolute improvement in VCRE AUC over the best prior published methods on Map-free localization (ECCV 2024). | Upstream-reported in ECCV 2024 / arXiv:2406.09756 abstract; OSR has not run visloc.py or reproduced localization metrics. |
Project notes
Architecture, checkpoints, and workflows
Initialized from DUSt3R_ViTLarge_BaseDecoder_512_dpt, AsymmetricMASt3R retains the ViT-Large encoder (dim=1024, depth=24) and ViT-Base cross-attending decoder (dim=768, depth=12) with RoPE100 embeddings, while replacing the head with a CatMLP+DPT head outputting metric 3D pointmaps, dense 24D local matching descriptors (pts3d+desc24), and dual confidence maps. A fast reciprocal nearest-neighbor scheme (fast_reciprocal_NNs) avoids quadratic dense matching overhead, and demo.py integrates MASt3R-SfM (retrieval-based pairing via compiled Cython ASMK + sparse global alignment) to scale beyond DUSt3R's dense pair optimizer. The repo also includes visloc.py for 6D visual localization on Aachen-Day-Night, InLoc, Cambridge Landmarks, and 7-Scenes; kapture_mast3r_mapping.py and demo_glomap.py are explicitly described as "toys" that were not extensively tested and may fail on edge cases. [1] [5]
The primary checkpoint naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric is hosted ungated on Hugging Face (model.safetensors, 2.755 GB, revision 06e7259f34c3060f322df5cb0c7b9094f57e41fc), with .pth and companion retrieval files (trainingfree.pth and codebook.pkl) hosted on download.europe.naverlabs.com. Partial support for DUNE+MASt3R checkpoints (dunemast3r_cvpr25_vitbase.pth / vitsmall.pth) is also documented under a separate Project NLE DUNE LICENSE.txt. Minimum inference VRAM and matching/SfM latency are unspecified in the README and independently unmeasured (checkpoint file size does not establish runtime GPU memory requirements), and upstream explicitly states that not all training datasets were publicly released, limiting full training reproducibility. [1] [4]
Upstream-reported benchmarks
The ECCV 2024 abstract (arXiv:2406.09756) reports that MASt3R outperforms prior published methods by +30% absolute improvement in VCRE AUC on the Map-free visual localization benchmark, alongside strong matching and multi-view localization results. All performance claims are upstream-reported; Open Source Robotics has not installed MASt3R, downloaded weights, run training, fine-tuning, or inference, or reproduced any visual localization benchmarks. [1] [5]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Jun 30, 2025 [3]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / f5209afc300cec36239a7ac992263f36847bbba0 · software repository
Separate license matrix
Include MASt3R as a leading 3D-grounded matching, SfM, and visual-localization model with public weights and training code, classified as source-available due to CC-BY-NC-SA-4.0 and restrictive upstream dataset terms in CHECKPOINTS_NOTICE.
| Layer | Availability / license |
|---|---|
| Code | available · CC-BY-NC-SA-4.0 (with BSD/MIT third-party notices) Root LICENSE and dust3r/croco submodules are CC-BY-NC-SA-4.0 (non-commercial, share-alike); NOTICE lists BSD PyTorch/Caffe2 and MIT gtolias/how excerpts. [1] [2] [3] |
| Model weights | available · CC-BY-NC-SA-4.0 plus restrictive CHECKPOINTS_NOTICE dataset licenses Ungated Hugging Face safetensors checkpoint governed by CC-BY-NC-SA-4.0 and CHECKPOINTS_NOTICE (including very restrictive Map-free dataset terms and IndoorVL CC-BY-NC-ND-4.0). Optional DUNE checkpoints use separate Project NLE DUNE LICENSE.txt. [1] [2] [4] [6] |
| Training datasets | available · Mixed restrictive upstream dataset licenses (including non-commercial and no-derivatives terms; InternalUnreleasedDataset not released) README states not all training datasets were released; users must download public datasets separately under their individual terms. [1] [6] |
| Documentation | available · CC-BY-NC-SA-4.0 README documents installation, ASMK build, MASt3R-SfM, DUNE usage, training hyperparameters, and visual localization commands. [1] [2] |
Licensing and CHECKPOINTS_NOTICE obligations
MASt3R's source code (LICENSE) and its dust3r and croco submodules are licensed under CC-BY-NC-SA-4.0 (non-commercial, share-alike). Critically, checkpoint usage is additionally subject to separate upstream dataset terms: the README requires agreeing to all training dataset licenses and explicitly flags the Map-free dataset license as "very restrictive," while its training hyperparameter command also lists InternalUnreleasedDataset as an unreleased training input. In addition, CHECKPOINTS_NOTICE documents inherited CroCo/DUSt3R obligations including 3D_Street_View (non-commercial) and IndoorVL (modified CC-BY-NC-ND-4.0). The supplied terms do not permit commercial deployment; any alternative commercial permissions would need to cover all applicable rights holders, whose availability has not been established by this review. [1] [2] [4] [6]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 MASt3R README
Installation (including ASMK), MASt3R/retrieval/DUNE checkpoints, MASt3R-SfM overview, matching API, 14-dataset training hyperparameters, and visual localization commands.Revision, date & where cited
NAVER Labs Europe · github.com · revision
f5209af· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 5 more - S002 MASt3R LICENSE (CC-BY-NC-SA-4.0)
MASt3R is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0.Revision, date & where cited
Naver Corporation · github.com · revision
f5209af· Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S003 GitHub repository metadata and pinned commit
Last commit 2025-06-30; no GitHub releases; not archived.Revision, date & where cited
naver/mast3r / GitHub · github.com · revision
f5209af· main / · Accessed Nov 21, 2026
Used for: Overview, What we checked, Separate license matrix - S004 Hugging Face MASt3R checkpoint metadata
Verified ungated safetensors checkpoint (2.565 GB / 2,754,661,648 bytes).Revision, date & where cited
NAVER Labs Europe / Hugging Face · huggingface.co · API metadata 2026-11-21 / · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Compute & fine-tuning, Architecture, checkpoints, and workflows and 1 more - S005 Grounding Image Matching in 3D with MASt3R (ECCV 2024 / arXiv:2406.09756)
Augments DUSt3R with a dense local feature head and fast reciprocal matching, reporting +30% absolute VCRE AUC improvement on Map-free localization.Revision, date & where cited
NAVER Labs Europe (ECCV 2024 / arXiv) · arxiv.org · arXiv:2406.09756v1 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Upstream-reported benchmarks and 1 more - S006 MASt3R CHECKPOINTS_NOTICE (upstream dataset license obligations)
Enumerates separate dataset license terms binding checkpoint use, including IndoorVL CC-BY-NC-ND-4.0 (modified) and 3D_Street_View non-commercial terms.Revision, date & where cited
Naver Corporation · github.com · revision
f5209af· Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Separate license matrix
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for MASt3R (verified 2.565 GB HF safetensors checkpoint, CC-BY-NC-SA-4.0 + CHECKPOINTS_NOTICE terms).