Open Source RoboticsRobots, software & learning

MASt3R (Matching And Stereo 3D Reconstruction)

Robotics model

3D-grounded dense image matching and metric stereo reconstruction transformer from NAVER Labs Europe extending DUSt3R with a 24D local feature head, fast reciprocal matching, visual localization, and MASt3R-SfM sparse global alignment. [1] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
3D vision, SLAM, and robotics localization researchers building SfM, relocalization, or wide-baseline matching pipelines; Academic and non-commercial teams evaluating metric pointmap and local-descriptor models [1] [2] [6]
Parameters
Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder + CatMLP+DPT head [1] [4]
Embodiments
Embodiment-agnostic 3D visual perception and localization module (monocular, stereo, or multi-view RGB cameras) [1] [5]
Source rights
Named licenses are documented by part. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware

Requirements · Research details and dates

On this page

Overview

MASt3R (naver/mast3r, ECCV 2024, arXiv:2406.09756) is a 3D matching and metric stereo reconstruction model from NAVER Labs Europe that builds on DUSt3R by adding a dense local feature head, metric-scale pointmap regression, fast reciprocal matching, and MASt3R-SfM (3DV 2025) sparse global alignment. At pinned commit f5209afc300cec36239a7ac992263f36847bbba0 (last updated June 2025), the repository includes inference demos, ASMK-based image retrieval, visual localization scripts (visloc.py), and PyTorch DDP training code. [1] [3] [5]

Model card summary

Architecture

AsymmetricMASt3R transformer initialized from DUSt3R_ViTLarge_BaseDecoder_512_dpt (ViT-Large encoder: dim 1024, depth 24, 16 heads; ViT-Base decoder: dim 768, depth 12, 12 heads; RoPE100) augmented with a CatMLP+DPT head outputting metric-scale 3D pointmaps, dense 24D local matching descriptors (pts3d+desc24), and dual confidence maps, paired with fast reciprocal nearest-neighbor matching and MASt3R-SfM sparse global alignment.

[1] [5]

Base model

DUSt3R (DUSt3R_ViTLarge_BaseDecoder_512_dpt)

Upstream training initializes AsymmetricMASt3R from DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth.

[1]

Parameter count

Not explicitly stated in README or abstract (ViT-Large encoder + ViT-Base decoder + CatMLP+DPT head; model.safetensors is 2.755 GB)

Verified exact Hugging Face model.safetensors size: 2,754,661,648 bytes (2.755 GB).

[1] [4]

Input modalities
rgb

Uncalibrated RGB image pairs or collections (512px max side for standard MASt3R; 518px for optional DUNE+MASt3R).

[1]

Output modalities
6d-pose

Outputs metric 3D pointmaps, dense 24D local matching descriptors, 2D-2D correspondences, and 6D camera poses via sparse global alignment (MASt3R-SfM) or PnP visual localization (visloc.py).

[1] [5]

Action space

Perception-only 3D matching and reconstruction model (no robot action output); produces metric 3D pointmaps, 24D local descriptors, 2D-2D correspondences, and 6D camera poses.

[1] [5]

Trained / supported embodiments
  • Embodiment-agnostic 3D visual perception and localization module (monocular, stereo, or multi-view RGB cameras)

[1] [5]

Training datasets
  • Habitat, BlendedMVS, MegaDepth, ARKitScenes, CO3Dv2, StaticThings3D, ScanNet++, TartanAir, UnrealStereo4K, VirtualKITTI, WildRGB-D, Niantic Map-free, DL3DV
  • InternalUnreleasedDataset (referenced in upstream hyperparameter commands; not publicly released)

Training mix combines 13 named datasets plus InternalUnreleasedDataset; CHECKPOINTS_NOTICE documents inherited CroCo/DUSt3R and MASt3R dataset terms.

[1] [6]

Interfaces
  • Python API (AsymmetricMASt3R.from_pretrained, fast_reciprocal_NNs)
  • Interactive MASt3R-SfM web UI (demo.py, port 7860) and Docker Compose launchers
  • Visual localization CLI (visloc.py) and PyTorch DDP training CLI (train.py)

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric (512px, CatMLP+DPT metric)safetensors2.755 GBopen06e7259f34c3

Primary MASt3R checkpoint verified via Hugging Face API (ungated model.safetensors, 2.565 GB); equivalent .pth, optional retrieval model (.pth + codebook .pkl), and partial-compatibility DUNE+MASt3R checkpoints (separate Project NLE DUNE LICENSE.txt) are hosted on download.europe.naverlabs.com.

[1] [4]

Is this for you?

Use cases
  • Wide-baseline 2D-2D correspondence matching via fast_reciprocal_NNs over dense 24D local descriptors
  • Metric-scale stereo 3D pointmap reconstruction and multi-view MASt3R-SfM sparse global alignment
  • 6D visual localization on Aachen-Day-Night, InLoc, Cambridge Landmarks, and 7-Scenes (visloc.py)
  • Optional image-pair retrieval using ASMK + training-free retrieval checkpoint

[1] [5]

Known limitations
  • Code is CC-BY-NC-SA-4.0 (non-commercial, share-alike), and CHECKPOINTS_NOTICE binds checkpoint use to numerous restrictive upstream dataset licenses—explicitly flagging the Map-free dataset license as very restrictive alongside IndoorVL CC-BY-NC-ND-4.0.
  • Training used 14 datasets including InternalUnreleasedDataset and NianticMapFree; upstream states not all training datasets were released.
  • Optional DUNE+MASt3R encoder checkpoints are under a separate Project NLE DUNE LICENSE.txt and have limited compatibility with the rest of the codebase; kapture_mast3r_mapping.py and demo_glomap.py are explicitly labeled untested 'toys'.
  • Open Source Robotics has not installed MASt3R, downloaded checkpoints, run inference or visual localization, or reproduced paper benchmarks.

[1] [2] [6]

Safety and limitations
  • Layered non-commercial and restrictive dataset licensing (CC-BY-NC-SA-4.0 plus CHECKPOINTS_NOTICE covering Niantic Map-free, IndoorVL CC-BY-NC-ND-4.0, and others) bars commercial robotics deployment.
  • Perception-only matching and 3D reconstruction module without robot control outputs or metric-guaranteed safety bounds.
  • No independent runtime, VRAM, or visual-localization accuracy verification was performed by Open Source Robotics.

[1] [2] [6]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA GPU recommended (default device='cuda'; Docker CPU fallback supported; optional compiled RoPE CUDA kernels and Cython ASMK for retrieval)
  • Precision: fp32
  • Edge / consumer capable: No

Exact minimum VRAM and inference latency are not specified in the README.

[1]

Fine-tuning

README documents a 4-GPU CO3Dv2 subset demo and 8-GPU full metric+matching training command.

[1]

Host computer

NVIDIA CUDA GPU recommended (2.755 GB checkpoint file; Docker CPU fallback supported); 4-GPU demo and 8-GPU full training commands documented via torchrun.

[1] [4]

Operating system
  • Linux

Conda/CUDA, Cython ASMK, Docker, and torchrun workflows target Linux.

[1] · Editorial assessment

Dependencies
  • Python 3.11, CMake, PyTorch, TorchVision (CUDA 12.1 example)
  • dust3r and croco git submodules
  • ASMK (Cython build from github.com/jenicek/asmk) for retrieval-based pairing
  • requirements.txt, dust3r/requirements.txt, optional dust3r/requirements_optional.txt (poselib/visloc)

[1]

Skills
  • Python, PyTorch, and Cython/CUDA extension builds (ASMK, optional RoPE kernels)
  • 3D matching, SfM, and PnP visual localization workflows

Inferred from installation, ASMK compilation, and visloc.py instructions.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone naver/mast3r recursively (including dust3r and croco submodules) and create a Python 3.11 environment with PyTorch, TorchVision, and CUDA.
  • Install requirements.txt and dust3r/requirements.txt, compile and install ASMK (Cython) for retrieval pairing, and optionally compile RoPE CUDA kernels in dust3r/croco/models/curope/.
  • Load via AsymmetricMASt3R.from_pretrained('naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric') or launch demo.py (MASt3R-SfM sparse global alignment) or visloc.py.

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
Map-free Visual Relocalization (VCRE AUC, arXiv:2406.09756)Upstream abstract reports +30% absolute improvement in VCRE AUC over the best prior published methods on Map-free localization (ECCV 2024).Upstream-reported in ECCV 2024 / arXiv:2406.09756 abstract; OSR has not run visloc.py or reproduced localization metrics.

[5]

Project notes

Architecture, checkpoints, and workflows

Initialized from DUSt3R_ViTLarge_BaseDecoder_512_dpt, AsymmetricMASt3R retains the ViT-Large encoder (dim=1024, depth=24) and ViT-Base cross-attending decoder (dim=768, depth=12) with RoPE100 embeddings, while replacing the head with a CatMLP+DPT head outputting metric 3D pointmaps, dense 24D local matching descriptors (pts3d+desc24), and dual confidence maps. A fast reciprocal nearest-neighbor scheme (fast_reciprocal_NNs) avoids quadratic dense matching overhead, and demo.py integrates MASt3R-SfM (retrieval-based pairing via compiled Cython ASMK + sparse global alignment) to scale beyond DUSt3R's dense pair optimizer. The repo also includes visloc.py for 6D visual localization on Aachen-Day-Night, InLoc, Cambridge Landmarks, and 7-Scenes; kapture_mast3r_mapping.py and demo_glomap.py are explicitly described as "toys" that were not extensively tested and may fail on edge cases. [1] [5]

The primary checkpoint naver/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric is hosted ungated on Hugging Face (model.safetensors, 2.755 GB, revision 06e7259f34c3060f322df5cb0c7b9094f57e41fc), with .pth and companion retrieval files (trainingfree.pth and codebook.pkl) hosted on download.europe.naverlabs.com. Partial support for DUNE+MASt3R checkpoints (dunemast3r_cvpr25_vitbase.pth / vitsmall.pth) is also documented under a separate Project NLE DUNE LICENSE.txt. Minimum inference VRAM and matching/SfM latency are unspecified in the README and independently unmeasured (checkpoint file size does not establish runtime GPU memory requirements), and upstream explicitly states that not all training datasets were publicly released, limiting full training reproducibility. [1] [4]

Upstream-reported benchmarks

The ECCV 2024 abstract (arXiv:2406.09756) reports that MASt3R outperforms prior published methods by +30% absolute improvement in VCRE AUC on the Map-free visual localization benchmark, alongside strong matching and multi-view localization results. All performance claims are upstream-reported; Open Source Robotics has not installed MASt3R, downloaded weights, run training, fine-tuning, or inference, or reproduced any visual localization benchmarks. [1] [5]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Jun 30, 2025 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / f5209afc300cec36239a7ac992263f36847bbba0 · software repository

Separate license matrix

Include MASt3R as a leading 3D-grounded matching, SfM, and visual-localization model with public weights and training code, classified as source-available due to CC-BY-NC-SA-4.0 and restrictive upstream dataset terms in CHECKPOINTS_NOTICE.

LayerAvailability / license
Codeavailable · CC-BY-NC-SA-4.0 (with BSD/MIT third-party notices)
Root LICENSE and dust3r/croco submodules are CC-BY-NC-SA-4.0 (non-commercial, share-alike); NOTICE lists BSD PyTorch/Caffe2 and MIT gtolias/how excerpts. [1] [2] [3]
Model weightsavailable · CC-BY-NC-SA-4.0 plus restrictive CHECKPOINTS_NOTICE dataset licenses
Ungated Hugging Face safetensors checkpoint governed by CC-BY-NC-SA-4.0 and CHECKPOINTS_NOTICE (including very restrictive Map-free dataset terms and IndoorVL CC-BY-NC-ND-4.0). Optional DUNE checkpoints use separate Project NLE DUNE LICENSE.txt. [1] [2] [4] [6]
Training datasetsavailable · Mixed restrictive upstream dataset licenses (including non-commercial and no-derivatives terms; InternalUnreleasedDataset not released)
README states not all training datasets were released; users must download public datasets separately under their individual terms. [1] [6]
Documentationavailable · CC-BY-NC-SA-4.0
README documents installation, ASMK build, MASt3R-SfM, DUNE usage, training hyperparameters, and visual localization commands. [1] [2]

Licensing and CHECKPOINTS_NOTICE obligations

MASt3R's source code (LICENSE) and its dust3r and croco submodules are licensed under CC-BY-NC-SA-4.0 (non-commercial, share-alike). Critically, checkpoint usage is additionally subject to separate upstream dataset terms: the README requires agreeing to all training dataset licenses and explicitly flags the Map-free dataset license as "very restrictive," while its training hyperparameter command also lists InternalUnreleasedDataset as an unreleased training input. In addition, CHECKPOINTS_NOTICE documents inherited CroCo/DUSt3R obligations including 3D_Street_View (non-commercial) and IndoorVL (modified CC-BY-NC-ND-4.0). The supplied terms do not permit commercial deployment; any alternative commercial permissions would need to cover all applicable rights holders, whose availability has not been established by this review. [1] [2] [4] [6]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for MASt3R (verified 2.565 GB HF safetensors checkpoint, CC-BY-NC-SA-4.0 + CHECKPOINTS_NOTICE terms).