MegaPose
Robotics modelApache-2.0 Render & Compare 6D pose estimation pipeline for novel objects using RGB or RGB-D images, camera intrinsics, a 2D bounding box, and a CAD mesh. [1] [2] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Robotics perception researchers and engineers needing commercially permissive 6D novel-object pose estimation; Manipulation teams integrating CAD-based pose estimation with external 2D detectors [1] [2]
- Parameters
- Not explicitly stated as a scalar in README or arXiv:2212.06870 abstract [4]
- Embodiments
- Embodiment-agnostic perception module (monocular RGB or RGB-D camera with calibrated intrinsics) [1]
- Source rights
- Some licenses are unresolved. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- Apache-2.0 · license not fully assessed for model weights, datasets · Details by part
Overview
MegaPose (megapose6d/megapose6d, CoRL 2022, arXiv:2212.06870) is a Render & Compare 6D object pose estimation pipeline for novel objects unseen during training. At pinned commit f3b8e1247f133f3d098833a251b8f2d744c03e1f (last commit December 2024), it pairs Apache-2.0 source code from Inria and NVIDIA with three publicly downloadable PyTorch checkpoints and a 2-million-image synthetic training dataset. [1] [2] [3] [5]
Model card summary
- Architecture
Two-stage Render & Compare pipeline using vanilla ResNet-34 backbones: a coarse pose network scoring rendered multi-view CAD hypotheses (sphere_26views) against the observed RGB crop, followed by an iterative multi-view refiner (4 views with normals, RGB or RGB-D) predicting 9D pose updates, with optional top-K multi-hypothesis refinement and depth ICP.
- Parameter count
Not explicitly stated as a scalar in README or arXiv:2212.06870 abstract; three vanilla ResNet-34 PyTorch checkpoints (coarse-rgb 0.082 GB, refiner-rgb 0.083 GB, refiner-rgbd 0.083 GB)
Backbone type (vanilla_resnet34) and checkpoint file sizes verified from Inria archive; tensors were not loaded to count parameters.
[4] · Editorial assessment
- Input modalities
- rgbrgbd
RGB image required (4:3 aspect ratio recommended); depth map in mm optional; also requires camera intrinsics K, 2D bounding box, and CAD mesh.
- Output modalities
- 6d-pose
Per-object 6D pose TWO (quaternion + 3D translation in camera frame).
- Action space
Perception-only model (no robot action output); outputs per-object 6D pose TWO (quaternion and 3D translation in camera frame) requiring an external planner or controller.
- Trained / supported embodiments
- Embodiment-agnostic perception module (monocular RGB or RGB-D camera with calibrated intrinsics)
- Training datasets
- MegaPose synthetic BlenderProc dataset (2M images, >20,000 objects) in webdataset format: gso_1M (Google Scanned Objects, 1M images) and shapenet_1M (ShapeNetCoreV2, 1M images)
- Interfaces
- Python CLI (megapose.scripts.run_inference_on_example, megapose.scripts.download)
- Python package (megapose)
- Docker container (ylabbe/megapose6d)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| coarse-rgb-906902141 | pytorch | 0.082 GB | open | 2023-01-03 |
| refiner-rgb-653307694 | pytorch | 0.083 GB | open | 2023-01-03 |
| refiner-rgbd-288182519 | pytorch | 0.083 GB | open | 2023-01-03 |
Three PyTorch checkpoint directories verified HTTP 200 on Inria archive (checkpoint.pth.tar sizes 82–83 MB, dated 2023-01-03). Combined into 4 Model Zoo configurations (RGB, RGBD, RGB-multi-hypothesis, RGB-multi-hypothesis-icp).
Is this for you?
- Use cases
- RGB-only 6D pose estimation of novel objects (megapose-1.0-RGB and megapose-1.0-RGB-multi-hypothesis)
- RGB-D 6D pose estimation with multi-hypothesis scoring and optional ICP depth refinement (megapose-1.0-RGBD and megapose-1.0-RGB-multi-hypothesis-icp)
- Training or benchmarking pose estimators on the 2M-image BlenderProc GSO/ShapeNet webdataset
- Known limitations
- Requires a 3D CAD mesh of the target object (in millimeters, .ply or .obj), camera intrinsics K, and an external 2D object bounding box at inference time.
- Does not perform 2D object detection or frame-to-frame temporal tracking out of the box.
- External Inria checkpoint archive does not contain an artifact-specific LICENSE file inside the model directories, and the GSO/ShapeNet training datasets require users to comply with original dataset terms.
- Open Source Robotics has not installed MegaPose, downloaded weights, or reproduced BOP/YCB-Video benchmarks.
- Safety and limitations
- Perception-only pose estimator that outputs object-in-camera 6D poses (quaternion + translation) and no robot control commands; grasp planning and collision checking must be handled externally.
- Requires accurate camera intrinsics and millimeter-scale object meshes; scale mismatches or poor 2D bounding boxes degrade depth initialization and pose accuracy.
- Before downloading the preprocessed GSO/ShapeNet meshes or synthetic webdatasets, upstream explicitly requires users to ensure they have permission to use the original datasets.
- No independent runtime, latency, or accuracy validation was performed by Open Source Robotics.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU (Conda or Docker with nvidia-docker2 / CUDA 11.0+ documented; exact minimum VRAM and latency not stated in README)
- Edge / consumer capable: No
Requires CUDA GPU for PyTorch and rendering; exact VRAM and latency figures are not published in README.
[1] · Editorial assessment
- Fine-tuning
- Supported: No
Designed for zero-shot application to novel objects without retraining; README documents inference and dataset loading rather than a user fine-tuning recipe.
- Host computer
CUDA-capable NVIDIA GPU host on Linux (Docker instructions reference Ubuntu 20.04 and CUDA 11.0+; training configs show 32-GPU runs).
Inferred from Docker/Conda setup and config.yaml training metadata.
- Operating system
- Linux
Ubuntu/Linux Docker and Conda environment setup documented.
[1] · Editorial assessment
- Dependencies
- Conda (conda/environment_full.yaml) or Docker with NVIDIA container toolkit
- PyTorch and Panda3D / rendering dependencies
- Pre-trained models from Inria archive
- External 2D object detector for bounding boxes
- Skills
- Python, PyTorch, and Conda/Docker
- Camera calibration and 3D CAD mesh preparation (.ply/.obj in mm)
Inferred from README installation and inference tutorial.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Clone megapose6d with submodules (git submodule update --init) and optionally set MEGAPOSE_DATA_DIR.
- Create the Conda environment from conda/environment_full.yaml (or pull the ylabbe/megapose6d Docker image) and run pip install -e .
- Download pre-trained models via python -m megapose.scripts.download --megapose_models and example data via --example_data.
- Run python -m megapose.scripts.run_inference_on_example barbecue-sauce --run-inference (optionally specifying --model from the Model Zoo).
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| ModelNet, YCB-Video, and 7 Core BOP Challenge Datasets (CoRL 2022, arXiv:2212.06870) | Upstream reports state-of-the-art novel-object pose estimation on ModelNet and YCB-Video, and competitive performance across the 7 core BOP datasets without target-object retraining. | Upstream-reported paper abstract summary; exact per-dataset numerical tables are in the CoRL 2022 paper PDF and were not independently reproduced by OSR. |
Project notes
Architecture, checkpoints, and input requirements
MegaPose operates as a perception-only two-stage pipeline backed by vanilla_resnet34 networks (verified in each checkpoint's config.yaml). Given an RGB image (with optional depth in millimeters), camera intrinsics K, an external 2D bounding box, and a millimeter-scale 3D CAD mesh (.ply or .obj), a coarse pose network scores rendered multi-view CAD hypotheses (sphere_26views) and an iterative refiner predicts pose updates to output per-object 6D poses (TWO: quaternion and 3D translation in camera frame). [1] [4] [5]
Three checkpoint directories were verified reachable on the Inria archive: coarse-rgb-906902141 (checkpoint.pth.tar, 82 MB), refiner-rgb-653307694 (83 MB), and refiner-rgbd-288182519 (83 MB), all dated January 2023. The Model Zoo combines these into four inference modes—megapose-1.0-RGB, megapose-1.0-RGBD, megapose-1.0-RGB-multi-hypothesis, and megapose-1.0-RGB-multi-hypothesis-icp—with upstream recommending megapose-1.0-RGB-multi-hypothesis for RGB input and megapose-1.0-RGB-multi-hypothesis-icp (which adds depth-based ICP refinement) for RGB-D input. Setup uses either conda/environment_full.yaml or the ylabbe/megapose6d Docker image on a CUDA-capable Linux host; the supplied README does not establish minimum GPU VRAM or inference latency, and compatibility with modern CUDA/PyTorch toolchains has not been independently tested. [1] [4]
Upstream-reported benchmarks
In the CoRL 2022 paper abstract (arXiv:2212.06870), the authors report state-of-the-art novel-object pose estimation on ModelNet and YCB-Video and competitive results across the 7 core BOP challenge datasets without target-object retraining. All benchmark claims are upstream-reported; Open Source Robotics has not installed MegaPose, downloaded the Inria checkpoints, run training, fine-tuning, or inference, or reproduced BOP evaluations. [5]
Apache-2.0 code and dataset terms
Repository source code is licensed under the permissive Apache License 2.0 (LICENSE). However, the Inria model directories do not carry a separate artifact-specific license file, and the README explicitly requires users to verify they have rights to the original Google Scanned Objects and ShapeNet datasets before downloading the gso_1M and shapenet_1M synthetic webdatasets or preprocessed meshes. [1] [2] [4]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Dec 12, 2024 [3]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / f3b8e1247f133f3d098833a251b8f2d744c03e1f · software repository
Separate license matrix
Include MegaPose as a partially open novel-object 6D pose pipeline: repository code is Apache-2.0, while Inria checkpoints lack a standalone license file and GSO/ShapeNet synthetic datasets require original upstream dataset rights.
| Layer | Availability / license |
|---|---|
| Code | available · Apache-2.0 Inria & NVIDIA Apache-2.0 license for repository source code. [1] [2] |
| Model weights | available · License not fully assessed Three public PyTorch checkpoint directories verified HTTP 200 on Inria archive; no separate artifact-specific license file is present inside the archive directories (repository code is Apache-2.0). [1] [4] |
| Training datasets | available · License not fully assessed 2M synthetic BlenderProc images (gso_1M and shapenet_1M) hosted on Inria archive; README explicitly warns users to ensure they have permission to use the original GSO and ShapeNet datasets before downloading. [1] |
| Documentation | available · Apache-2.0 README, inference tutorial, and visualization notebooks included in repository. [1] [2] |
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 MegaPose README
Model zoo, Conda/Docker installation, inference tutorial, Inria checkpoint/dataset URLs, and Apache-2.0 statement.Revision, date & where cited
megapose6d/megapose6d contributors · github.com · revision
f3b8e12· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 5 more - S002 MegaPose Apache-2.0 License
Apache License 2.0 covering repository source code.Revision, date & where cited
Inria & NVIDIA Corporation · github.com · revision
f3b8e12· Accessed Nov 21, 2026
Used for: Overview, Apache-2.0 code and dataset terms, Separate license matrix - S003 GitHub repository metadata and pinned commit
Last commit 2024-12-12; no GitHub releases; not archived.Revision, date & where cited
megapose6d/megapose6d / GitHub · github.com · revision
f3b8e12· master / · Accessed Nov 21, 2026
Used for: Overview, What we checked - S004 Inria MegaPose pre-trained models archive and config.yaml
Verified 3 model directories (coarse-rgb-906902141 82 MB, refiner-rgb-653307694 83 MB, refiner-rgbd-288182519 83 MB), each using vanilla_resnet34 backbones.Revision, date & where cited
Inria / Yann Labbe · paris.inria.fr · Directory listing & config.yaml, 2026-11-21 · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Is this for you?, Compute & fine-tuning and 3 more - S005 MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare (arXiv:2212.06870)
Coarse-to-fine Render & Compare 6D pose estimation of unseen objects from RGB/RGB-D + 2D bbox + CAD mesh; trained on 2M synthetic GSO/ShapeNet images; evaluated on ModelNet, YCB-Video, and 7 core BOP datasets.Revision, date & where cited
Inria & NVIDIA (CoRL 2022 / arXiv) · arxiv.org · arXiv:2212.06870v1 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Compute & fine-tuning, Upstream-reported benchmarks and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for MegaPose (Apache-2.0 code, 3 verified Inria checkpoints).