Open Source RoboticsRobots, software & learning

MegaPose

Robotics model

Apache-2.0 Render & Compare 6D pose estimation pipeline for novel objects using RGB or RGB-D images, camera intrinsics, a 2D bounding box, and a CAD mesh. [1] [2] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Robotics perception researchers and engineers needing commercially permissive 6D novel-object pose estimation; Manipulation teams integrating CAD-based pose estimation with external 2D detectors [1] [2]
Parameters
Not explicitly stated as a scalar in README or arXiv:2212.06870 abstract [4]
Embodiments
Embodiment-agnostic perception module (monocular RGB or RGB-D camera with calibrated intrinsics) [1]
Source rights
Some licenses are unresolved. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
License
Apache-2.0 · license not fully assessed for model weights, datasets · Details by part

Requirements · Research details and dates

On this page

Overview

MegaPose (megapose6d/megapose6d, CoRL 2022, arXiv:2212.06870) is a Render & Compare 6D object pose estimation pipeline for novel objects unseen during training. At pinned commit f3b8e1247f133f3d098833a251b8f2d744c03e1f (last commit December 2024), it pairs Apache-2.0 source code from Inria and NVIDIA with three publicly downloadable PyTorch checkpoints and a 2-million-image synthetic training dataset. [1] [2] [3] [5]

Model card summary

Architecture

Two-stage Render & Compare pipeline using vanilla ResNet-34 backbones: a coarse pose network scoring rendered multi-view CAD hypotheses (sphere_26views) against the observed RGB crop, followed by an iterative multi-view refiner (4 views with normals, RGB or RGB-D) predicting 9D pose updates, with optional top-K multi-hypothesis refinement and depth ICP.

[1] [4] [5]

Parameter count

Not explicitly stated as a scalar in README or arXiv:2212.06870 abstract; three vanilla ResNet-34 PyTorch checkpoints (coarse-rgb 0.082 GB, refiner-rgb 0.083 GB, refiner-rgbd 0.083 GB)

Backbone type (vanilla_resnet34) and checkpoint file sizes verified from Inria archive; tensors were not loaded to count parameters.

[4] · Editorial assessment

Input modalities
rgbrgbd

RGB image required (4:3 aspect ratio recommended); depth map in mm optional; also requires camera intrinsics K, 2D bounding box, and CAD mesh.

[1]

Output modalities
6d-pose

Per-object 6D pose TWO (quaternion + 3D translation in camera frame).

[1]

Action space

Perception-only model (no robot action output); outputs per-object 6D pose TWO (quaternion and 3D translation in camera frame) requiring an external planner or controller.

[1]

Trained / supported embodiments
  • Embodiment-agnostic perception module (monocular RGB or RGB-D camera with calibrated intrinsics)

[1]

Training datasets
  • MegaPose synthetic BlenderProc dataset (2M images, >20,000 objects) in webdataset format: gso_1M (Google Scanned Objects, 1M images) and shapenet_1M (ShapeNetCoreV2, 1M images)

[1] [5]

Interfaces
  • Python CLI (megapose.scripts.run_inference_on_example, megapose.scripts.download)
  • Python package (megapose)
  • Docker container (ylabbe/megapose6d)

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
coarse-rgb-906902141pytorch0.082 GBopen2023-01-03
refiner-rgb-653307694pytorch0.083 GBopen2023-01-03
refiner-rgbd-288182519pytorch0.083 GBopen2023-01-03

Three PyTorch checkpoint directories verified HTTP 200 on Inria archive (checkpoint.pth.tar sizes 82–83 MB, dated 2023-01-03). Combined into 4 Model Zoo configurations (RGB, RGBD, RGB-multi-hypothesis, RGB-multi-hypothesis-icp).

[1] [4]

Is this for you?

Use cases
  • RGB-only 6D pose estimation of novel objects (megapose-1.0-RGB and megapose-1.0-RGB-multi-hypothesis)
  • RGB-D 6D pose estimation with multi-hypothesis scoring and optional ICP depth refinement (megapose-1.0-RGBD and megapose-1.0-RGB-multi-hypothesis-icp)
  • Training or benchmarking pose estimators on the 2M-image BlenderProc GSO/ShapeNet webdataset

[1]

Known limitations
  • Requires a 3D CAD mesh of the target object (in millimeters, .ply or .obj), camera intrinsics K, and an external 2D object bounding box at inference time.
  • Does not perform 2D object detection or frame-to-frame temporal tracking out of the box.
  • External Inria checkpoint archive does not contain an artifact-specific LICENSE file inside the model directories, and the GSO/ShapeNet training datasets require users to comply with original dataset terms.
  • Open Source Robotics has not installed MegaPose, downloaded weights, or reproduced BOP/YCB-Video benchmarks.

[1] [4]

Safety and limitations
  • Perception-only pose estimator that outputs object-in-camera 6D poses (quaternion + translation) and no robot control commands; grasp planning and collision checking must be handled externally.
  • Requires accurate camera intrinsics and millimeter-scale object meshes; scale mismatches or poor 2D bounding boxes degrade depth initialization and pose accuracy.
  • Before downloading the preprocessed GSO/ShapeNet meshes or synthetic webdatasets, upstream explicitly requires users to ensure they have permission to use the original datasets.
  • No independent runtime, latency, or accuracy validation was performed by Open Source Robotics.

[1]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA GPU (Conda or Docker with nvidia-docker2 / CUDA 11.0+ documented; exact minimum VRAM and latency not stated in README)
  • Edge / consumer capable: No

Requires CUDA GPU for PyTorch and rendering; exact VRAM and latency figures are not published in README.

[1] · Editorial assessment

Fine-tuning
  • Supported: No

Designed for zero-shot application to novel objects without retraining; README documents inference and dataset loading rather than a user fine-tuning recipe.

[1] [5] · Editorial assessment

Host computer

CUDA-capable NVIDIA GPU host on Linux (Docker instructions reference Ubuntu 20.04 and CUDA 11.0+; training configs show 32-GPU runs).

Inferred from Docker/Conda setup and config.yaml training metadata.

[1] [4] · Editorial assessment

Operating system
  • Linux

Ubuntu/Linux Docker and Conda environment setup documented.

[1] · Editorial assessment

Dependencies
  • Conda (conda/environment_full.yaml) or Docker with NVIDIA container toolkit
  • PyTorch and Panda3D / rendering dependencies
  • Pre-trained models from Inria archive
  • External 2D object detector for bounding boxes

[1]

Skills
  • Python, PyTorch, and Conda/Docker
  • Camera calibration and 3D CAD mesh preparation (.ply/.obj in mm)

Inferred from README installation and inference tutorial.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone megapose6d with submodules (git submodule update --init) and optionally set MEGAPOSE_DATA_DIR.
  • Create the Conda environment from conda/environment_full.yaml (or pull the ylabbe/megapose6d Docker image) and run pip install -e .
  • Download pre-trained models via python -m megapose.scripts.download --megapose_models and example data via --example_data.
  • Run python -m megapose.scripts.run_inference_on_example barbecue-sauce --run-inference (optionally specifying --model from the Model Zoo).

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
ModelNet, YCB-Video, and 7 Core BOP Challenge Datasets (CoRL 2022, arXiv:2212.06870)Upstream reports state-of-the-art novel-object pose estimation on ModelNet and YCB-Video, and competitive performance across the 7 core BOP datasets without target-object retraining.Upstream-reported paper abstract summary; exact per-dataset numerical tables are in the CoRL 2022 paper PDF and were not independently reproduced by OSR.

[5]

Project notes

Architecture, checkpoints, and input requirements

MegaPose operates as a perception-only two-stage pipeline backed by vanilla_resnet34 networks (verified in each checkpoint's config.yaml). Given an RGB image (with optional depth in millimeters), camera intrinsics K, an external 2D bounding box, and a millimeter-scale 3D CAD mesh (.ply or .obj), a coarse pose network scores rendered multi-view CAD hypotheses (sphere_26views) and an iterative refiner predicts pose updates to output per-object 6D poses (TWO: quaternion and 3D translation in camera frame). [1] [4] [5]

Three checkpoint directories were verified reachable on the Inria archive: coarse-rgb-906902141 (checkpoint.pth.tar, 82 MB), refiner-rgb-653307694 (83 MB), and refiner-rgbd-288182519 (83 MB), all dated January 2023. The Model Zoo combines these into four inference modes—megapose-1.0-RGB, megapose-1.0-RGBD, megapose-1.0-RGB-multi-hypothesis, and megapose-1.0-RGB-multi-hypothesis-icp—with upstream recommending megapose-1.0-RGB-multi-hypothesis for RGB input and megapose-1.0-RGB-multi-hypothesis-icp (which adds depth-based ICP refinement) for RGB-D input. Setup uses either conda/environment_full.yaml or the ylabbe/megapose6d Docker image on a CUDA-capable Linux host; the supplied README does not establish minimum GPU VRAM or inference latency, and compatibility with modern CUDA/PyTorch toolchains has not been independently tested. [1] [4]

Upstream-reported benchmarks

In the CoRL 2022 paper abstract (arXiv:2212.06870), the authors report state-of-the-art novel-object pose estimation on ModelNet and YCB-Video and competitive results across the 7 core BOP challenge datasets without target-object retraining. All benchmark claims are upstream-reported; Open Source Robotics has not installed MegaPose, downloaded the Inria checkpoints, run training, fine-tuning, or inference, or reproduced BOP evaluations. [5]

Apache-2.0 code and dataset terms

Repository source code is licensed under the permissive Apache License 2.0 (LICENSE). However, the Inria model directories do not carry a separate artifact-specific license file, and the README explicitly requires users to verify they have rights to the original Google Scanned Objects and ShapeNet datasets before downloading the gso_1M and shapenet_1M synthetic webdatasets or preprocessed meshes. [1] [2] [4]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Dec 12, 2024 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / f3b8e1247f133f3d098833a251b8f2d744c03e1f · software repository

Separate license matrix

Include MegaPose as a partially open novel-object 6D pose pipeline: repository code is Apache-2.0, while Inria checkpoints lack a standalone license file and GSO/ShapeNet synthetic datasets require original upstream dataset rights.

LayerAvailability / license
Codeavailable · Apache-2.0
Inria & NVIDIA Apache-2.0 license for repository source code. [1] [2]
Model weightsavailable · License not fully assessed
Three public PyTorch checkpoint directories verified HTTP 200 on Inria archive; no separate artifact-specific license file is present inside the archive directories (repository code is Apache-2.0). [1] [4]
Training datasetsavailable · License not fully assessed
2M synthetic BlenderProc images (gso_1M and shapenet_1M) hosted on Inria archive; README explicitly warns users to ensure they have permission to use the original GSO and ShapeNet datasets before downloading. [1]
Documentationavailable · Apache-2.0
README, inference tutorial, and visualization notebooks included in repository. [1] [2]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for MegaPose (Apache-2.0 code, 3 verified Inria checkpoints).