Open Source RoboticsRobots, software & learning

FoundationPose

Software

Research implementation for 6D object pose estimation and tracking from RGB-D inputs, using a CAD model or reference views. [1]

Installation & first result →Official website

For
Researchers evaluating object-pose perception [1]
Source rights
Some licenses are unresolved. Includes noncommercial terms. Review licenses
Unknowns
operating system, compute requirements not fully established. Review gaps

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
License
NVIDIA Source Code License (noncommercial research/evaluation) (noncommercial) · license not fully assessed for model weights, datasets · Details by part

Requirements · Research details and dates

On this page

Overview

FoundationPose estimates and tracks an object's 6D pose from visual observations. The released implementation supports a model-based route using a mesh and a model-free route using reference views. For a bounded first investigation, its model-based demo exposes the perception inputs and pose outputs without requiring a robot controller. [1] [2]

Is this for you?

Known limitations
  • Repository license limits use to noncommercial research or evaluation.
  • External weights and data were not downloaded or validated in this review.
  • run_demo.py removes existing debug-directory contents at startup.
  • The released model excludes diffusion-augmented training data and weights.

[1] [2] [7]

What you need

Languages / runtimes
  • Python

[2] [3]

Dependencies
  • CUDA-capable NVIDIA environment
  • PyTorch, PyTorch3D and NVDiffRast
  • Compiled extension and external scorer/refiner weights
  • RGB-D demo inputs and a mesh

[1] [2]

Installation & first result

  • Prepare the documented CUDA environment and build extensions.
  • Obtain the named scorer/refiner weights and demo data.
  • Provide a mesh, RGB-D sequence, camera intrinsics and initial object mask.
  • Run run_demo.py with a disposable debug directory and inspect saved pose matrices.

Editorial assessment from cited documentation; no runtime testing.

[1] [2] · Editorial assessment

Installation

Follow one documented Docker or Conda build route; acquire the separate weights and demo inputs before run_demo.py.

Editorial assessment from cited documentation; no runtime testing.

[1] · Editorial assessment

Hardware, models & data

Inputs and outputs

Mesh, RGB-D frames, camera intrinsics and initial object mask produce per-frame 4x4 object poses and optional visualizations.

[2]

Project notes

The demo needs more than a mesh

At commit a1b694b83e633c2cb6115b9063d940a687759392, run_demo.py loads a mesh and an RGB-D sequence, then uses camera intrinsics and an object mask to register the first frame. Later frames use tracking. Each result is saved as a 4×4 object-in-camera pose matrix. This demo does not discover an unknown object's initial mask for you or command a robot to grasp it. [2] [6]

The README identifies separate refiner and scorer weight directories, plus an external demo-data download. These payloads are not supplied by the source archive and were not downloaded in this review. The documented setup uses CUDA, PyTorch, PyTorch3D, NVDiffRast and compiled extensions. Choose one complete Docker or Conda route and match the CUDA/PyTorch dependencies; the README also calls out newer-GPU setup differences. [1] [3] [4]

Before running python run_demo.py, set a disposable --debug_dir: the script starts by removing that directory's existing contents through a shell command. Avoid spaces or shell metacharacters in the path. Its default debug level opens an OpenCV window; saved pose matrices and visualizations should be examined against the supplied camera and object geometry before drawing conclusions about accuracy. These are source-inspection findings, not a reproduced run. [2]

What we checked

Source researched only; no installation, training, benchmark or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Requirements: operating system, host computer
  • Version-specific hardware compatibility (ecosystem tags alone do not establish it)

Updates

Upstream last updated
Apr 29, 2026 [6]
Latest release
Not established
Profile updated
Oct 18, 2026
Sources last checked
Oct 18, 2026
Next review
Nov 18, 2026
Reviewed revision
commit snapshot; no stable GitHub release / a1b694b83e633c2cb6115b9063d940a687759392 · software repository

Source files and licenses

Include the inspectable robotics pose-estimation implementation under a restricted research scope; distinguish source availability from unverified external weights and production rights.

LayerAvailability / license
softwareavailable · NVIDIA Source Code License (noncommercial research/evaluation)
Repository materials only; external components retain their own terms. [7]
model weightsavailable · License not fully assessed
Public Drive folder listing retrieved successfully; payloads not downloaded or validated and artifact-specific terms remain unverified. [1] [8]
datasetsavailable · License not fully assessed
Public Drive folder listing retrieved successfully; payloads not downloaded or validated and artifact-specific terms remain unverified. [1] [9]
documentationavailable · NVIDIA Source Code License (noncommercial research/evaluation)
Repository materials only; external components retain their own terms. [7]

Research rights and model scope

The repository's NVIDIA Source Code License limits use to noncommercial research or evaluation. It should not be presented as a permissively licensed production perception stack. The README also says diffusion-augmented data and the weights trained on them could not be released; the public version omits that training component and may perform differently. Linked artifacts and third-party components need their own rights checks. [1] [7]

We inspected the estimator and demo source, but did not install dependencies, load model weights, evaluate accuracy or test hardware. The implementation is a useful research reference; any deployed perception result remains unverified here.

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.7.0-draft.5

Change history