Open Source RoboticsRobots, software & learning

Contact-GraspNet

Robotics model

End-to-end PointNet++ model predicting 6-DoF parallel-jaw grasp poses and widths directly from scene depth maps or 3D point clouds in cluttered environments. [1] [4] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Robotic grasping researchers evaluating 6-DoF point-cloud grasp synthesis under non-commercial terms [1] [2]
Parameters
Not stated in upstream README or arXiv:2103.14127 abstract (PointNet++ encoder-decoder in TensorFlow [1] [4] [6]
Embodiments
Parallel-jaw grippers (Franka/Panda-style gripper models included in gripper_models/ [1] [5]
Source rights
Some licenses are unresolved. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
License
NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation) (noncommercial) · license not fully assessed for model weights, datasets · Details by part

Requirements · Research details and dates

On this page

Overview

Contact-GraspNet (NVlabs/contact_graspnet, ICRA 2021, arXiv:2103.14127) is an end-to-end PointNet++ network that predicts 6-DoF parallel-jaw grasp poses and opening widths directly from scene depth maps or 3D point clouds. By rooting each grasp at observed 3D surface contact points, the model reduces grasp prediction to a 4-DoF representation per point. At pinned commit 721706067cc7f7889c32afe7b4f7955eb90bd866, the repository provides inference and training scripts, custom PointNet++ TensorFlow ops, and links to three pre-trained checkpoints on Google Drive. [1] [3] [4] [5]

Model card summary

Architecture

End-to-end PointNet++ network (multi-scale grouping set abstraction and feature propagation modules implemented in TensorFlow 2.2 compat.v1 graph mode) processing 20,000-point scene clouds to predict a 4-DoF contact-rooted grasp representation (approach direction, baseline orientation, binned grasp width, and contact confidence) reconstructing full 6-DoF parallel-jaw grasps.

[1] [4] [5]

Parameter count

Not stated in upstream README or arXiv:2103.14127 abstract (PointNet++ encoder-decoder in TensorFlow; Google Drive checkpoint binaries were not downloaded)

Parameter count omitted from README and abstract; TensorFlow checkpoints were not downloaded.

[1] [4] [6] · Editorial assessment

Input modalities
depthpoint-cloudrgbd

Depth map (in meters) + camera matrix K (with optional RGB and 2D segmentation map), or direct Nx3 3D point cloud in meters.

[1]

Output modalities
6d-posegripper-command

6-DoF parallel-jaw grasp poses rooted at scene contact points along with predicted gripper opening widths and confidence scores.

[1] [5]

Action space

Grasp-pose perception model (no arm trajectory controller); predicts 6-DoF parallel-jaw grasp poses, gripper opening widths, and contact confidence scores requiring an external motion planner and robot controller.

[1] [5]

Trained / supported embodiments
  • Parallel-jaw grippers (Franka/Panda-style gripper models included in gripper_models/; embodiment-agnostic arm via external motion planner)

[1] [5]

Training datasets
  • 10,000 simulated tabletop scenes generated from the ACRONYM grasp dataset and watertight ShapeNet meshes (17 million simulated grasps per arXiv:2103.14127)

[1] [5]

Interfaces
  • Python CLI (contact_graspnet/inference.py, contact_graspnet/train.py)
  • NumPy .npy/.npz input/output format

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
scene_test_2048_bs3_hor_sigma_001 (default)otherNot statedopenmain / 721706067cc7
scene_2048_bs3_rad2_32 (clean depth)otherNot statedopenmain / 721706067cc7
scene_test_2048_bs3_hor_sigma_0025 (noisy depth)otherNot statedopenmain / 721706067cc7

Three named TensorFlow checkpoints documented in README inside a public Google Drive folder (HTTP 200 verified); payloads were not downloaded so exact file sizes are unknown.

[1] [6]

Is this for you?

Use cases
  • Full-scene 6-DoF parallel-jaw grasp generation from raw depth maps or Nx3 point clouds
  • Object-wise cluttered-scene grasp filtering when paired with an external unknown-object instance segmentation mask (--local_regions --filter_grasps)
  • Training custom tabletop grasp generators on ACRONYM + ShapeNet scenes

[1] [5]

Known limitations
  • License.pdf (NVIDIA Source Code License for Contact-GraspNet) strictly restricts use to non-commercial research or evaluation only.
  • Tested stack is legacy: Python 3.7, TensorFlow 2.2 (tensorflow.compat.v1 graph mode), CUDA 11.1, and custom compiled PointNet++ TensorFlow ops (compile_pointnet_tfops.sh).
  • For object-wise grasps and background removal, upstream strongly recommends pairing with an external unknown-object segmentation model (e.g., UOIS).
  • Checkpoints, test data, and 10,000 training scenes are hosted on Google Drive; binary payloads were not downloaded or validated by OSR.

[1] [2] [4] [6]

Safety and limitations
  • NVIDIA Source Code License (Section 3.3) prohibits commercial use; only NVIDIA and affiliates may use the Work commercially.
  • Outputs candidate 6-DoF parallel-jaw grasp poses and widths only—no robot arm motion planning, inverse kinematics, or collision-free trajectory execution is included.
  • Legacy TensorFlow 2.2 / CUDA 11.1 dependencies and compiled custom TF ops create significant integration friction on modern GPUs.
  • Open Source Robotics has not compiled PointNet++ ops, downloaded Google Drive checkpoints, run inference, or tested real-robot grasping.

[1] [2] [4] [6]

Compute & fine-tuning

Inference compute
  • Minimum VRAM: 8 GB
  • Documented hardware: 1x NVIDIA GPU >= 8 GB VRAM for inference (README notes 'might work with less'); training requires >= 24 GB VRAM and >= 64 GB RAM
  • Precision: fp32
  • Edge / consumer capable: No

8 GB VRAM minimum explicitly stated in README Hardware section; fp32 inferred from tf.float32 placeholders in contact_graspnet.py.

[1] [4]

Fine-tuning

[1]

Host computer

Inference: 1x NVIDIA GPU >= 8 GB VRAM (README notes may work with less). Training: 1x NVIDIA GPU >= 24 GB VRAM and >= 64 GB system RAM.

[1]

Operating system
  • Linux

Shell build script (compile_pointnet_tfops.sh), EGL headless rendering, and CUDA 11.1 conda setup target Linux.

[1] · Editorial assessment

Dependencies
  • Python 3.7, TensorFlow 2.2, CUDA 11.1
  • Compiled PointNet++ TensorFlow ops
  • Conda (contact_graspnet_env.yml)
  • Optional external unknown-object segmentation (e.g., UOIS)

[1] [4]

Skills
  • Python 3.7, TensorFlow 2.2, and CUDA C++ op compilation
  • 3D point-cloud processing and robot motion planning

Inferred from README installation and compile_pointnet_tfops.sh.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Create the Conda environment from contact_graspnet_env.yml (tested with Python 3.7, TensorFlow 2.2, CUDA 11.1).
  • Compile the custom PointNet++ TensorFlow ops with sh compile_pointnet_tfops.sh if needed.
  • Download trained checkpoints from the Google Drive folder into checkpoints/ and test data into test_data/.
  • Run python contact_graspnet/inference.py --np_path=test_data/*.npy --local_regions --filter_grasps (or pass a point cloud with --z_range).

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
Real-World Unseen Object Grasping in Structured Clutter (ICRA 2021, arXiv:2103.14127)Over 90% grasp success rate, cutting the failure rate in half compared to a prior state-of-the-art baseline.Upstream-reported robotic grasping study in paper abstract; OSR has not reproduced hardware or simulation evaluations.

[5]

Project notes

Architecture, checkpoints, and legacy stack requirements

In contact_graspnet/contact_graspnet.py, the network uses PointNet++ multi-scale grouping set-abstraction and feature-propagation modules implemented in tensorflow.compat.v1 graph mode over 20,000-point clouds (tf.float32). Inference accepts .npy/.npz files containing a depth map (in meters) and camera matrix K (plus optional RGB and 2D segmentation map) or a direct N×3 point cloud. Upstream strongly recommends pairing inference with an external unknown-object instance segmentation model (using --local_regions --filter_grasps) to isolate object-wise grasps and suppress background contacts. [1] [4]

The README documents three TensorFlow checkpoints in a public Google Drive folder (verified HTTP 200, though payloads were not downloaded): scene_test_2048_bs3_hor_sigma_001 (default), scene_2048_bs3_rad2_32 (no depth noise), and scene_test_2048_bs3_hor_sigma_0025 (strong depth noise). Upstream lists hardware requirements of an NVIDIA GPU with at least 8 GB VRAM for inference (noting it "might work with less"), and at least 24 GB VRAM plus 64 GB RAM for training on the 10,000 ACRONYM/ShapeNet tabletop scenes. A key practical constraint is the legacy software stack: upstream tested Python 3.7, TensorFlow 2.2, and CUDA 11.1 with custom compiled PointNet++ TF ops (compile_pointnet_tfops.sh). [1] [6]

Upstream-reported grasping performance

In arXiv:2103.14127, the authors report training on 17 million simulated grasps and achieving over 90% grasp success on unseen objects in structured clutter during a real-world robotic grasping study, cutting the failure rate in half relative to a prior state-of-the-art method. All benchmark claims are upstream-reported; Open Source Robotics has not compiled the TensorFlow ops, downloaded the Google Drive checkpoints, run training, fine-tuning, or inference, or tested Contact-GraspNet on a physical robot. [5]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Nov 21, 2024 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / 721706067cc7f7889c32afe7b4f7955eb90bd866 · software repository

Separate license matrix

Include the inspectable 6-DoF grasp-generation implementation under a non-commercial research scope; clearly highlight the NVIDIA non-commercial license and legacy TensorFlow 2.2 / CUDA 11.1 dependency stack.

LayerAvailability / license
Codeavailable · NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation)
Section 3.3 restricts use of the Work and derivative works to non-commercial research or evaluation purposes only. [2] [3]
Model weightsavailable · License not fully assessed
Public Google Drive folder verified HTTP 200; binary TensorFlow checkpoint payloads were not downloaded and artifact-specific terms remain unverified outside the repository License.pdf. [1] [6]
Training datasetsavailable · License not fully assessed
10,000 tabletop contact-grasp scenes on Google Drive plus external ACRONYM and ShapeNet meshes required for training; dataset-specific terms not separately verified. [1]
Documentationavailable · NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation)
Repository README and example scripts. [1] [2]

Non-commercial NVIDIA license

The repository's License.pdf (NVIDIA Source Code License for Contact-GraspNet, Section 3.3) strictly limits use of the software and derivative works to non-commercial research or evaluation purposes, reserving commercial rights to NVIDIA and its affiliates. External Google Drive checkpoints and ACRONYM/ShapeNet training datasets were not downloaded to verify separate artifact-specific license notices. [1] [2] [6]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for Contact-GraspNet (NVIDIA non-commercial license, 3 TF checkpoints).