Contact-GraspNet
Robotics modelEnd-to-end PointNet++ model predicting 6-DoF parallel-jaw grasp poses and widths directly from scene depth maps or 3D point clouds in cluttered environments. [1] [4] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Robotic grasping researchers evaluating 6-DoF point-cloud grasp synthesis under non-commercial terms [1] [2]
- Parameters
- Not stated in upstream README or arXiv:2103.14127 abstract (PointNet++ encoder-decoder in TensorFlow [1] [4] [6]
- Embodiments
- Parallel-jaw grippers (Franka/Panda-style gripper models included in gripper_models/ [1] [5]
- Source rights
- Some licenses are unresolved. Includes noncommercial terms. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation) (noncommercial) · license not fully assessed for model weights, datasets · Details by part
Overview
Contact-GraspNet (NVlabs/contact_graspnet, ICRA 2021, arXiv:2103.14127) is an end-to-end PointNet++ network that predicts 6-DoF parallel-jaw grasp poses and opening widths directly from scene depth maps or 3D point clouds. By rooting each grasp at observed 3D surface contact points, the model reduces grasp prediction to a 4-DoF representation per point. At pinned commit 721706067cc7f7889c32afe7b4f7955eb90bd866, the repository provides inference and training scripts, custom PointNet++ TensorFlow ops, and links to three pre-trained checkpoints on Google Drive. [1] [3] [4] [5]
Model card summary
- Architecture
End-to-end PointNet++ network (multi-scale grouping set abstraction and feature propagation modules implemented in TensorFlow 2.2 compat.v1 graph mode) processing 20,000-point scene clouds to predict a 4-DoF contact-rooted grasp representation (approach direction, baseline orientation, binned grasp width, and contact confidence) reconstructing full 6-DoF parallel-jaw grasps.
- Parameter count
Not stated in upstream README or arXiv:2103.14127 abstract (PointNet++ encoder-decoder in TensorFlow; Google Drive checkpoint binaries were not downloaded)
Parameter count omitted from README and abstract; TensorFlow checkpoints were not downloaded.
- Input modalities
- depthpoint-cloudrgbd
Depth map (in meters) + camera matrix K (with optional RGB and 2D segmentation map), or direct Nx3 3D point cloud in meters.
- Output modalities
- 6d-posegripper-command
6-DoF parallel-jaw grasp poses rooted at scene contact points along with predicted gripper opening widths and confidence scores.
- Action space
Grasp-pose perception model (no arm trajectory controller); predicts 6-DoF parallel-jaw grasp poses, gripper opening widths, and contact confidence scores requiring an external motion planner and robot controller.
- Trained / supported embodiments
- Parallel-jaw grippers (Franka/Panda-style gripper models included in gripper_models/; embodiment-agnostic arm via external motion planner)
- Training datasets
- 10,000 simulated tabletop scenes generated from the ACRONYM grasp dataset and watertight ShapeNet meshes (17 million simulated grasps per arXiv:2103.14127)
- Interfaces
- Python CLI (contact_graspnet/inference.py, contact_graspnet/train.py)
- NumPy .npy/.npz input/output format
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| scene_test_2048_bs3_hor_sigma_001 (default) | other | Not stated | open | main / 721706067cc7 |
| scene_2048_bs3_rad2_32 (clean depth) | other | Not stated | open | main / 721706067cc7 |
| scene_test_2048_bs3_hor_sigma_0025 (noisy depth) | other | Not stated | open | main / 721706067cc7 |
Three named TensorFlow checkpoints documented in README inside a public Google Drive folder (HTTP 200 verified); payloads were not downloaded so exact file sizes are unknown.
Is this for you?
- Use cases
- Full-scene 6-DoF parallel-jaw grasp generation from raw depth maps or Nx3 point clouds
- Object-wise cluttered-scene grasp filtering when paired with an external unknown-object instance segmentation mask (--local_regions --filter_grasps)
- Training custom tabletop grasp generators on ACRONYM + ShapeNet scenes
- Known limitations
- License.pdf (NVIDIA Source Code License for Contact-GraspNet) strictly restricts use to non-commercial research or evaluation only.
- Tested stack is legacy: Python 3.7, TensorFlow 2.2 (tensorflow.compat.v1 graph mode), CUDA 11.1, and custom compiled PointNet++ TensorFlow ops (compile_pointnet_tfops.sh).
- For object-wise grasps and background removal, upstream strongly recommends pairing with an external unknown-object segmentation model (e.g., UOIS).
- Checkpoints, test data, and 10,000 training scenes are hosted on Google Drive; binary payloads were not downloaded or validated by OSR.
- Safety and limitations
- NVIDIA Source Code License (Section 3.3) prohibits commercial use; only NVIDIA and affiliates may use the Work commercially.
- Outputs candidate 6-DoF parallel-jaw grasp poses and widths only—no robot arm motion planning, inverse kinematics, or collision-free trajectory execution is included.
- Legacy TensorFlow 2.2 / CUDA 11.1 dependencies and compiled custom TF ops create significant integration friction on modern GPUs.
- Open Source Robotics has not compiled PointNet++ ops, downloaded Google Drive checkpoints, run inference, or tested real-robot grasping.
Compute & fine-tuning
- Inference compute
- Minimum VRAM: 8 GB
- Documented hardware: 1x NVIDIA GPU >= 8 GB VRAM for inference (README notes 'might work with less'); training requires >= 24 GB VRAM and >= 64 GB RAM
- Precision:
fp32 - Edge / consumer capable: No
8 GB VRAM minimum explicitly stated in README Hardware section; fp32 inferred from tf.float32 placeholders in contact_graspnet.py.
- Fine-tuning
- Supported: Yes
- Methods: full training / retraining via contact_graspnet/train.py on ACRONYM tabletop scenes
- Minimum VRAM: 24 GB
- Recipe: Upstream fine-tuning documentation
- Host computer
Inference: 1x NVIDIA GPU >= 8 GB VRAM (README notes may work with less). Training: 1x NVIDIA GPU >= 24 GB VRAM and >= 64 GB system RAM.
- Operating system
- Linux
Shell build script (compile_pointnet_tfops.sh), EGL headless rendering, and CUDA 11.1 conda setup target Linux.
[1] · Editorial assessment
- Dependencies
- Python 3.7, TensorFlow 2.2, CUDA 11.1
- Compiled PointNet++ TensorFlow ops
- Conda (contact_graspnet_env.yml)
- Optional external unknown-object segmentation (e.g., UOIS)
- Skills
- Python 3.7, TensorFlow 2.2, and CUDA C++ op compilation
- 3D point-cloud processing and robot motion planning
Inferred from README installation and compile_pointnet_tfops.sh.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Create the Conda environment from contact_graspnet_env.yml (tested with Python 3.7, TensorFlow 2.2, CUDA 11.1).
- Compile the custom PointNet++ TensorFlow ops with sh compile_pointnet_tfops.sh if needed.
- Download trained checkpoints from the Google Drive folder into checkpoints/ and test data into test_data/.
- Run python contact_graspnet/inference.py --np_path=test_data/*.npy --local_regions --filter_grasps (or pass a point cloud with --z_range).
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Real-World Unseen Object Grasping in Structured Clutter (ICRA 2021, arXiv:2103.14127) | Over 90% grasp success rate, cutting the failure rate in half compared to a prior state-of-the-art baseline. | Upstream-reported robotic grasping study in paper abstract; OSR has not reproduced hardware or simulation evaluations. |
Project notes
Architecture, checkpoints, and legacy stack requirements
In contact_graspnet/contact_graspnet.py, the network uses PointNet++ multi-scale grouping set-abstraction and feature-propagation modules implemented in tensorflow.compat.v1 graph mode over 20,000-point clouds (tf.float32). Inference accepts .npy/.npz files containing a depth map (in meters) and camera matrix K (plus optional RGB and 2D segmentation map) or a direct N×3 point cloud. Upstream strongly recommends pairing inference with an external unknown-object instance segmentation model (using --local_regions --filter_grasps) to isolate object-wise grasps and suppress background contacts. [1] [4]
The README documents three TensorFlow checkpoints in a public Google Drive folder (verified HTTP 200, though payloads were not downloaded): scene_test_2048_bs3_hor_sigma_001 (default), scene_2048_bs3_rad2_32 (no depth noise), and scene_test_2048_bs3_hor_sigma_0025 (strong depth noise). Upstream lists hardware requirements of an NVIDIA GPU with at least 8 GB VRAM for inference (noting it "might work with less"), and at least 24 GB VRAM plus 64 GB RAM for training on the 10,000 ACRONYM/ShapeNet tabletop scenes. A key practical constraint is the legacy software stack: upstream tested Python 3.7, TensorFlow 2.2, and CUDA 11.1 with custom compiled PointNet++ TF ops (compile_pointnet_tfops.sh). [1] [6]
Upstream-reported grasping performance
In arXiv:2103.14127, the authors report training on 17 million simulated grasps and achieving over 90% grasp success on unseen objects in structured clutter during a real-world robotic grasping study, cutting the failure rate in half relative to a prior state-of-the-art method. All benchmark claims are upstream-reported; Open Source Robotics has not compiled the TensorFlow ops, downloaded the Google Drive checkpoints, run training, fine-tuning, or inference, or tested Contact-GraspNet on a physical robot. [5]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Nov 21, 2024 [3]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / 721706067cc7f7889c32afe7b4f7955eb90bd866 · software repository
Separate license matrix
Include the inspectable 6-DoF grasp-generation implementation under a non-commercial research scope; clearly highlight the NVIDIA non-commercial license and legacy TensorFlow 2.2 / CUDA 11.1 dependency stack.
| Layer | Availability / license |
|---|---|
| Code | available · NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation) Section 3.3 restricts use of the Work and derivative works to non-commercial research or evaluation purposes only. [2] [3] |
| Model weights | available · License not fully assessed Public Google Drive folder verified HTTP 200; binary TensorFlow checkpoint payloads were not downloaded and artifact-specific terms remain unverified outside the repository License.pdf. [1] [6] |
| Training datasets | available · License not fully assessed 10,000 tabletop contact-grasp scenes on Google Drive plus external ACRONYM and ShapeNet meshes required for training; dataset-specific terms not separately verified. [1] |
| Documentation | available · NVIDIA Source Code License for Contact-GraspNet (non-commercial research/evaluation) Repository README and example scripts. [1] [2] |
Non-commercial NVIDIA license
The repository's License.pdf (NVIDIA Source Code License for Contact-GraspNet, Section 3.3) strictly limits use of the software and derivative works to non-commercial research or evaluation purposes, reserving commercial rights to NVIDIA and its affiliates. External Google Drive checkpoints and ACRONYM/ShapeNet training datasets were not downloaded to verify separate artifact-specific license notices. [1] [2] [6]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 Contact-GraspNet README
Installation (Python 3.7, TF 2.2, CUDA 11.1), GPU VRAM requirements, 3 Google Drive checkpoints, inference flags, and ACRONYM training workflow.Revision, date & where cited
NVlabs/contact_graspnet contributors · github.com · revision
7217060· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 4 more - S002 NVIDIA Source Code License for Contact-GraspNet (License.pdf)
Section 3.3 restricts the Work and derivative works to non-commercial research or evaluation purposes only; NVIDIA retains commercial rights.Revision, date & where cited
NVIDIA Corporation · github.com · revision
7217060· Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S003 GitHub repository metadata and pinned commit
Last commit 2024-11-21; no GitHub releases; not archived.Revision, date & where cited
NVlabs/contact_graspnet / GitHub · github.com · revision
7217060· main / · Accessed Nov 21, 2026
Used for: Overview, What we checked, Separate license matrix - S004 Contact-GraspNet model implementation (contact_graspnet/contact_graspnet.py)
PointNet++ architecture using pointnet_sa_module, pointnet_sa_module_msg, and pointnet_fp_module in tensorflow.compat.v1 graph mode on 20,000-point clouds.Revision, date & where cited
NVlabs/contact_graspnet contributors · github.com · revision
7217060· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Compute & fine-tuning and 1 more - S005 Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes (arXiv:2103.14127)
End-to-end 4-DoF contact-rooted grasp representation predicting 6-DoF parallel-jaw grasps from depth/point clouds; trained on 17M simulated grasps; reports >90% real-world grasping success in structured clutter.Revision, date & where cited
NVIDIA Research & DLR/TUM (ICRA 2021 / arXiv) · arxiv.org · arXiv:2103.14127v1 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Upstream-reported benchmarks and 1 more - S006 Google Drive checkpoints folder reachability
Public Google Drive checkpoints folder returned HTTP 200; binary TensorFlow payloads were not downloaded.Revision, date & where cited
NVlabs/contact_graspnet / Google Drive · drive.google.com · HTTP GET 2026-11-21 · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Is this for you?, Architecture, checkpoints, and legacy stack requirements and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for Contact-GraspNet (NVIDIA non-commercial license, 3 TF checkpoints).