Open Source RoboticsRobots, software & learning

GraspNet-Baseline

Robotics model

Two-stage PointNet++ 6-DoF parallel-jaw grasp pose detector from RGB-D point clouds, serving as the official baseline for the GraspNet-1Billion benchmark. [1] [4] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Academic and non-profit robotics researchers benchmarking 6-DoF grasp pose detection on GraspNet-1Billion [1] [2]
Parameters
Not stated in upstream README or models/graspnet.py (two-stage PointNet++ + ApproachNet + CloudCrop + OperationNet +… [1] [4] [6]
Embodiments
Parallel-jaw grippers with RealSense or Kinect RGB-D cameras (embodiment-agnostic arm via external motion planner) [1] [4]
Source rights
Named licenses are documented by part. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware

Requirements · Research details and dates

On this page

Overview

GraspNet-Baseline (graspnet/graspnet-baseline, CVPR 2020, arXiv:1912.13470) is the official reference grasp-pose detection model for the GraspNet-1Billion benchmark from Shanghai Jiao Tong University's MVIG Lab. At pinned commit 280c215129f759ed8649cb4e89fc5dfee55f4f80, the repository provides PyTorch training, evaluation, and RGB-D demo code along with links to two pretrained checkpoints (checkpoint-rs.tar for RealSense and checkpoint-kn.tar for Kinect). [1] [3] [5]

Model card summary

Architecture

Two-stage PyTorch PointNet++ grasp network (models/graspnet.py): GraspNetStage1 pairs a Pointnet2Backbone with ApproachNet (300 candidate viewpoints) to score objectness and approach vectors; GraspNetStage2 applies CloudCrop cylinder grouping around seed points followed by OperationNet (12 in-plane rotation angles, 4 grasp depths, widths, and scores) and ToleranceNet.

[1] [4]

Parameter count

Not stated in upstream README or models/graspnet.py (two-stage PointNet++ + ApproachNet + CloudCrop + OperationNet + ToleranceNet; Google Drive .tar checkpoints were not downloaded)

Parameter count is not reported in README; checkpoints were not downloaded.

[1] [4] [6] · Editorial assessment

Input modalities
rgbdpoint-cloud

RGB-D images with camera intrinsics and depth scale factor (plus optional workspace mask) converted into 3D scene point clouds.

[1] [4]

Output modalities
6d-posegripper-command

Outputs 6-DoF grasp center and 3x3 rotation matrix alongside gripper width, height, depth, and tolerance-weighted grasp score.

[1] [4]

Action space

Grasp-pose perception model (no arm trajectory controller); predicts scored 6-DoF parallel-jaw grasp poses, gripper widths, and grasp depths for execution by an external planner.

[1] [4]

Trained / supported embodiments
  • Parallel-jaw grippers with RealSense or Kinect RGB-D cameras (embodiment-agnostic arm via external motion planner)

[1] [4]

Training datasets
  • GraspNet-1Billion dataset (RealSense and Kinect RGB-D splits; arXiv:1912.13470 reports 87,040 RGB-D images and 370M+ grasp poses in v2, expanded to 1B+ in CVPR 2020) plus generated tolerance labels

[1] [5]

Interfaces
  • Python CLIs (demo.py, train.py, test.py) and shell wrappers (command_demo.sh, command_train.sh, command_test.sh)
  • graspnetAPI evaluation interface

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
checkpoint-rs.tar (RealSense, recommended by upstream)pytorchNot statedopenmain / 280c215129f7
checkpoint-kn.tar (Kinect)pytorchNot statedopenmain / 280c215129f7

Both Google Drive links returned HTTP 200 (Baidu Pan mirrors also listed in README); binary payloads were not downloaded so exact file sizes are unknown.

[1] [6]

Is this for you?

Use cases
  • 6-DoF parallel-jaw grasp detection and visualization from RGB-D images and camera intrinsics via demo.py
  • Standardized training and evaluation on the GraspNet-1Billion RealSense and Kinect splits with graspnetAPI
  • Architectural reference for downstream grasp detectors (including AnyGrasp's README pointing users here for inspectable network code)

[1] [4]

Known limitations
  • Root LICENSE restricts use to academic or non-profit non-commercial internal research only, prohibits redistribution/sublicensing, and assigns ownership of derivatives back to Shanghai Jiao Tong University.
  • Requirements target legacy PyTorch 1.6 and require compiling custom pointnet2 and knn CUDA extensions.
  • Training requires downloading the external GraspNet-1Billion dataset and generating or downloading separate tolerance labels (tolerance.tar).
  • Checkpoints are hosted on Google Drive and Baidu Pan; binary payloads were not downloaded or validated by OSR.

[1] [2] [6]

Safety and limitations
  • SJTU Software License Agreement restricts use to academic/non-profit non-commercial research and claims Licensor ownership over derivatives; README notes commercial inquiries require direct email permission.
  • Outputs candidate 6-DoF parallel-jaw grasp poses only—no robot kinematics, trajectory planning, or physical collision avoidance beyond optional single-view point-cloud collision filtering.
  • Legacy PyTorch 1.6 and custom CUDA operator builds (`pointnet2`, `knn`) can fail on modern CUDA/PyTorch toolchains without manual patching.
  • Open Source Robotics has not compiled CUDA extensions, downloaded Google Drive checkpoints, run training, fine-tuning, or inference, or executed grasps on a robot.

[1] [2] [4] [6]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA GPU required for compiled pointnet2 and knn CUDA extensions (specific GPU model and VRAM not stated in README)
  • Precision: fp32
  • Edge / consumer capable: No

CUDA GPU required by pointnet2 and pytorch_knn_cuda extensions; fp32 inferred from .float() tensor decoding in models/graspnet.py.

[1] [4] · Editorial assessment

Fine-tuning

[1]

Host computer

Linux host with NVIDIA CUDA GPU for compiled pointnet2 and knn operators; exact VRAM and RAM minimums are not stated in README.

Inferred from CUDA extension build steps and shell scripts in README.

[1] · Editorial assessment

Operating system
  • Linux

Shell scripts (command_train.sh, command_test.sh, command_demo.sh) and CUDA PyTorch C++ extensions target Linux.

[1] · Editorial assessment

Dependencies
  • Python 3, PyTorch 1.6, Open3D >= 0.8, TensorBoard 2.3, NumPy, SciPy, Pillow, tqdm
  • Compiled pointnet2 and knn CUDA operators
  • graspnetAPI (for GraspNet-1Billion evaluation)

[1]

Skills
  • PyTorch CUDA C++ extension compilation (pointnet2 and knn)
  • RGB-D camera calibration, 3D point-cloud processing, and external robot motion planning

Inferred from README installation and demo instructions.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone graspnet/graspnet-baseline and install Python dependencies from requirements.txt (PyTorch 1.6, Open3D >= 0.8, TensorBoard 2.3).
  • Compile and install the bundled pointnet2 and knn CUDA extensions (python setup.py install in each subdirectory).
  • Clone and install graspnet/graspnetAPI via pip install . for dataset evaluation.
  • Download checkpoint-rs.tar (recommended) or checkpoint-kn.tar from Google Drive or Baidu Pan, then run command_demo.sh or command_test.sh.

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
GraspNet-1Billion RealSense split (in-repo evaluation with --collision_thresh 0.01)Seen AP 47.47 (AP0.8 55.90, AP0.4 41.33); Similar AP 42.27 (AP0.8 51.01, AP0.4 35.40); Novel AP 16.61 (AP0.8 20.84, AP0.4 8.30). Paper baseline without single-view collision post-processing: Seen 27.56, Similar 26.11, Novel 10.55.Upstream-reported in repository README using graspnetAPI evaluation; OSR has not reproduced inference or evaluation.
GraspNet-1Billion Kinect split (in-repo evaluation with --collision_thresh 0.01)Seen AP 42.02 (AP0.8 49.91, AP0.4 35.34); Similar AP 37.35 (AP0.8 44.82, AP0.4 30.40); Novel AP 12.17 (AP0.8 15.17, AP0.4 5.51). Paper baseline: Seen 29.88, Similar 27.84, Novel 11.51.Upstream-reported in repository README using graspnetAPI evaluation; OSR has not reproduced inference or evaluation.

[1]

Project notes

Architecture, checkpoints, and legacy build stack

In models/graspnet.py, the model splits 6-DoF parallel-jaw grasp generation into two stages. GraspNetStage1 runs a Pointnet2Backbone followed by ApproachNet over 300 candidate viewpoints to estimate objectness and top approaching directions. GraspNetStage2 then crops cylindrical local point-cloud neighborhoods (CloudCrop) around seed points and runs OperationNet (scoring 12 in-plane rotation angles, 4 grasp depths, and gripper widths) alongside ToleranceNet, outputting tolerance-weighted 6-DoF grasp poses (3×3 rotation matrix, center, width, height, and depth). [4]

Both pretrained PyTorch checkpoints (checkpoint-rs.tar, recommended by upstream for cross-sensor transfer, and checkpoint-kn.tar) are hosted on Google Drive (verified HTTP 200, though payloads were not downloaded) and Baidu Pan. The repository targets a legacy environment—Python 3, PyTorch 1.6, Open3D >= 0.8, and TensorBoard 2.3—and requires compiling two bundled CUDA C++ extensions (pointnet2 and knn) plus installing graspnetAPI for benchmark evaluation. Training additionally requires downloading the GraspNet-1Billion dataset and generating or downloading separate tolerance labels (tolerance.tar). [1] [6]

Upstream-reported GraspNet-1Billion benchmarks

In the repository README, upstream reports GraspNet-1Billion evaluation results with single-view collision detection post-processing (--collision_thresh 0.01): on the RealSense split, in-repo AP reaches 47.47 (Seen), 42.27 (Similar), and 16.61 (Novel), compared to 27.56, 26.11, and 10.55 in the CVPR 2020 paper; on the Kinect split, in-repo AP reaches 42.02 (Seen), 37.35 (Similar), and 12.17 (Novel), compared to 29.88, 27.84, and 11.51 in the paper. All scores are upstream-reported; Open Source Robotics has not compiled the CUDA extensions, downloaded the checkpoints, run training, fine-tuning, or inference, or verified grasping on physical hardware. [1]

What we checked

Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Feb 17, 2025 [3]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub release / 280c215129f759ed8649cb4e89fc5dfee55f4f80 · software repository

Separate license matrix

Include GraspNet-Baseline as a widely cited inspectable 6-DoF grasp detection reference model with public PyTorch checkpoints, while clearly flagging its restrictive SJTU non-commercial license and legacy PyTorch 1.6 CUDA extension stack.

LayerAvailability / license
Codeavailable · SJTU GraspNet-Baseline Software License Agreement (academic/non-profit non-commercial research only)
Restricts use to academic or non-profit non-commercial internal research, prohibits distribution/sublicensing, and asserts Licensor ownership over derivatives. [2] [3]
Model weightsavailable · Non-commercial research use only (per README License section)
checkpoint-rs.tar and checkpoint-kn.tar Google Drive URLs verified HTTP 200; binary payloads were not downloaded. README states all data, labels, code, and models are for free non-commercial use only. [1] [6]
Training datasetsavailable · Non-commercial research use only (per README License section)
GraspNet-1Billion dataset hosted via graspnet.net plus tolerance labels on Google Drive/Baidu Pan; full dataset archives were not downloaded. [1]
Documentationavailable · SJTU GraspNet-Baseline Software License Agreement (academic/non-profit non-commercial research only)
Repository README, example scripts, and external graspnetAPI ReadTheDocs. [1] [2]

Restrictive SJTU non-commercial license

The root LICENSE file (GRASPNET-BASELINE SOFTWARE LICENSE AGREEMENT) limits use strictly to academic or non-profit organizations (or individuals) for internal non-commercial research, prohibits distributing, copying, or sublicensing the software except as explicitly permitted in the agreement, and states that any derivatives or modifications are owned by Shanghai Jiao Tong University. By contrast, the repository README states that all data, labels, code, and models are available for free non-commercial use "and may be redistributed under these conditions," with commercial inquiries directed to the authors by email. Because the README's redistribution permission conflicts with the stricter non-distribution clause in the root LICENSE, adopters should clarify redistribution terms directly with the maintainers before sharing code or modified copies. [1] [2]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile for GraspNet-Baseline (SJTU non-commercial license, 2 PyTorch checkpoints).