GraspNet-Baseline
Robotics modelTwo-stage PointNet++ 6-DoF parallel-jaw grasp pose detector from RGB-D point clouds, serving as the official baseline for the GraspNet-1Billion benchmark. [1] [4] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Academic and non-profit robotics researchers benchmarking 6-DoF grasp pose detection on GraspNet-1Billion [1] [2]
- Parameters
- Not stated in upstream README or models/graspnet.py (two-stage PointNet++ + ApproachNet + CloudCrop + OperationNet +… [1] [4] [6]
- Embodiments
- Parallel-jaw grippers with RealSense or Kinect RGB-D cameras (embodiment-agnostic arm via external motion planner) [1] [4]
- Source rights
- Named licenses are documented by part. Includes noncommercial terms. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- SJTU GraspNet-Baseline Software License Agreement (academic/non-profit non-commercial research only) (noncommercial), Non-commercial research use only (per README License section) (noncommercial) · Details by part
Overview
GraspNet-Baseline (graspnet/graspnet-baseline, CVPR 2020, arXiv:1912.13470) is the official reference grasp-pose detection model for the GraspNet-1Billion benchmark from Shanghai Jiao Tong University's MVIG Lab. At pinned commit 280c215129f759ed8649cb4e89fc5dfee55f4f80, the repository provides PyTorch training, evaluation, and RGB-D demo code along with links to two pretrained checkpoints (checkpoint-rs.tar for RealSense and checkpoint-kn.tar for Kinect). [1] [3] [5]
Model card summary
- Architecture
Two-stage PyTorch PointNet++ grasp network (models/graspnet.py): GraspNetStage1 pairs a Pointnet2Backbone with ApproachNet (300 candidate viewpoints) to score objectness and approach vectors; GraspNetStage2 applies CloudCrop cylinder grouping around seed points followed by OperationNet (12 in-plane rotation angles, 4 grasp depths, widths, and scores) and ToleranceNet.
- Parameter count
Not stated in upstream README or models/graspnet.py (two-stage PointNet++ + ApproachNet + CloudCrop + OperationNet + ToleranceNet; Google Drive .tar checkpoints were not downloaded)
Parameter count is not reported in README; checkpoints were not downloaded.
- Input modalities
- rgbdpoint-cloud
RGB-D images with camera intrinsics and depth scale factor (plus optional workspace mask) converted into 3D scene point clouds.
- Output modalities
- 6d-posegripper-command
Outputs 6-DoF grasp center and 3x3 rotation matrix alongside gripper width, height, depth, and tolerance-weighted grasp score.
- Action space
Grasp-pose perception model (no arm trajectory controller); predicts scored 6-DoF parallel-jaw grasp poses, gripper widths, and grasp depths for execution by an external planner.
- Trained / supported embodiments
- Parallel-jaw grippers with RealSense or Kinect RGB-D cameras (embodiment-agnostic arm via external motion planner)
- Training datasets
- GraspNet-1Billion dataset (RealSense and Kinect RGB-D splits; arXiv:1912.13470 reports 87,040 RGB-D images and 370M+ grasp poses in v2, expanded to 1B+ in CVPR 2020) plus generated tolerance labels
- Interfaces
- Python CLIs (demo.py, train.py, test.py) and shell wrappers (command_demo.sh, command_train.sh, command_test.sh)
- graspnetAPI evaluation interface
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| checkpoint-rs.tar (RealSense, recommended by upstream) | pytorch | Not stated | open | main / 280c215129f7 |
| checkpoint-kn.tar (Kinect) | pytorch | Not stated | open | main / 280c215129f7 |
Both Google Drive links returned HTTP 200 (Baidu Pan mirrors also listed in README); binary payloads were not downloaded so exact file sizes are unknown.
Is this for you?
- Use cases
- 6-DoF parallel-jaw grasp detection and visualization from RGB-D images and camera intrinsics via demo.py
- Standardized training and evaluation on the GraspNet-1Billion RealSense and Kinect splits with graspnetAPI
- Architectural reference for downstream grasp detectors (including AnyGrasp's README pointing users here for inspectable network code)
- Known limitations
- Root LICENSE restricts use to academic or non-profit non-commercial internal research only, prohibits redistribution/sublicensing, and assigns ownership of derivatives back to Shanghai Jiao Tong University.
- Requirements target legacy PyTorch 1.6 and require compiling custom pointnet2 and knn CUDA extensions.
- Training requires downloading the external GraspNet-1Billion dataset and generating or downloading separate tolerance labels (tolerance.tar).
- Checkpoints are hosted on Google Drive and Baidu Pan; binary payloads were not downloaded or validated by OSR.
- Safety and limitations
- SJTU Software License Agreement restricts use to academic/non-profit non-commercial research and claims Licensor ownership over derivatives; README notes commercial inquiries require direct email permission.
- Outputs candidate 6-DoF parallel-jaw grasp poses only—no robot kinematics, trajectory planning, or physical collision avoidance beyond optional single-view point-cloud collision filtering.
- Legacy PyTorch 1.6 and custom CUDA operator builds (`pointnet2`, `knn`) can fail on modern CUDA/PyTorch toolchains without manual patching.
- Open Source Robotics has not compiled CUDA extensions, downloaded Google Drive checkpoints, run training, fine-tuning, or inference, or executed grasps on a robot.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU required for compiled pointnet2 and knn CUDA extensions (specific GPU model and VRAM not stated in README)
- Precision:
fp32 - Edge / consumer capable: No
CUDA GPU required by pointnet2 and pytorch_knn_cuda extensions; fp32 inferred from .float() tensor decoding in models/graspnet.py.
- Fine-tuning
- Supported: Yes
- Methods: full training / retraining via train.py and command_train.sh on GraspNet-1Billion with tolerance labels
- Recipe: Upstream fine-tuning documentation
- Host computer
Linux host with NVIDIA CUDA GPU for compiled pointnet2 and knn operators; exact VRAM and RAM minimums are not stated in README.
Inferred from CUDA extension build steps and shell scripts in README.
[1] · Editorial assessment
- Operating system
- Linux
Shell scripts (command_train.sh, command_test.sh, command_demo.sh) and CUDA PyTorch C++ extensions target Linux.
[1] · Editorial assessment
- Dependencies
- Python 3, PyTorch 1.6, Open3D >= 0.8, TensorBoard 2.3, NumPy, SciPy, Pillow, tqdm
- Compiled pointnet2 and knn CUDA operators
- graspnetAPI (for GraspNet-1Billion evaluation)
- Skills
- PyTorch CUDA C++ extension compilation (pointnet2 and knn)
- RGB-D camera calibration, 3D point-cloud processing, and external robot motion planning
Inferred from README installation and demo instructions.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Clone graspnet/graspnet-baseline and install Python dependencies from requirements.txt (PyTorch 1.6, Open3D >= 0.8, TensorBoard 2.3).
- Compile and install the bundled pointnet2 and knn CUDA extensions (python setup.py install in each subdirectory).
- Clone and install graspnet/graspnetAPI via pip install . for dataset evaluation.
- Download checkpoint-rs.tar (recommended) or checkpoint-kn.tar from Google Drive or Baidu Pan, then run command_demo.sh or command_test.sh.
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| GraspNet-1Billion RealSense split (in-repo evaluation with --collision_thresh 0.01) | Seen AP 47.47 (AP0.8 55.90, AP0.4 41.33); Similar AP 42.27 (AP0.8 51.01, AP0.4 35.40); Novel AP 16.61 (AP0.8 20.84, AP0.4 8.30). Paper baseline without single-view collision post-processing: Seen 27.56, Similar 26.11, Novel 10.55. | Upstream-reported in repository README using graspnetAPI evaluation; OSR has not reproduced inference or evaluation. |
| GraspNet-1Billion Kinect split (in-repo evaluation with --collision_thresh 0.01) | Seen AP 42.02 (AP0.8 49.91, AP0.4 35.34); Similar AP 37.35 (AP0.8 44.82, AP0.4 30.40); Novel AP 12.17 (AP0.8 15.17, AP0.4 5.51). Paper baseline: Seen 29.88, Similar 27.84, Novel 11.51. | Upstream-reported in repository README using graspnetAPI evaluation; OSR has not reproduced inference or evaluation. |
Project notes
Architecture, checkpoints, and legacy build stack
In models/graspnet.py, the model splits 6-DoF parallel-jaw grasp generation into two stages. GraspNetStage1 runs a Pointnet2Backbone followed by ApproachNet over 300 candidate viewpoints to estimate objectness and top approaching directions. GraspNetStage2 then crops cylindrical local point-cloud neighborhoods (CloudCrop) around seed points and runs OperationNet (scoring 12 in-plane rotation angles, 4 grasp depths, and gripper widths) alongside ToleranceNet, outputting tolerance-weighted 6-DoF grasp poses (3×3 rotation matrix, center, width, height, and depth). [4]
Both pretrained PyTorch checkpoints (checkpoint-rs.tar, recommended by upstream for cross-sensor transfer, and checkpoint-kn.tar) are hosted on Google Drive (verified HTTP 200, though payloads were not downloaded) and Baidu Pan. The repository targets a legacy environment—Python 3, PyTorch 1.6, Open3D >= 0.8, and TensorBoard 2.3—and requires compiling two bundled CUDA C++ extensions (pointnet2 and knn) plus installing graspnetAPI for benchmark evaluation. Training additionally requires downloading the GraspNet-1Billion dataset and generating or downloading separate tolerance labels (tolerance.tar). [1] [6]
Upstream-reported GraspNet-1Billion benchmarks
In the repository README, upstream reports GraspNet-1Billion evaluation results with single-view collision detection post-processing (--collision_thresh 0.01): on the RealSense split, in-repo AP reaches 47.47 (Seen), 42.27 (Similar), and 16.61 (Novel), compared to 27.56, 26.11, and 10.55 in the CVPR 2020 paper; on the Kinect split, in-repo AP reaches 42.02 (Seen), 37.35 (Similar), and 12.17 (Novel), compared to 29.88, 27.84, and 11.51 in the paper. All scores are upstream-reported; Open Source Robotics has not compiled the CUDA extensions, downloaded the checkpoints, run training, fine-tuning, or inference, or verified grasping on physical hardware. [1]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Feb 17, 2025 [3]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / 280c215129f759ed8649cb4e89fc5dfee55f4f80 · software repository
Separate license matrix
Include GraspNet-Baseline as a widely cited inspectable 6-DoF grasp detection reference model with public PyTorch checkpoints, while clearly flagging its restrictive SJTU non-commercial license and legacy PyTorch 1.6 CUDA extension stack.
| Layer | Availability / license |
|---|---|
| Code | available · SJTU GraspNet-Baseline Software License Agreement (academic/non-profit non-commercial research only) Restricts use to academic or non-profit non-commercial internal research, prohibits distribution/sublicensing, and asserts Licensor ownership over derivatives. [2] [3] |
| Model weights | available · Non-commercial research use only (per README License section) checkpoint-rs.tar and checkpoint-kn.tar Google Drive URLs verified HTTP 200; binary payloads were not downloaded. README states all data, labels, code, and models are for free non-commercial use only. [1] [6] |
| Training datasets | available · Non-commercial research use only (per README License section) GraspNet-1Billion dataset hosted via graspnet.net plus tolerance labels on Google Drive/Baidu Pan; full dataset archives were not downloaded. [1] |
| Documentation | available · SJTU GraspNet-Baseline Software License Agreement (academic/non-profit non-commercial research only) Repository README, example scripts, and external graspnetAPI ReadTheDocs. [1] [2] |
Restrictive SJTU non-commercial license
The root LICENSE file (GRASPNET-BASELINE SOFTWARE LICENSE AGREEMENT) limits use strictly to academic or non-profit organizations (or individuals) for internal non-commercial research, prohibits distributing, copying, or sublicensing the software except as explicitly permitted in the agreement, and states that any derivatives or modifications are owned by Shanghai Jiao Tong University. By contrast, the repository README states that all data, labels, code, and models are available for free non-commercial use "and may be redistributed under these conditions," with commercial inquiries directed to the authors by email. Because the README's redistribution permission conflicts with the stricter non-distribution clause in the root LICENSE, adopters should clarify redistribution terms directly with the maintainers before sharing code or modified copies. [1] [2]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 GraspNet-Baseline README
Requirements (PyTorch 1.6, Open3D >= 0.8), pointnet2/knn build steps, Google Drive/Baidu Pan links for checkpoint-rs.tar and checkpoint-kn.tar, RealSense/Kinect AP results tables, and non-commercial license notice.Revision, date & where cited
MVIG Lab, Shanghai Jiao Tong University · github.com · revision
280c215· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S002 GraspNet-Baseline LICENSE
GRASPNET-BASELINE SOFTWARE LICENSE AGREEMENT restricting use to academic or non-profit non-commercial internal research, prohibiting redistribution/sublicensing, and assigning ownership of derivatives to SJTU.Revision, date & where cited
Shanghai Jiao Tong University · github.com · revision
280c215· Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S003 GitHub repository metadata and pinned commit
Last commit 2025-02-17; not archived; no GitHub releases.Revision, date & where cited
graspnet/graspnet-baseline / GitHub · github.com · revision
280c215· main / · Accessed Nov 21, 2026
Used for: Overview, What we checked, Separate license matrix - S004 GraspNet-Baseline model implementation (models/graspnet.py)
Two-stage architecture: GraspNetStage1 (Pointnet2Backbone + ApproachNet over 300 views) and GraspNetStage2 (CloudCrop + OperationNet over 12 angles and 4 depths + ToleranceNet).Revision, date & where cited
graspnet/graspnet-baseline contributors · github.com · revision
280c215· Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Compute & fine-tuning, Architecture, checkpoints, and legacy build stack - S005 GraspNet: A Large-Scale Clustered and Densely Annotated Dataset for Object Grasping (arXiv:1912.13470)
Introduces the clustered RGB-D object grasping dataset and analytic force-closure evaluation system aligned with real-world grasping experiments.Revision, date & where cited
MVIG Lab, SJTU (CVPR 2020 / arXiv) · arxiv.org · arXiv:1912.13470v2 abstract · Accessed Nov 21, 2026
Used for: Overview, Model card summary - S006 Google Drive checkpoints reachability (checkpoint-rs.tar and checkpoint-kn.tar)
Both checkpoint-rs.tar and checkpoint-kn.tar Google Drive URLs returned HTTP 200; binary payloads were not downloaded.Revision, date & where cited
graspnet / Google Drive · drive.google.com · HTTP GET 2026-11-21 · Accessed Nov 21, 2026
Used for: Model card summary, Checkpoints, Is this for you?, Architecture, checkpoints, and legacy build stack and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for GraspNet-Baseline (SJTU non-commercial license, 2 PyTorch checkpoints).