UniDexGrasp
Robotics modelTwo-stage universal dexterous grasping framework for ShadowHand combining probabilistic grasp proposal generation (GraspIPDF, GraspGlow, ContactNet) with an Isaac Gym goal-conditioned RL and DAgger vision policy. [1] [2] [3] [5]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Dexterous manipulation and RL researchers reproducing or extending CVPR 2023 ShadowHand grasping benchmarks in Isaac Gym [1] [3]
- Parameters
- Not stated in upstream READMEs or arXiv:2303.00938 abstract (multi-module pipeline: GraspIPDF, GraspGlow,… [2] [3] [5]
- Source rights
- Named licenses are documented by part. Includes noncommercial terms. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- CC-BY-NC-4.0 (declared in root README; no root LICENSE file) (noncommercial), CC-BY-NC-4.0 (declared in root README) (noncommercial) · Details by part
Overview
UniDexGrasp (PKU-EPIC/UniDexGrasp, CVPR 2023, arXiv:2303.00938) is a two-stage universal tabletop dexterous grasping framework for the ShadowHand from Peking University's EPIC Lab. At pinned commit 36c9bfcf7cedc987dd15e2384c548e51d5aadc21, the repository provides source code for both grasp proposal generation (dexgrasp_generation) and goal-conditioned RL/DAgger policy learning (dexgrasp_policy), alongside a PKU dataset mirror—though no pretrained model checkpoint links were found in the checked repository READMEs or project page as of 2026-11-21. [1] [2] [3] [4] [6]
Model card summary
- Architecture
Two-stage dexterous grasping architecture: Stage 1 (dexgrasp_generation) factorizes grasp pose generation into GraspIPDF (Implicit PDF over SO(3) rotation), GraspGlow (normalizing flow via nflows over translation and hand articulation), and ContactNet (PointNet++ contact refinement); Stage 2 (dexgrasp_policy) trains a goal-conditioned PPO teacher policy with state canonicalization and object curriculum in Isaac Gym, then distills it via DAgger into a PointNet++ vision-based execution policy.
- Parameter count
Not stated in upstream READMEs or arXiv:2303.00938 abstract (multi-module pipeline: GraspIPDF, GraspGlow, ContactNet, and PointNet++/MLP PPO+DAgger policies; no pretrained checkpoints released)
Parameter counts are omitted from upstream READMEs and abstract, and no pretrained checkpoints are distributed.
- Input modalities
- point-cloudproprioception
Stage-1 and Stage-2 vision policy take tabletop object point clouds (plus hand proprioception and target grasp goal pose); Stage-2 teacher uses simulator oracle states.
- Output modalities
- 6d-posejoint-positions
Stage-1 outputs 6-DoF wrist pose and ShadowHand articulation proposals; Stage-2 outputs closed-loop ShadowHand joint/wrist execution actions.
- Action space
Two-stage dexterous control: Stage 1 generates diverse 6-DoF wrist poses and ShadowHand joint articulation proposals; Stage 2 executes closed-loop goal-conditioned ShadowHand actions in Isaac Gym.
- Trained / supported embodiments
- ShadowHand (simulated dexterous robotic hand in Isaac Gym / MJCF)
- Training datasets
- UniDexGrasp synthesized dexterous grasp dataset (DFCData + ShadowHand MJCF on PKU mirror; 3,200 training object instances, 141 seen-category test instances, 100 unseen-category test instances)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| UniDexGrasp Stage-1 Grasp Proposal Networks (GraspIPDF, GraspGlow, ContactNet — train from scratch) | pytorch | Not stated | weights-not-released | Not pinned |
| UniDexGrasp Stage-2 Goal-Conditioned Execution Policy (PPO teacher + DAgger vision student — train from scratch) | pytorch | Not stated | weights-not-released | Not pinned |
Neither the GitHub READMEs nor the project page link pretrained weight checkpoints; the PKU mirror provides dataset/MJCF assets for training from scratch.
Is this for you?
- Use cases
- Training Stage-1 probabilistic dexterous grasp proposal generators (GraspIPDF, GraspGlow, ContactNet) on DFCData
- Training Stage-2 ShadowHand state-based PPO teachers and distilling point-cloud vision policies via DAgger in Isaac Gym
- Benchmarking universal tabletop dexterous grasping across 3,200 training objects, 141 seen-category test objects, and 100 unseen-category test objects
- Known limitations
- No pretrained model checkpoints are released in the GitHub repository or on the PKU dataset mirror; users must train both Stage-1 proposal networks and Stage-2 RL/DAgger policies from scratch.
- Licensed under CC-BY-NC-4.0 (declared in root README with no standalone root LICENSE file), restricting use to non-commercial purposes.
- Depends on legacy software stacks: Ubuntu 20.04, Python 3.8, PyTorch 1.10 / CUDA 11.3, PyTorch3D 0.6.2, and NVIDIA Isaac Gym Preview Release 3.
- Simulation-only tabletop ShadowHand pipeline; no real-robot hardware deployment code is included.
- Safety and limitations
- CC-BY-NC-4.0 notice in README prohibits commercial use; there is no separate root LICENSE file in the repository.
- Pretrained weights are not released, requiring full simulation training from scratch before evaluation.
- Evaluated and implemented only in Isaac Gym Preview Release 3 for a simulated ShadowHand; no physical robot safety controllers or sim-to-real hardware wrappers are provided.
- Open Source Robotics has not downloaded the PKU dataset mirror, installed Isaac Gym, trained models, or run inference.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA CUDA GPU required (CUDA 11.3 + Isaac Gym Preview Release 3 + PyTorch3D 0.6.2 + pointnet2_ops; exact VRAM minimum not stated)
- Edge / consumer capable: No
CUDA 11.3 and NVIDIA Isaac Gym GPU requirements stated in installation instructions; scalar VRAM minimum and latency are omitted.
- Fine-tuning
- Supported: Yes
- Methods: Stage-1 training from scratch (GraspIPDF, GraspGlow, ContactNet via dexgrasp_generation/network/train.py); Stage-2 PPO state/vision RL training and DAgger state-to-vision distillation via dexgrasp_policy/dexgrasp/script/run_train_*.sh
- Recipe: Upstream fine-tuning documentation
- Host computer
Ubuntu 20.04 host with NVIDIA CUDA 11.3-compatible GPU capable of running Isaac Gym Preview Release 3; exact VRAM requirements are not stated in READMEs.
- Operating system
- Linux
dexgrasp_generation/README.md explicitly states tested on Ubuntu 20.04 and installs linux-64 PyTorch3D; Isaac Gym targets Linux.
- Dependencies
- Ubuntu 20.04, Python 3.8, PyTorch 1.10.0, CUDA 11.3
- PyTorch3D 0.6.2, pytorch_kinematics, nflows, CSDF (Stage 1)
- NVIDIA Isaac Gym Preview Release 3 and Pointnet2_PyTorch pointnet2_ops_lib (Stage 2)
Setup & usage
Open the upstream getting-started guide [2] · Model card / documentation [1]
- For Stage 1 (dexgrasp_generation), create a Python 3.8 conda env on Ubuntu 20.04 with PyTorch 1.10.0 / CUDA 11.3, PyTorch3D 0.6.2, pytorch_kinematics, nflows, and CSDF.
- Download the ShadowHand MJCF and DFCData grasp labels from the PKU mirror into dexgrasp_generation/data/ and train GraspIPDF, GraspGlow, and ContactNet.
- For Stage 2 (dexgrasp_policy), create a Python 3.8 conda env, install NVIDIA Isaac Gym Preview Release 3, install dexgrasp and pointnet2_ops_lib.
- Run script/run_train_ppo_state.sh, run_train_ppo_vision.sh, or run_train_dagger.sh in dexgrasp_policy/dexgrasp/.
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| UniDexGrasp Tabletop Dexterous Grasping in Isaac Gym (CVPR 2023, arXiv:2303.00938) | Over 60% average grasp success rate across thousands of tabletop object instances (seen and unseen categories) with minimal generalization gap. | Upstream-reported simulation evaluation in arXiv:2303.00938 abstract using 3,200 train / 141 seen-category test / 100 unseen-category test object splits; OSR has not reproduced training or evaluation. |
Project notes
Two-stage architecture and legacy simulation stack
UniDexGrasp decouples dexterous grasping from raw object point clouds into two stages. Stage 1 (dexgrasp_generation) factorizes grasp pose synthesis into three modules trained on the synthesized DFCData dataset and ShadowHand MJCF assets: GraspIPDF (Implicit PDF over SO(3) rotations), GraspGlow (normalizing flow via nflows over wrist translation and finger articulation), and ContactNet (PointNet++ contact refinement). Stage 2 (dexgrasp_policy) replaces classical motion planning with a goal-conditioned RL execution policy in NVIDIA Isaac Gym, training a state-based PPO teacher with state canonicalization and an object curriculum before distilling it via DAgger into a PointNet++ vision-based student policy. [2] [3] [5]
Because no pretrained checkpoint links were found in the checked READMEs or project page as of 2026-11-21 (and mirror payloads were not downloaded or inspected), reproducing the full pipeline from those materials requires training both stages. Both sub-projects rely on legacy 2021–2023 environments: Stage 1 was tested on Ubuntu 20.04 with Python 3.8, PyTorch 1.10.0, CUDA 11.3, PyTorch3D 0.6.2, pytorch_kinematics, nflows, and CSDF, while Stage 2 requires Python 3.8, NVIDIA Isaac Gym Preview Release 3, and Pointnet2_PyTorch (pointnet2_ops_lib). The supplied READMEs do not specify minimum GPU VRAM or training wall-clock time, so the checked sources do not support a training compute or cost estimate. [2] [3] [6]
Upstream-reported simulation performance
In arXiv:2303.00938, the authors evaluate the integrated two-stage pipeline across tabletop splits comprising 3,200 training object instances, 141 seen-category test instances, and 100 unseen-category test instances, reporting an average grasp success rate above 60% with minimal generalization gap. All benchmark figures are upstream-reported in simulation; Open Source Robotics has not downloaded the PKU dataset mirror, installed Isaac Gym, run training, fine-tuning, or inference, or tested UniDexGrasp on hardware. [3] [5]
What we checked
Source researched only; no installation, training, fine-tuning, inference, benchmark, or physical hardware test performed. No affiliate links or reused media.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- May 25, 2024 [4]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- commit snapshot; no GitHub release / 36c9bfcf7cedc987dd15e2384c548e51d5aadc21 · software repository
Separate license matrix
Include UniDexGrasp as a foundational inspectable dexterous grasping training codebase and dataset release, while clearly flagging that pretrained weights are not released, licensing is CC-BY-NC-4.0, and the stack targets legacy Isaac Gym Preview Release 3.
| Layer | Availability / license |
|---|---|
| Code | available · CC-BY-NC-4.0 (declared in root README; no root LICENSE file) Root README states 'This work and the dataset are licensed under CC BY-NC 4.0'; no standalone root LICENSE file is present in the repository. [1] [4] |
| Model weights | not found · License not fully assessed No pretrained model checkpoints are published in the repository READMEs or project page; only training code and dataset/MJCF files are distributed. [2] [3] [6] |
| Training datasets | available · CC-BY-NC-4.0 (declared in root README) PKU mirror URL verified HTTP 200 (provides ShadowHand MJCF and DFCData grasp labels); archive payloads were not downloaded. [1] [2] [6] |
| Documentation | available · CC-BY-NC-4.0 (declared in root README) Root README plus sub-READMEs in dexgrasp_generation/ and dexgrasp_policy/. [1] [2] [3] |
Non-commercial license notice
The repository has no standalone root LICENSE file, but the root README.md explicitly states that "This work and the dataset are licensed under CC BY-NC 4.0," restricting code and data use to non-commercial purposes. The repository has had no commits since 2024-05-25 and no versioned GitHub releases. [1] [4]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 UniDexGrasp root README
Links to CVPR 2023 paper, dexgrasp_generation and dexgrasp_policy subdirectories, and declares CC BY-NC 4.0 license for work and dataset.Revision, date & where cited
EPIC Lab, Peking University · github.com · revision
36c9bfc· Accessed Nov 21, 2026
Used for: Overview, Is this for you?, Setup & usage, Separate license matrix - S002 UniDexGrasp dexgrasp_generation/README.md
Ubuntu 20.04, Python 3.8, PyTorch 1.10.0, CUDA 11.3, PyTorch3D 0.6.2, nflows, CSDF setup; PKU mirror data download (mjcf, DFCData); training commands for GraspIPDF, GraspGlow, and ContactNet.Revision, date & where cited
EPIC Lab, Peking University · github.com · revision
36c9bfc· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 4 more - S003 UniDexGrasp dexgrasp_policy/README.md
Isaac Gym Preview Release 3, Python 3.8, pointnet2_ops_lib setup; object splits (3,200 train, 141 seen-cat test, 100 unseen-cat test); PPO state/vision and DAgger distillation scripts.Revision, date & where cited
EPIC Lab, Peking University · github.com · revision
36c9bfc· Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S004 GitHub repository metadata and pinned commit
Last commit 2024-05-25; no root LICENSE file; not archived; no GitHub releases.Revision, date & where cited
PKU-EPIC/UniDexGrasp / GitHub · github.com · revision
36c9bfc· main / · Accessed Nov 21, 2026
Used for: Overview, Is this for you?, What we checked, Separate license matrix - S005 UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy (arXiv:2303.00938)
Two-stage tabletop dexterous grasping from point clouds with state canonicalization, object curriculum, and teacher-student distillation; reports >60% average success rate across thousands of object instances.Revision, date & where cited
EPIC Lab, Peking University (CVPR 2023 / arXiv) · arxiv.org · arXiv:2303.00938v2 abstract · Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Upstream-reported benchmarks, Two-stage architecture and legacy simulation stack and 1 more - S006 UniDexGrasp project page and PKU mirror reachability
Project page and PKU mirror URL returned HTTP 200; no pretrained weight links appear on the project page or READMEs.Revision, date & where cited
Peking University Open Source Mirror · mirrors.pku.edu.cn · HTTP GET 2026-11-21 · Accessed Nov 21, 2026
Used for: Overview, Checkpoints, Is this for you?, Two-stage architecture and legacy simulation stack and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile for UniDexGrasp (training code + dataset only, weights-not-released, CC-BY-NC-4.0).