HumanPlus (HST & HIT Humanoid Shadowing and Imitation)
Robotics modelFull-stack Stanford humanoid learning repository combining the Humanoid Shadowing Transformer (HST, Isaac Gym PPO low-level whole-body controller) and Humanoid Imitation Transformer (HIT, ACT/DETR-derived stereo RGB visuomotor behavior cloning policy) plus Unitree H1 hardware code and Google Drive demonstrations; no root license or pretrained checkpoints are published. [1] [2] [3] [4]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Humanoid robotics and imitation learning researchers; Whole-body teleoperation and bimanual manipulation practitioners; Unitree H1 hardware developers [1] [4]
- Parameters
- Not published as a single scalar (HIT uses shared ResNet18 + 6-layer, 512-dim DETR decoder [1] [2]
- Embodiments
- Customized 33-DoF 180 cm Unitree H1 humanoid (19-DoF H1 body + dexterous hands + stereo egocentric cameras) [1] [2] [4]
- Source rights
- Some licenses are unresolved. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- Ecosystem
- ROS 2, NVIDIA Jetson, MuJoCo
- License
- not fully assessed for software, datasets, documentation · Details by part
Overview
HumanPlus is Stanford University's full-stack humanoid learning system (MarkFzp/humanplus) that links low-level whole-body human shadowing with high-level autonomous egocentric skill imitation on a customized 33-DoF Unitree H1 humanoid. The repository bundles two distinct policy stacks—the Humanoid Shadowing Transformer (HST) for Isaac Gym reinforcement learning and the Humanoid Imitation Transformer (HIT) for stereo RGB behavior cloning—together with Unitree ROS 2 hardware scripts and a Google Drive link to real-world HIT demonstrations. [1] [4]
Model card summary
- Architecture
Two-level humanoid policy suite: (1) HST (Humanoid Shadowing Transformer) — PPO ActorCriticTransformer in Isaac Gym (obs_context_len: 8, 19-DoF H1 PD control) for real-time whole-body human shadowing; and (2) HIT (Humanoid Imitation Transformer) — ACT/DETR-derived decoder-only transformer (--no_encoder, dec_layers: 6, hidden_dim: 512, dim_feedforward: 512, chunk_size: 50) with shared ResNet18 stereo RGB backbones, L1 action loss, and future image feature prediction loss (feature_loss_weight: 0.005).
- Base model
ResNet18 (shared stereo visual backbone in HIT)
HIT uses --backbone resnet18 --same_backbones; HST is trained from scratch via PPO.
- Parameter count
Not published as a single scalar (HIT uses shared ResNet18 + 6-layer, 512-dim DETR decoder; HST uses ActorCriticTransformer over 8-step observation context)
Inferred from HIT CLI flags (--backbone resnet18 --dec_layers 6 --hidden_dim 512) and HST ActorCriticTransformer config; no scalar parameter count is reported in README.
- Input modalities
- rgbproprioceptionimu
HIT consumes stereo egocentric RGB (cam_left, cam_right) + task-dependent proprioception (35-D with wrist for data_fold_clothes; 33-D qpos + hand_action for other tasks, with task-specific state_mask); HST consumes joint proprioception, IMU orientation/angular velocity, and retargeted human pose targets.
- Output modalities
- joint-positionsgripper-command
HST outputs 19-D H1 PD joint position targets; HIT outputs masked 40-D action chunks covering 10 leg, 1 waist, 8 arm, 5 IMU/orientation, and 16 dexterous hand/gripper dimensions.
- Action space
HST: 19-D continuous PD joint position targets (action_scale: 1.0, decimation: 4). HIT: masked 40-D whole-body + dexterous hand action chunks (chunk_size: 50; 10 leg + 1 waist + 8 arm + 5 IMU + 16 hand/gripper).
- Control frequency
HST: 50 Hz (sim.dt: 0.005 s, decimation: 4); HIT: 25 Hz (DT: 0.04 s, FPS: 25 in HIT/constants.py)
- Trained / supported embodiments
- Customized 33-DoF 180 cm Unitree H1 humanoid (19-DoF H1 body + dexterous hands + stereo egocentric cameras)
- Training datasets
- 40-hour human motion datasets (e.g., ACCAD) for HST low-level RL
- HumanPlus real-world teleoperated HIT demonstration dataset (Google Drive: fold clothes, rearrange objects, two-robot greeting, warehouse, etc.)
- Interfaces
- HST Isaac Gym CLI (train.py, play.py)
- HIT imitation training CLI (imitate_episodes_h1_train.py)
- Unitree SDK2 / unitree_ros2 hardware script (hardware_whole_body.py)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| HST and HIT policy checkpoints (not released upstream; train in Isaac Gym / PyTorch) | pytorch | Not stated | weights-not-released | Not pinned |
No pretrained HST or HIT checkpoints are committed or linked; hardware README instructs users to place their own trained policy.pt in hardware-script/ckpt.
Is this for you?
- Use cases
- Real-time RGB human-to-humanoid shadowing via HST
- Collecting whole-body teleoperated humanoid manipulation demonstrations
- Training autonomous egocentric stereo-vision humanoid skill policies via HIT
- Known limitations
- No root repository LICENSE file is present; only inherited HIT/detr/LICENSE (Apache-2.0) and robot asset licenses appear in subdirectories, leaving reuse terms for HumanPlus code ambiguous.
- No pretrained HST or HIT checkpoints are published; users must train both low-level RL shadowing and high-level imitation policies from scratch.
- HST requires proprietary NVIDIA Isaac Gym v4, while hardware deployment pins legacy Python 3.8, NVIDIA JetPack 4.6.1, PyTorch 1.11.0, and unitree_ros2.
- Human pose and hand estimation rely on external repositories (WHAM and HaMeR) not bundled in this repo.
- Safety and limitations
- No root repository license is declared, creating legal uncertainty for commercial or derivative reuse.
- No pretrained HST or HIT checkpoints are released; reproducing real-robot performance requires training both low-level RL and high-level imitation policies.
- Hardware code targets a customized 33-DoF Unitree H1 running legacy JetPack 4.6.1 / PyTorch 1.11.0 and depends on external WHAM/HaMeR pose estimators.
Compute & fine-tuning
- Inference compute
- Documented hardware: NVIDIA Jetson (JetPack 4.6.1, PyTorch 1.11.0) for low-level HST control; single CUDA GPU for HIT training/inference
- Precision:
fp32 - Edge / consumer capable: Yes
- Fine-tuning
- Supported: Yes
- Methods: HST PPO RL training in Isaac Gym (HST/legged_gym/legged_gym/scripts/train.py); HIT supervised behavior cloning on stereo RGB + proprioception demonstrations (HIT/imitate_episodes_h1_train.py)
- Recipe: Upstream fine-tuning documentation
- Host computer
Single CUDA GPU for HST Isaac Gym RL (num_envs=2048, --sim_device cuda:0) and HIT training (--gpu_id 0, batch_size 48); onboard NVIDIA Jetson with JetPack 4.6.1 for low-level hardware control.
- Operating system
- Linux (Ubuntu / NVIDIA Jetson JetPack 4.6.1)
- Dependencies
- Python 3.8 / 3.8.10
- PyTorch (1.11.0 for Jetson hardware)
- NVIDIA Isaac Gym v4 (HST)
- mujoco==2.3.7 and dm_control==1.0.14 (HIT)
- Bundled HIT/detr, HST/rsl_rl, and HST/legged_gym
- unitree_sdk2, unitree_ros2, NVIDIA JetPack 4.6.1 (hardware)
- Skills
- Python
- PyTorch / Isaac Gym / Legged Gym
- ACT / DETR imitation learning
- ROS 2 / Unitree SDK2 / Jetson setup
Inferred from three-part HST, HIT, and hardware README instructions.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- For HST: install Isaac Gym v4 inside HST/, then install HST/rsl_rl and HST/legged_gym in editable mode and run train.py / play.py
- For HIT: create Python 3.8.10 conda env with PyTorch, mujoco==2.3.7, dm_control==1.0.14, install HIT/detr in editable mode, download the Google Drive dataset, and run imitate_episodes_h1_train.py
- For hardware deployment: install unitree_sdk2, unitree_ros2, JetPack 4.6.1, PyTorch 1.11.0, place trained policy.pt in hardware-script/ckpt, and run hardware_whole_body.py
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Real-world autonomous humanoid skills on customized 33-DoF 180 cm Unitree H1 (arXiv:2406.10454v1) | 60–100% success rates across tasks (wearing a shoe to stand up and walk, unloading warehouse racks, folding a sweatshirt, rearranging objects, typing, greeting another robot) using up to 40 demonstrations | Upstream-reported real-robot evaluation on customized 33-DoF Unitree H1; not independently verified by OSR. |
Project notes
HST low-level shadowing and HIT visuomotor imitation
At the pinned commit (ff7148903303), HST (HST/legged_gym/legged_gym/envs/h1/h1_config.py) trains an ActorCriticTransformer PPO policy across 2,048 Isaac Gym environments (obs_context_len = 8, 19-DoF H1 PD position targets with action_scale = 1.0, dt = 0.005, decimation = 4, giving a 50 Hz control rate) on 40-hour human motion datasets so the robot can shadow human operators in real time from an RGB camera. HIT (HIT/policy.py, HIT/constants.py) builds on ACT/DETR as a decoder-only transformer (--no_encoder, dec_layers = 6, hidden_dim = 512, dim_feedforward = 512, chunk_size = 50) with a shared ResNet18 stereo image backbone (cam_left, cam_right) and task-dependent proprioceptive state vectors—35-D for data_fold_clothes (including wrist observations) and 33-D (qpos, hand_action) for the other listed tasks, filtered by task-specific state_mask and action_mask vectors—trained with L1 action loss plus a future image feature MSE loss (feature_loss_weight = 0.005) to output masked 40-D whole-body and dexterous hand actions at 25 Hz (DT = 0.04). Exact parameter counts are not published as single scalars in the README. [1] [2]
Setup requirements and upstream-reported benchmarks
Running the full stack spans three separate environments: proprietary NVIDIA Isaac Gym v4 for HST, Python 3.8.10 with mujoco==2.3.7 and dm_control==1.0.14 for HIT, and legacy NVIDIA JetPack 4.6.1 / PyTorch 1.11.0 / unitree_ros2 on an onboard Jetson for low-level deployment, plus external pose estimators (WHAM and HaMeR) not bundled in this repository. In arXiv:2406.10454v1, the authors report 60–100% autonomous success rates across six real-world tasks—wearing a shoe to stand up and walk, unloading warehouse racks, folding a sweatshirt, rearranging objects, typing, and greeting another robot—using up to 40 demonstrations per task. All reported metrics are upstream-reported; Open Source Robotics has not run training, fine-tuning, inference, or robot evaluations for this profile. [1] [4]
What we checked
None. Prepared from public upstream sources without hands-on training, inference, or robot execution.
Updates
- Upstream last updated
- Jul 1, 2024 [1]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- Feb 19, 2027
- Reviewed revision
- MarkFzp/humanplus@ff7148903303ac2951857cf0b7df686d77323917 · software repository
Separate license matrix
Public full-stack humanoid shadowing (HST) and stereo-vision imitation (HIT) repository with downloadable Google Drive demonstrations, included as partial because the repository lacks a root license file, publishes no pretrained HST/HIT checkpoints, and depends on proprietary Isaac Gym v4.
| Layer | Availability / license |
|---|---|
| Code | available · License not fully assessed No root LICENSE file exists in MarkFzp/humanplus; individual files/subtrees carry HIT/detr/LICENSE (Apache-2.0), file-level BSD-3-Clause headers (e.g., HST h1_config.py), and robot asset licenses under HST/legged_gym/resources/robots/, leaving uncovered code terms unspecified. [1] [3] |
| Model weights | unavailable · License not fully assessed No pretrained HST or HIT policy checkpoints are released in the repository or linked from README. [1] |
| Training datasets | available · License not fully assessed HIT real-world humanoid demonstration dataset is linked on Google Drive without a separately declared dataset license. [1] |
| Documentation | available · License not fully assessed No root license is declared for repository documentation. [1] |
Licensing and weight availability boundaries
Crucially, MarkFzp/humanplus contains no root LICENSE file at ff7148903303. While individual subtrees and files carry upstream notices—including HIT/detr/LICENSE (Apache-2.0 from Facebook Research DETR), file-level BSD-3-Clause headers in HST/legged_gym/legged_gym/envs/h1/h1_config.py, and inherited robot asset license files—reuse terms for HumanPlus code and documentation not covered by those notices remain unspecified and require a file-level license mapping before broader reuse. Upstream also does not publish pretrained HST or HIT checkpoints (the hardware README directs users to place their own trained policy.pt in hardware-script/ckpt), and the linked Google Drive HIT dataset does not declare a separate dataset license. [1] [2] [3]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 MarkFzp/humanplus GitHub repository and README.md
Documents HST (Isaac Gym v4 + legged_gym/rsl_rl), HIT (Python 3.8.10, MuJoCo 2.3.7, DETR/ACT ResNet18 6-layer decoder, Google Drive dataset), Unitree ROS2/JetPack 4.6.1 hardware setup, and external WHAM/HaMeR pose links.Revision, date & where cited
Zipeng Fu / Stanford (GitHub) · github.com · revision
ff71489· README.md at commit ff7148903303 (2024-07-01) · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S002 HumanPlus HST and HIT model configurations (h1_config.py, HIT/policy.py, HIT/constants.py)
HST configures ActorCriticTransformer PPO on 19-DoF H1 (dt: 0.005, decimation: 4 -> 50 Hz, obs_context_len: 8, num_envs: 2048); HIT configures stereo RGB + 33-D state to 40-D masked action chunks at 25 Hz (DT: 0.04) with L1 + future feature MSE loss.Revision, date & where cited
Zipeng Fu / Stanford (GitHub) · github.com · revision
ff71489· HST/.../h1_config.py, HIT/policy.py, HIT/constants.py at commit ff7148903303 · Accessed Nov 21, 2026
Used for: Model card summary, Compute & fine-tuning, HST low-level shadowing and HIT visuomotor imitation, Separate license matrix - S003 MarkFzp/humanplus repository tree and HIT/detr/LICENSE
Verified absence of any root LICENSE file; only bundled HIT/detr/LICENSE (Apache-2.0) and robot asset license files exist in subdirectories.Revision, date & where cited
Zipeng Fu / Facebook Research (GitHub) · github.com · revision
ff71489· Git tree and HIT/detr/LICENSE at commit ff7148903303 · Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S004 HumanPlus: Humanoid Shadowing and Imitation from Humans (arXiv:2406.10454v1)
Reports 60–100% autonomous task success rates across 6 real-world humanoid tasks on a customized 33-DoF 180 cm humanoid using up to 40 demonstrations collected via RGB shadowing.Revision, date & where cited
arXiv (2406.10454v1) · arxiv.org · arXiv abstract 2406.10454v1 · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Is this for you?, Upstream-reported benchmarks and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile drafted from MarkFzp/humanplus@ff7148903303.