Open Source RoboticsRobots, software & learning

H2O & OmniH2O (Human-to-Humanoid Whole-Body Control)

Robotics model

CC-BY-NC-4.0 (non-commercial) Isaac Gym sim-to-real RL and DAgger distillation codebase for Unitree H1 whole-body humanoid teleoperation and autonomy (H2O and OmniH2O); upstream releases motion retargeting scripts, filtered AMASS motion data, and hardware wrappers, but not pretrained humanoid policy checkpoints. [1] [2] [3] [4] [5]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Humanoid whole-body control researchers; Sim-to-real RL engineers; Humanoid teleoperation system developers [1] [5]
Parameters
Config-dependent compact MLP / RNN actor-critic (not published as a single scalar [1]
Embodiments
Unitree H1 / H1 EDU humanoid (with optional Inspire Hands and DM-J4310-2EC wrists) [1]
Source rights
Some licenses are unresolved. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
Ecosystem
NVIDIA Jetson
License
CC-BY-NC-4.0 (noncommercial) · license not fully assessed for datasets · Details by part

Requirements · Research details and dates

On this page

Overview

H2O (IROS 2024) and OmniH2O (CoRL 2024) form CMU LeCAR Lab's whole-body humanoid teleoperation and reinforcement learning control stack for the Unitree H1. Distributed in LeCAR-Lab/human2humanoid, the repository combines AMASS-to-H1 motion retargeting (via PHC), massively parallel Isaac Gym PPO training, DAgger teacher-student policy distillation, and real-robot hardware interface snippets. [1] [4] [5]

Model card summary

Architecture

Two-stage RL and DAgger distillation architecture in RSL-RL/Legged Gym: a privileged PPO teacher policy (num_observations: 913, num_privileged_obs: 990) distilled into history-conditioned MLP students (0, 5, 25, or 50 history steps; default 25-step student has num_observations: 1665) or recurrent ActorCriticRecurrent (LSTM/GRU) students supporting 3-point VR, 8-point, or 23-point keypoint tracking; also supports direct RL H2O 8-point tracking (num_observations: 138).

[1] [2]

Parameter count

Config-dependent compact MLP / RNN actor-critic (not published as a single scalar; varies across teacher, 0/5/25/50-step MLP students, and LSTM/GRU variants)

Inferred from README Hydra commands showing varying observation dimensions (90 to 3240) and MLP vs ActorCriticRecurrent LSTM/GRU architectures without a single published scalar.

[1] · Editorial assessment

Input modalities
proprioceptionimu

Policy takes proprioceptive joint/IMU state history plus retargeted kinematic keypoint targets (VR, RGB pose estimation, or odometry are handled externally).

[1] [2]

Output modalities
joint-positions

PD target joint positions (control_type: 'P', action_scale: 0.25) for Unitree H1 body joints.

[2]

Action space

Continuous PD joint position targets (control_type: 'P', action_scale: 0.25) for Unitree H1 whole-body joints (hip, knee, ankle, torso, shoulder, elbow); Inspire hands and Damiao wrists are controlled via separate hardware wrappers.

[1] [2]

Control frequency

50 Hz (sim.dt: 0.005 s, control.decimation: 4)

[2]

Trained / supported embodiments
  • Unitree H1 / H1 EDU humanoid (with optional Inspire Hands and DM-J4310-2EC wrists)

[1]

Training datasets
  • AMASS motion capture dataset (SMPL+H G format, retargeted via PHC)
  • Filtered retargeted Unitree H1 motion dataset (amass_phc_filtered.pkl, CMU Box)

Raw AMASS and SMPL assets must be downloaded separately from MPI under their respective licenses.

[1]

Interfaces
  • Hydra CLI training and playback scripts (legged_gym/scripts/train_hydra.py, play_hydra.py)
  • Unitree SDK2, Inspire Hand C++, Damiao wrist C++, and ZED mini odometry Python wrappers

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
H2O / OmniH2O humanoid policy checkpoints (not released upstream; train teacher and student in Isaac Gym)pytorchNot statedweights-not-releasedNot pinned

Repository only contains inherited legged_gym anydrive_v3_lstm.pt actuator weights, not trained H2O or OmniH2O humanoid policies.

[1]

Is this for you?

Use cases
  • Real-time whole-body humanoid teleoperation via VR headset, RGB camera, or sparse keypoints
  • Privileged teacher-to-student DAgger distillation for sparse-observation humanoid tracking
  • AMASS human motion retargeting and feasibility filtering for Unitree H1

[1] [4] [5]

Known limitations
  • Non-commercial license: root repository is licensed under CC-BY-NC-4.0, prohibiting commercial use (e.g., commercial product demos).
  • No pretrained H2O or OmniH2O humanoid policy checkpoints are released; users must train teacher and student policies in Isaac Gym.
  • Depends on proprietary Isaac Gym Preview 4, Python 3.8, and numpy < 1.24; full retargeting additionally requires separate registration for AMASS and SMPL.
  • Hardware deployment code is tailored to the authors' Unitree H1 EDU + ZED mini + Orin NX + Inspire Hand + Damiao wrist rig and relies on external repos for VR/RGB pose estimation.

[1] [3]

Safety and limitations
  • CC-BY-NC-4.0 license prohibits commercial use.
  • No pretrained humanoid checkpoints are distributed; sim-to-real transfer requires training from scratch in Isaac Gym and careful gain/safety validation on physical humanoids.
  • Hardware wrappers are specific to the authors' Unitree H1 EDU + Inspire Hand + Damiao wrist configuration and are not supported for tuning on other robots.

[1] [3]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA Jetson Orin NX (16 GB) onboard Unitree H1 EDU
  • Precision: fp32
  • Edge / consumer capable: Yes

README documents real-world deployment on an onboard Jetson Orin NX (16 GB).

[1]

Fine-tuning
  • Supported: Yes
  • Methods: Privileged PPO teacher RL training in Isaac Gym followed by DAgger student distillation (legged_gym/scripts/train_hydra.py)
  • Recipe: Upstream fine-tuning documentation

[1]

Host computer

Single CUDA GPU for Isaac Gym training (default num_envs=4096, sim_device=cuda:0); real-world deployment runs on NVIDIA Jetson Orin NX (16 GB) onboard Unitree H1 EDU.

[1] [2]

Operating system
  • Linux

Inferred from Isaac Gym Preview 4 Linux bindings (gym_38.so) and Jetson Orin NX deployment.

[1] · Editorial assessment

Dependencies
  • Python 3.8
  • PyTorch
  • NVIDIA Isaac Gym Preview 4
  • numpy==1.20 (< 1.24)
  • setuptools==59.5.0
  • Hydra / TensorBoard / WandB
  • Bundled rsl_rl, legged_gym, and phc

[1]

Skills
  • Python
  • PyTorch / RSL-RL
  • Isaac Gym / Legged Gym
  • Unitree SDK2 / C++ (for hardware deployment)

Inferred from README installation, Hydra RL commands, and C++ Inspire Hand / Damiao wrist hardware files.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Create a Python 3.8 conda environment and install PyTorch and NVIDIA Isaac Gym Preview 4
  • Install bundled rsl_rl, legged_gym, phc in editable mode, plus numpy==1.20 (<1.24), setuptools==59.5.0, and requirements.txt
  • Download filtered motion data (amass_phc_filtered.pkl from CMU Box) or retarget AMASS + SMPL via grad_fit_h1_shape.py and grad_fit_h1.py
  • Train the privileged teacher policy with legged_gym/scripts/train_hydra.py, then distill the sim-to-real student policy via DAgger (train.distill=True)

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
H2O real-world Unitree H1 whole-body RGB teleoperation (arXiv:2403.04436v1)Zero-shot sim-to-real teleoperation of dynamic whole-body motions (walking, back jumping, kicking, turning, waving, pushing, boxing)Upstream-reported qualitative/system evaluation on Unitree H1; not independently verified by OSR.
OmniH2O universal teleoperation and OmniH2O-6 autonomy suite (arXiv:2406.08858v1)Demonstrated real-world whole-body teleoperation (VR, RGB, verbal) and autonomous skill learning across six everyday tasks (OmniH2O-6)Upstream-reported paper result on Unitree H1 with dexterous hands; not independently verified by OSR.

[4] [5]

Project notes

Privileged teacher and sparse-observation student distillation

At the pinned commit (750f1fa05264), training runs in Isaac Gym (sim.dt: 0.005, control.decimation: 4, yielding a 50 Hz policy loop) with PD joint-position actions (control_type: 'P', action_scale: 0.25) across 4,096 parallel environments. OmniH2O first trains a privileged PPO teacher (num_observations: 913, num_privileged_obs: 990) and then distills it via DAgger (train.distill=True) into sim-to-real student policies supporting configurable history lengths (0, 5, 25, or 50 steps; default 25-step MLP uses num_observations: 1665), recurrent ActorCriticRecurrent (LSTM/GRU) backbones, and 3-point VR, 8-point, or 23-point tracking targets. The repository also preserves the direct RL H2O 8-point tracking recipe (num_observations: 138). Exact parameter counts are not stated as single scalars and vary by history/architecture configuration. [1] [2]

Hardware stack, setup caveats, and upstream-reported results

Real-world deployment is documented for a Unitree H1 EDU with an onboard NVIDIA Jetson Orin NX (16 GB), ZED mini odometry, Inspire dexterous hands, and Damiao DM-J4310-2EC wrist actuators; upstream notes the hardware wrappers are specific to that rig and will not be tuned for other robots. Training requires NVIDIA Isaac Gym Preview 4, Python 3.8, and numpy<1.24, while custom motion retargeting requires registering separately for AMASS and SMPL (a pre-filtered amass_phc_filtered.pkl file is linked on CMU Box). In arXiv:2403.04436v1 and arXiv:2406.08858v1, the authors report zero-shot sim-to-real teleoperation of dynamic motions (walking, kicking, waving, boxing, sports) and autonomous skill learning across the OmniH2O-6 task suite. All reported results are upstream-reported; Open Source Robotics has not run training, fine-tuning, inference, or robot evaluations for this profile. [1] [4] [5]

What we checked

None. Prepared from public upstream sources without hands-on training, inference, or robot execution.

Updates

Upstream last updated
Feb 21, 2025 [1]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
LeCAR-Lab/human2humanoid@750f1fa052641f0fde43669d50cb4e407dabe6c8 · software repository

Separate license matrix

Publicly available whole-body humanoid RL and DAgger codebase with filtered retargeted AMASS motion data, included as partial because the repository uses a non-commercial CC-BY-NC-4.0 license, publishes no pretrained humanoid policy checkpoints, and depends on proprietary Isaac Gym Preview 4.

LayerAvailability / license
Codeavailable · CC-BY-NC-4.0
Root license is Creative Commons Attribution-NonCommercial 4.0 International (non-commercial, not OSI-approved); inherited legged_gym and rsl_rl directories retain BSD-3-Clause notices from ETH Zurich and NVIDIA. [1] [3]
Model weightsunavailable · License not fully assessed
No pretrained H2O or OmniH2O humanoid policy checkpoints are published in the repository or linked from README. [1]
Training datasetsavailable · License not fully assessed
Filtered retargeted Unitree H1 motion file (amass_phc_filtered.pkl) is linked via CMU Box without a separately declared dataset license; full retargeting requires registering for AMASS and SMPL under their upstream non-commercial licenses. [1]
Documentationavailable · CC-BY-NC-4.0
[1] [3]

Licensing and weight availability boundaries

The root repository and documentation are licensed under Creative Commons Attribution-NonCommercial 4.0 International (LICENSE-CC-BY-NC-4.0.md), which explicitly forbids commercial use; bundled legged_gym and rsl_rl retain inherited BSD-3-Clause notices from ETH Zurich and NVIDIA. Upstream does not publish pretrained H2O or OmniH2O humanoid policy checkpoints (only an inherited ANYmal anydrive_v3_lstm.pt actuator network appears in legged_gym), and the linked CMU Box motion file does not declare a separate dataset license. [1] [3]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile drafted from LeCAR-Lab/human2humanoid@750f1fa05264.