HOVER (Versatile Neural Whole-Body Controller)
Robotics modelApache-2.0 NVIDIA Research Isaac Lab 2.0.0 extension (neural_wbc) for training multi-mode generalist and specialist whole-body humanoid controllers on Unitree H1 with MuJoCo sim-to-sim and Unitree H1 sim-to-real wrappers; upstream does not publish pretrained checkpoints or retargeted AMASS motion data. [1] [2] [3] [4]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Humanoid whole-body control researchers; Isaac Lab / Isaac Sim robotics engineers; Sim-to-sim and sim-to-real Unitree H1 developers [1] [2]
- Parameters
- Compact [512, 256, 128] ELU MLP for teacher and student policies (exact scalar parameter count not stated in README [2]
- Source rights
- Named licenses are documented by part. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- Apache-2.0 · Details by part
Overview
HOVER (*Humanoid Versatile Controller*) is an Apache-2.0 NVIDIA Research Isaac Lab 2.0.0 extension (NVlabs/HOVER, package neural_wbc) for training multi-mode and specialist whole-body controllers on the 19-DoF Unitree H1 humanoid. Building on OmniH2O's kinematic motion-imitation pipeline, HOVER consolidates multiple humanoid control interfaces—root velocity tracking, upper-body joint tracking, and sparse VR/keypoint tracking—into a single distilled student policy while adding MuJoCo sim-to-sim validation and a Unitree H1 sim-to-real deployment wrapper. [1] [3] [4]
Model card summary
- Architecture
Two-stage multi-mode policy distillation framework in Isaac Lab (neural_wbc): a privileged PPO teacher MLP (ActorCritic, actor_hidden_dims: [512, 256, 128], critic_hidden_dims: [512, 256, 128], ELU, observation_space: 913, state_space: 990) distilled via DAgger into a [512, 256, 128] ELU student MLP with 25-step observation history (single_history_dim: 63) and selectable command masks (DISTILL_MASK_MODES_ALL: exbody, humanplus, h2o, omnih2o).
- Parameter count
Compact [512, 256, 128] ELU MLP for teacher and student policies (exact scalar parameter count not stated in README; varies with masked student observation size)
Derived from PolicyCfg and StudentPolicyTrainerCfg hidden layer definitions.
[2] · Editorial assessment
- Input modalities
- proprioceptionimu
25-step history of robot proprioception/IMU state (63-D per step) plus masked kinematic command targets (head/hands, joints, or root velocity depending on mode).
- Output modalities
- joint-positions
19-D continuous PD joint position actions (action_space: 19, action_scale: 0.25) for Unitree H1.
- Action space
19-D continuous PD joint position targets (action_space: 19, action_scale: 0.25) for Unitree H1 legs, torso, shoulders, and elbows.
- Control frequency
50 Hz (dt: 0.005 s, decimation: 4 in neural_wbc_env_cfg.py)
- Trained / supported embodiments
- Unitree H1 humanoid (19 DoF; Isaac Lab, MuJoCo, and real-hardware wrapper)
- Training datasets
- AMASS motion capture dataset (SMPL+H G format, user-downloaded and retargeted to Unitree H1 via ./retarget_h1.sh)
Not bundled in the repo due to AMASS licensing; users must register for AMASS and SMPL.
- Interfaces
- Isaac Lab CLI scripts (train_teacher_policy.py, train_student_policy.py, play.py, eval.py)
- YAML config overwrite system (--env_config_overwrite)
- MuJoCo sim-to-sim wrapper (neural_wbc/mujoco_wrapper) and Unitree H1 hardware wrapper (neural_wbc/hw_wrappers)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| HOVER teacher and student policy checkpoints (not released upstream; train in Isaac Lab) | pytorch | Not stated | weights-not-released | Not pinned |
README explicitly notes there is no provided teacher policy in the repo, and neither GitHub release attaches weight files.
Is this for you?
- Use cases
- Training generalist multi-mode whole-body controllers supporting ExBody, HumanPlus, H2O, and OmniH2O command masks
- Training specialist OmniH2O or HumanPlus mode policies in Isaac Lab 2.0.0
- Sim-to-sim validation in MuJoCo and sim-to-real deployment on Unitree H1
- Known limitations
- No pretrained teacher or student checkpoints are distributed in the repository or GitHub releases; users must train both stages from scratch.
- Neither retargeted AMASS motions nor the toy stable_punch.pkl file are distributed due to AMASS license restrictions; retargeting full AMASS takes up to 4 days on 32 CPU cores.
- Default config trains an OmniH2O specialist policy; generalist multi-mode training requires editing distill_mask_modes in neural_wbc_env_cfg_h1.py, and upstream recommends disabling sparsity randomization to avoid motion ambiguity.
- Sim-to-real hardware wrapper currently supports only the Unitree H1 robot.
- Safety and limitations
- No pretrained checkpoints or retargeted motion datasets are published; reproducing full AMASS policies requires multi-day CPU retargeting and ~23–53 GPU-hours of teacher training.
- Sparsity randomization is disabled by default because upstream notes the current randomization strategy can cause motion ambiguity.
- Sim-to-real wrapper only supports Unitree H1 and requires careful validation before physical robot execution.
Compute & fine-tuning
- Inference compute
- Minimum VRAM: 13 GB
- Documented hardware: NVIDIA RTX 4090 / RTX A6000 / L40 (training); ~13 GB VRAM for 1024-env evaluation; single-env MuJoCo/H1 deployment
- Precision:
fp32 - Edge / consumer capable: Yes
13 GB VRAM is reported for 1024-environment batched Isaac Lab evaluation; single-robot student MLP runs on-robot for Unitree H1 sim-to-real deployment.
- Fine-tuning
- Supported: Yes
- Methods: Privileged PPO teacher training (train_teacher_policy.py) and multi-mode or specialist DAgger student distillation (train_student_policy.py)
- Recipe: Upstream fine-tuning documentation
Single GPU required; README benchmarks RTX 4090, RTX A6000, and L40.
- Host computer
Single NVIDIA GPU required for training and evaluation (~13 GB VRAM for 1024-env evaluation; README benchmarks teacher 100k iters at 23.3h on RTX 4090 / 44.6h on L40 / 52.8h on RTX A6000, and student 10k iters at 0.27–0.50h).
- Operating system
- Linux (Ubuntu 22.04)
- Dependencies
- Ubuntu 22.04 / Python 3.10
- NVIDIA Isaac Sim 4.5.0
- NVIDIA Isaac Lab v2.0.0
- Customized bundled third_party/rsl_rl
- MuJoCo (for sim-to-sim validation)
- unitree_sdk2_python (for Unitree H1 deployment)
- Skills
- Python 3.10
- NVIDIA Isaac Lab / Isaac Sim
- PyTorch / RSL-RL
- MuJoCo / Unitree SDK2
Inferred from README installation, Isaac Lab Docker workflow, MuJoCo wrapper, and Unitree H1 deployment instructions.
[1] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [1] · Model card / documentation [1]
- Install Isaac Sim 4.5.0 and Isaac Lab v2.0.0 on Ubuntu 22.04 with Python 3.10, and export ISAACLAB_PATH
- Clone NVlabs/HOVER with --recurse-submodules and run ./install_deps.sh
- Register and download AMASS (SMPL+H G) and SMPL v1.1.0, then run ./retarget_h1.sh (optionally with --motions-file punch.yaml) and re-run ./install_deps.sh
- Train the privileged teacher (scripts/rsl_rl/train_teacher_policy.py), distill the student (scripts/rsl_rl/train_student_policy.py), and validate in MuJoCo or deploy on Unitree H1
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Multi-mode humanoid whole-body control evaluation suite (Isaac Lab, MuJoCo sim-to-sim, and Unitree H1 sim-to-real in arXiv:2410.21229v2) | Consolidates ExBody, HumanPlus, H2O, and OmniH2O command modes into a unified distilled policy with seamless mode transitions | Upstream-reported paper result and built-in eval.py metrics (success rate, mpjpe_g/l/pa, joint/root errors); no pretrained checkpoint is released for direct score reproduction. Not independently verified by OSR. |
Project notes
Multi-mode teacher-student distillation in Isaac Lab
At the pinned commit (8088f6cfb42a), neural_wbc_env_cfg.py and neural_wbc_env_cfg_h1.py run 19-DoF PD joint-position control (action_space = 19, action_scale = 0.25, dt = 0.005, decimation = 4, giving a 50 Hz policy frequency). Stage 1 trains a privileged PPO teacher MLP (ActorCritic, [512, 256, 128] ELU layers, observation_space = 913, state_space = 990). Stage 2 distills the teacher via DAgger into a [512, 256, 128] ELU student MLP conditioned on a 25-step proprioceptive history (single_history_dim = 63) and configurable command masks (DISTILL_MASK_MODES_ALL: exbody, humanplus, h2o, omnih2o). By default, the repository trains an OmniH2O specialist policy; enabling generalist multi-mode training requires switching distill_mask_modes to DISTILL_MASK_MODES_ALL, and upstream advises keeping sparsity randomization disabled to avoid motion ambiguity. [1] [2]
Compute requirements, validation, and upstream-reported scope
HOVER targets Ubuntu 22.04, Python 3.10, Isaac Sim 4.5.0, and Isaac Lab v2.0.0 (with Docker container instructions and unit tests included). Upstream benchmarks 100,000 teacher iterations at 23.3 hours on an NVIDIA RTX 4090 (44.6 hours on L40, 52.8 hours on RTX A6000) and 10,000 student iterations at 0.27–0.50 hours, with 1,024-environment evaluation using roughly 13 GB of GPU memory. The repository ships a single-environment MuJoCo evaluator (neural_wbc/mujoco_wrapper) and a Unitree H1 hardware wrapper (neural_wbc/hw_wrappers). In arXiv:2410.21229v2, the authors report consolidating multiple control modes into a unified controller with seamless transitions across modes. All reported results and benchmarks are upstream-reported; Open Source Robotics has not run training, fine-tuning, inference, or robot evaluations for this profile. [1] [4]
What we checked
None. Prepared from public upstream sources without hands-on training, inference, or robot execution.
Updates
Separate license matrix
Apache-2.0 NVIDIA Research Isaac Lab whole-body controller with MuJoCo sim-to-sim and Unitree H1 sim-to-real wrappers, included as partial because upstream publishes neither pretrained teacher/student checkpoints nor retargeted AMASS motion data.
| Layer | Availability / license |
|---|---|
| Code | available · Apache-2.0 Root LICENCE file is Apache-2.0; third_party/rsl_rl retains BSD-3-Clause and third_party/mujoco_viewer retains its upstream license. [1] [3] |
| Model weights | unavailable · License not fully assessed Upstream does not distribute pretrained teacher or student policy checkpoints. [1] |
| Training datasets | unavailable · License not fully assessed README states that due to AMASS license limitations, neither retargeted AMASS motions nor stable_punch.pkl are distributed; users must obtain AMASS and SMPL directly under their upstream licenses. [1] |
| Documentation | available · Apache-2.0 [1] [3] |
Licensing and artifact availability boundaries
The codebase and documentation are released under the Apache License 2.0 (LICENCE at 8088f6cfb42a), while bundled third_party/rsl_rl and third_party/mujoco_viewer retain their upstream licenses. Crucially, upstream publishes no pretrained teacher or student checkpoints (neither in the repository nor in its initial-release / deployment-release GitHub releases), and due to AMASS license restrictions it distributes neither retargeted AMASS motions nor stable_punch.pkl; users must register for AMASS and SMPL and run ./retarget_h1.sh (up to 4 days on 32 CPU cores for the full dataset). [1] [3]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 NVlabs/HOVER GitHub repository and README.md
Documents Isaac Sim 4.5.0 / Isaac Lab 2.0.0 / Ubuntu 22.04 setup, AMASS license restriction, absence of bundled teacher checkpoint, RTX 4090/A6000/L40 training timings, 13 GB eval VRAM, MuJoCo sim-to-sim wrapper, and Unitree H1 sim-to-real deployment.Revision, date & where cited
NVIDIA Research (GitHub) · github.com · revision
8088f6c· README.md at commit 8088f6cfb42a (2025-07-30) · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 7 more - S002 HOVER Isaac Lab environment and teacher/student policy configs
Defines 19-DoF H1 PD control (dt: 0.005, decimation: 4 -> 50 Hz, action_scale: 0.25), [512, 256, 128] ELU teacher and student MLPs, 25-step student history (single_history_dim: 63), and DISTILL_MASK_MODES_ALL (exbody, humanplus, h2o, omnih2o).Revision, date & where cited
NVIDIA Research (GitHub) · github.com · revision
8088f6c· neural_wbc_env_cfg_h1.py, neural_wbc_env_cfg.py, teacher_policy_cfg.py, student_policy_trainer_cfg.py at commit 8088f6cfb42a · Accessed Nov 21, 2026
Used for: Model card summary, Is this for you?, Multi-mode teacher-student distillation in Isaac Lab - S003 NVlabs/HOVER LICENCE
Apache License 2.0.Revision, date & where cited
NVIDIA Research (GitHub) · github.com · revision
8088f6c· LICENCE at commit 8088f6cfb42a · Accessed Nov 21, 2026
Used for: Overview, Separate license matrix - S004 HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots (arXiv:2410.21229v2)
Introduces HOVER multi-mode policy distillation unifying navigation, loco-manipulation, and tabletop manipulation control modes into a single humanoid controller.Revision, date & where cited
arXiv (2410.21229v2) · arxiv.org · arXiv abstract 2410.21229v2 · Accessed Nov 21, 2026
Used for: Overview, Is this for you?, Upstream-reported benchmarks, Compute requirements, validation, and upstream-reported scope
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model profile drafted from NVlabs/HOVER@8088f6cfb42a.