Open Source RoboticsRobots, software & learning

GVHMR

Robotics model

Recover human motion in world coordinates from video, with CUDA inference and noncommercial research source terms. [1]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Human-motion researchers preparing motion estimates for analysis or downstream retargeting [1]
Parameters
Not stated in upstream README or arXiv:2409.06662 abstract (released as gvhmr_siga24_release.ckpt plus external… [1] [2] [8]
Embodiments
Human SMPL / SMPL-X parametric body models (monocular RGB video input [2] [8]
Source rights
Some licenses are unresolved. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
License
Custom noncommercial research license (noncommercial) · license not fully assessed for model weights, datasets · Details by part

Requirements · Research details and dates

On this page

Overview

GVHMR (zju3dv/GVHMR, SIGGRAPH Asia 2024, arXiv:2409.06662) recovers world-grounded 3D human motion from monocular RGB video by estimating per-frame human poses in a Gravity-View (GV) coordinate system and transforming them into world coordinates via camera rotations. For robotics workflows, GVHMR is a perception-only motion estimator producing SMPL/SMPL-X parameters (hmr4d_results.pt)—a separate downstream retargeting pipeline is required to map human trajectories onto a humanoid robot. [1] [4] [5] [8]

Model card summary

Architecture

Per-frame Gravity-View (GV) coordinate human motion recovery network (SIGGRAPH Asia 2024) operating on monocular video with external YOLO detection, ViTPose-H 2D keypoints, HMR2 visual features, and SimpleVO (default) or optional DPVO camera rotation estimation.

[1] [2] [4] [8]

Parameter count

Not stated in upstream README or arXiv:2409.06662 abstract (released as gvhmr_siga24_release.ckpt plus external YOLO, ViTPose-H, and HMR2 preprocessing weights)

Google Drive checkpoint binaries were not downloaded to count parameters.

[1] [2] [8] · Editorial assessment

Input modalities
rgb

Monocular RGB video input (static camera with -s flag, or moving camera via default SimpleVO / optional --use_dpvo).

[1] [4] [8]

Output modalities
6d-posejoint-positions

Outputs SMPL/SMPL-X global root 6D orientation/translation and 3D human body joint rotations per frame in world and camera coordinates (hmr4d_results.pt; mapped to closest schema enums 6d-pose and joint-positions).

[2] [4] [5] [8] · Editorial assessment

Action space

Perception-only human motion estimator (no direct robot action output); outputs SMPL/SMPL-X world-grounded and camera-space human motion sequences (hmr4d_results.pt) requiring separate downstream retargeting for robots.

[2] [4] [5] [8]

Trained / supported embodiments
  • Human SMPL / SMPL-X parametric body models (monocular RGB video input; humanoid robot retargeting requires a separate downstream pipeline)

[2] [8]

Training datasets
  • Preprocessed human-motion training/evaluation datasets (3DPW, RICH, and EMDB evaluation suites referenced in README; upstream restricts its supplied preprocessed data to research use, while original datasets must be obtained from their owners under their respective terms)

[1] [2]

Interfaces
  • Python CLI (tools/demo/demo.py, demo_folder.py, tools/train.py)
  • PyTorch .pt / Lightning .ckpt files

[1] [4] [5]

Checkpoints

CheckpointFormatSizeAccessPinned revision
gvhmr_siga24_release.ckptpytorchNot statedopengvhmr_siga24_release (SIGGRAPH Asia 2024)

Hosted on Google Drive alongside preprocessing checkpoints (yolov8x.pt, vitpose-h-multi-coco.pth, hmr2, optional dpvo.pth); SMPL/SMPL-X body models require separate registration. Binary payloads were not downloaded.

[2] [5]

Is this for you?

Use cases
  • World-grounded 3D human motion recovery from monocular video
  • Preparing SMPL/SMPL-X motion trajectories for downstream humanoid retargeting research

[1] [8]

Known limitations
  • Source terms restrict use to educational, research and non-profit purposes.
  • The -s option assumes a static camera; default moving-camera estimation uses SimpleVO, not DPVO.
  • External assets retain separate terms.

Editorial assessment from cited documentation; no runtime testing.

[4] [7] [2] · Editorial assessment

Safety and limitations
  • Repository LICENSE restricts source use to educational, research, and non-profit purposes and requires any modification based on the work to remain open source and noncommercial.
  • SMPL and SMPL-X body models require separate user registration and licensing; original training datasets must be obtained from their owners under their respective terms, while upstream restricts its supplied preprocessed data to research use.
  • The -s demo flag assumes a static camera; moving-camera mode defaults to SimpleVO rather than DPVO (despite an older README sentence), with DPVO as an optional --use_dpvo path.
  • No checkpoint download, installation, training, inference, or robot retargeting was executed by Open Source Robotics.

[1] [2] [4] [5] [7]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA CUDA 12.1 GPU (PyTorch 2.3.0; training used 2x RTX 4090 for 420 epochs per README)
  • Edge / consumer capable: No

Demo runs on CUDA; exact minimum inference VRAM and latency_ms are not established.

[1] [2] [3] [4] · Editorial assessment

Fine-tuning

[1]

Host computer

Linux x86_64 with NVIDIA CUDA 12.1 GPU (PyTorch 2.3.0 + cu121); README notes gvhmr_siga24_release.ckpt was trained on 2x RTX 4090 for 420 epochs, while minimum inference VRAM is not stated.

[1] [2] [3] [4]

Operating system
  • Linux

requirements.txt pins a Linux x86_64 PyTorch3D wheel and CUDA 12.1.

[2] [3]

Dependencies
  • Python 3.10
  • PyTorch 2.3.0 with CUDA 12.1
  • Linux x86_64 PyTorch3D wheel in requirements
  • Separately obtained SMPL/SMPL-X and pretrained checkpoints

[2] [3]

Setup & usage

Open the upstream getting-started guide [2] · Model card / documentation [1]

  • Prepare the Python 3.10 environment and install the pinned source and requirements.
  • Place licensed body models and pretrained assets in the documented checkpoint folders.
  • Run the bundled static-camera tennis demo shown in the overview.

Editorial assessment from cited documentation; no runtime testing.

[1] [2] · Editorial assessment

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
3DPW, RICH, and EMDB human motion recovery evaluation suites (README reproduction commands; SIGGRAPH Asia 2024, arXiv:2409.06662)README documents test commands to reproduce 3DPW, RICH, and EMDB evaluations with gvhmr_siga24_release.ckpt (no numeric scores extracted in this review)Upstream-documented evaluation suites in README (S001); OSR has not extracted paper scores, downloaded checkpoints, or reproduced 3DPW/RICH/EMDB evaluations.

[1] [8]

Project notes

Checkpoints, preprocessing stack, and setup boundaries

At pinned commit ee960bb6e2ea2d381aa97f08e9b71ef320b624b1 (no GitHub releases listed), docs/INSTALL.md specifies a Python 3.10 environment pinning PyTorch 2.3.0 with CUDA 12.1 and a Linux x86_64 PyTorch3D wheel—making the supplied requirements Linux/CUDA-specific rather than portable to CPU or macOS. Running inference requires gvhmr_siga24_release.ckpt plus external preprocessing weights (yolov8x.pt, vitpose-h-multi-coco.pth, hmr2, and optional dpvo.pth) from the linked Google Drive folder, along with separately registered SMPL and SMPL-X body models. [2] [3] [5] [6]

The bundled static-camera example (python tools/demo/demo.py --video=docs/example_video/tennis.mp4 -s) uses -s to skip visual odometry. Although an older sentence in the README mentions DPVO for moving cameras, the reviewed code and demo.yaml default to SimpleVO, with DPVO available as an optional --use_dpvo route requiring extra installation steps. Minimum inference VRAM and exact parameter counts are not stated in the README, and weight binaries were not downloaded here. [1] [2] [4] [5]

Upstream-reported benchmarks and unverified execution

The README documents reproduction commands for testing gvhmr_siga24_release.ckpt (trained on 2× RTX 4090 GPUs for 420 epochs) on the 3DPW, RICH, and EMDB evaluation suites. All benchmark suites are upstream-documented; Open Source Robotics has not extracted paper scores, installed the environment, downloaded checkpoints, run training or inference, or reproduced 3DPW, RICH, or EMDB evaluations. [1]

What we checked

Source researched only; no installation, training, benchmark or physical hardware test performed. No affiliate links or reused media.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Sep 15, 2026 [6]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
commit snapshot; no GitHub releases listed / ee960bb6e2ea2d381aa97f08e9b71ef320b624b1 · software repository

Separate license matrix

Research-use source with explicit noncommercial scope; model/data rights separately unresolved.

LayerAvailability / license
Codeavailable · Custom noncommercial research license
Repository materials only; external components retain their own terms. [7]
Model weightsavailable · License not fully assessed
Download routes documented only; payloads not acquired. Separate body-model/checkpoint/dataset terms unresolved; data instructions specify research use. [2]
Training datasetsavailable · License not fully assessed
Download routes documented only; payloads not acquired. Separate body-model/checkpoint/dataset terms unresolved; data instructions specify research use. [2]
Documentationavailable · Custom noncommercial research license
Repository materials only; external components retain their own terms. [7]

Noncommercial license and separate data rights

The repository LICENSE limits code and documentation to educational, research, and non-profit purposes and requires any modification based on this work to remain open source and noncommercial; commercial use requires contacting the rights holder. External Google Drive checkpoints and SMPL/SMPL-X body models retain separate terms not settled by the repository license. For training and evaluation data, original datasets must be obtained from their owners under their respective terms, while upstream restricts use of its supplied preprocessed data to research. [2] [7]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Migrated from 0.7.0-draft.5 project to 0.8.0-draft.1 model; added Gravity-View architecture, gvhmr_siga24_release.ckpt entry, and upstream-reported 3DPW/RICH/EMDB benchmarks.