Open Source RoboticsRobots, software & learning

MolmoAct

Robotics model

Action-reasoning model source with distinct7B-D/7B-O backbones and pretrain/post-training checkpoint routes. [1] [3]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Researchers studying action reasoning, policy adaptation and robot-learning evaluation [1]
Parameters
8,119,417,808 (~8.12B F32 for 7B-D variants) [32]
Embodiments
Franka single-arm (MolmoAct Dataset, 93 tasks), Google Robot (SimplerEnv), LIBERO simulation (Franka, via LoRA… [1] [31]
Source rights
Named licenses are documented by part. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
Ecosystem
LeRobot

Requirements · Research details and dates

On this page

Overview

MolmoAct (allenai/molmoact) is an open-weights action-reasoning Vision-Language-Action model family from the Allen Institute for AI. Rather than predicting actions directly, MolmoAct autoregressively generates depth perception tokens, a 2D end-effector visual trajectory trace, and a tokenized 7-DoF robot action. Its two primary variants use distinct backbones: MolmoAct-7B-D-0812 pairs Qwen/Qwen2.5-7B with google/siglip2-so400m-patch14-384 (8,119,417,808 F32 parameters, 30.67 GB safetensors on disk), whereas MolmoAct-7B-O-0812 pairs allenai/OLMo-2-1124-7B with openai/clip-vit-large-patch14-336 (7,656,630,272 F32 parameters, 28.53 GB safetensors on disk). [1] [30] [31] [32]

Model card summary

Architecture

Autoregressive action-reasoning VLA generating depth perception tokens, 2D end-effector trajectory traces, and tokenized actions; 7B-D uses Qwen2.5-7B + SigLIP2-SO400M-384, while 7B-O uses OLMo-2-1124-7B + CLIP-ViT-L/14-336.

[1] [30] [31] [32]

Base model

Qwen/Qwen2.5-7B + google/siglip2-so400m-patch14-384 (7B-D); allenai/OLMo-2-1124-7B + openai/clip-vit-large-patch14-336 (7B-O)

[30] [31] [32]

Parameter count

8,119,417,808 (~8.12B F32 for 7B-D variants); 7,656,630,272 (~7.66B F32 for 7B-O-0812) per HF safetensors metadata

Exact tensor counts verified via Hugging Face API.

[32]

Input modalities
rgblanguage

Multi-view RGB images (side + wrist) and structured reasoning prompt.

[31]

Output modalities
ee-pose-deltagripper-commandwaypoints

Generates depth perception tokens, 2D visual trajectory waypoints (parse_trace), and 7-DoF action vector (parse_action).

[31]

Action space

Autoregressively generated depth tokens, 2D end-effector trajectory trace, and 7-DoF robot action vector unnormalized via dataset_statistics.json / unnorm_key (e.g., molmoact).

[1] [31]

Trained / supported embodiments
  • Franka single-arm (MolmoAct Dataset, 93 tasks)
  • Google Robot (SimplerEnv)
  • LIBERO simulation (Franka, via LoRA post-training)

[1] [31]

Training datasets
  • allenai/MolmoAct-Pretraining-Mixture (OXE subset)
  • allenai/MolmoAct-Midtraining-Mixture (10k single-arm Franka trajectories across 93 tasks)

[1] [31]

Interfaces
  • Hugging Face Transformers (AutoProcessor / AutoModelForImageTextToText with trust_remote_code=True)
  • parse_depth / parse_trace / parse_action helpers

[30] [31]

Checkpoints

CheckpointFormatSizeAccessPinned revision
allenai/MolmoAct-7B-D-0812safetensors30.67 GBopen936bcc40a2d9
allenai/MolmoAct-7B-O-0812safetensors28.53 GBopenb614d9041b68
allenai/MolmoAct-7B-D-Pretrain-0812safetensors30.26 GBopene97f9e3bb0b2
allenai/MolmoAct-7B-D-Pretrain-RT-1-0812safetensors30.26 GBopen5eca3376a325

All four ungated FP32 safetensors checkpoints verified via Hugging Face API.

[30] [31] [32]

Is this for you?

Use cases
  • Downstream post-training / LoRA adaptation of 7B-D-0812 or 7B-O-0812 on custom manipulation datasets
  • Inspectable spatial action reasoning (depth tokens + 2D trajectory trace before action execution)
  • SimplerEnv (Google Robot) and LIBERO benchmark evaluation

[1] [31]

Known limitations
  • 7B-O uses OLMo2/CLIP,7B-D uses Qwen2.5/SigLIP2; do not treat them as identical checkpoints.
  • Model-card preview language and later repository training-release updates have different scopes/dates.
  • Code/checkpoint declarations do not settle every OXE/data-mixture right; no model payload, inference or physical behavior verified.

Editorial scope assessment from the cited implementation/documentation; no execution or compatibility test.

[1] [2] [3] [4] [30] [31] · Editorial assessment

Safety and limitations
  • 7B-O (OLMo-2/CLIP) and 7B-D (Qwen2.5/SigLIP2) use distinct backbones and parameter counts; select the checkpoint matching the intended adaptation or SimplerEnv evaluation scope.
  • Inference quickstarts require trust_remote_code=True and a strict reasoning prompt; parsed actions require matching dataset_statistics.json / unnorm_key.
  • Data mixtures containing Open X-Embodiment and MolmoAct datasets require per-dataset rights review beyond the Apache-2.0 code and model-card declarations.

[1] [2] [30] [31]

Compute & fine-tuning

Inference compute
  • Documented hardware: CUDA GPU (FP32 checkpoints cast to BF16 at load in quickstart)
  • Precision: bf16
  • Edge / consumer capable: No

Published in FP32 (28.5-30.7 GB on disk) and cast to BF16 for inference; exact minimum VRAM not stated.

[30] [31]

Fine-tuning
  • Supported: Yes
  • Methods: LoRA adaptation (8x A100/H100 example in README; r=32, alpha=16 in LIBERO recipe); Full-parameter fine-tuning (supported route; hardware example not assigned)
  • Recipe: Upstream fine-tuning documentation

Adaptation preprocessing requires LeRobot 0.3.3, Depth Anything V2, and MolmoAct VQVAE checkpoints.

[1]

Host computer

CUDA GPU (FP32 safetensors checkpoints cast to BF16 for inference; documented LoRA adaptation recipe illustrates 8x A100/H100 GPUs).

[1] [30] [31]

Operating system
  • Linux

Dockerfile and torchrun Linux training workflow documented.

[1] · Editorial assessment

Dependencies
  • Repository recommends its Dockerfile for reproducing training environment
  • Inspected D/O quickstarts name transformers4.52, einops, torchvision and accelerate
  • Model cards publish FP32 weights with BF16 loading illustrated; actual GPU-memory minimum not measured

[1] [2] [3] [4] [30] [31]

Skills
  • Python
  • PyTorch
  • Hugging Face Transformers
  • Docker / multi-GPU torchrun

Inferred from documented Dockerfile, Transformers inference, and torchrun training workflows.

[1] [31] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1]

  • Choose7B-O or7B-D and the intended adaptation/evaluation checkpoint; use its model card and pinned implementation together, then inspect parsed actions before any hardware integration.
  • Preserve exact source/dependency/model versions and evaluate the selected workflow before connecting physical hardware or reusing data.

Editorial scope assessment from the cited implementation/documentation; no execution or compatibility test.

[1] [2] [3] [4] [30] [31] · Editorial assessment

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
SimplerEnv (Google Robot) and LIBERO (arXiv:2508.07917)See arXiv:2508.07917 (upstream provides 7B-D-Pretrain-0812 for zero-shot SimplerEnv, 7B-D-Pretrain-RT-1-0812 for RT-1 fine-tuned SimplerEnv, and LIBERO post-training recipes)Upstream-reported evaluation claims in arXiv:2508.07917 and repository README; not independently verified by OSR.

[1] [31]

Project notes

Checkpoints, output contract, and preprocessing

Four ungated FP32 safetensors checkpoints are verified on Hugging Face Hub: 7B-D-0812 and 7B-O-0812 are intended for downstream fine-tuning, while 7B-D-Pretrain-0812 (30.26 GB) and 7B-D-Pretrain-RT-1-0812 (30.26 GB) target zero-shot and RT-1-fine-tuned SimplerEnv Google Robot evaluations respectively. The illustrated inference workflow runs on CUDA and installs einops, torchvision, and accelerate alongside transformers==4.52, loading the FP32 weights in BF16 with trust_remote_code=True. It feeds side and wrist RGB images with a prescribed reasoning prompt and decodes outputs via parse_depth, parse_trace, and parse_action(..., unnorm_key="molmoact"). For custom-data adaptation, the repository's manual training environment specifies Python 3.11 and PyTorch (with a Dockerfile recommended), and preprocessing requires lerobot 0.3.3, Depth Anything V2, and a MolmoAct VQVAE checkpoint to produce processed data and dataset_statistics.json; those statistics must be retained during checkpoint conversion (passing --norm_stats_path to add missing statistics or replace existing ones for the custom dataset) and matched via unnorm_key at inference. Inspecting parsed actions offline is recommended as a next step before any hardware integration. [1] [30] [31] [32]

Fine-tuning compute and upstream-reported benchmarks

Upstream's documented adaptation recipe enables LoRA (including an 8x A100/H100 example workload—not a measured minimum—and r=32, alpha=16 for LIBERO post-training), while full-parameter fine-tuning is also supported as a separate route. Minimum inference VRAM, inference latency, and control frequency are not established in this review, and the listed safetensors sizes (28.53–30.67 GB) are disk sizes rather than runtime memory requirements. Benchmark claims across SimplerEnv (Google Robot), LIBERO, and real-robot tasks in arXiv:2508.07917 are upstream-reported; Open Source Robotics has not downloaded checkpoint binaries or run training, fine-tuning, inference, or robot evaluations. [1] [31]

What we checked

Source research only. Selected implementation/model text, repository tree and license declarations were inspected. No upstream compilation, dependency installation, simulator, network/USB/device integration, data collection, benchmark or physical robot test was performed. Source files are linked rather than redistributed; external services and paid products are not endorsed.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
May 11, 2026 [1]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
a5cf980 · combined hardware and software repository

Separate license matrix

Distinct robot-facing model contribution with actual implementation/model content and a bounded documented next step. Review covers source artifacts, not operational performance or a complete tested deployment.

LayerAvailability / license
Codeavailable · Apache-2.0
Declaration for the scoped source implementation/documentation; dependencies, external services, model/data assets and inherited file-specific terms require separate checks. No blanket redistribution or operational clearance. [1] [2] [3]
Model weightsavailable · Apache-2.0
All four inspected Hugging Face checkpoints (7B-D-0812, 7B-O-0812, 7B-D-Pretrain-0812, 7B-D-Pretrain-RT-1-0812) are ungated and declare Apache-2.0; weight binaries not downloaded or executed. [30] [31] [32]
Training datasetsavailable · Mixed (MolmoAct data mixtures and OXE constituent dataset licenses apply)
Pretraining includes Open X-Embodiment subsets alongside the MolmoAct Dataset; constituent dataset licenses apply. [1] [31]
Documentationavailable · Apache-2.0
Declaration for the scoped source implementation/documentation; dependencies, external services, model/data assets and inherited file-specific terms require separate checks. No blanket redistribution or operational clearance. [1] [2] [3]

Separate code, weights, and data-mixture rights

Both the allenai/molmoact repository and all four inspected Hugging Face checkpoints declare Apache-2.0 licenses. Pretraining and mid-training draw on Open X-Embodiment subsets alongside the 10k-trajectory single-arm Franka MolmoAct Dataset; constituent dataset licenses must be reviewed separately before reuse. [1] [2] [30] [31] [32]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Migrated 0.7.0-draft.5 -> 0.8.0-draft.1 (kind=model); added verified HF API metadata (S032) for 4 checkpoints while preserving historical source dates.