MolmoAct
Robotics modelAction-reasoning model source with distinct7B-D/7B-O backbones and pretrain/post-training checkpoint routes. [1] [3]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Researchers studying action reasoning, policy adaptation and robot-learning evaluation [1]
- Parameters
- 8,119,417,808 (~8.12B F32 for 7B-D variants) [32]
- Embodiments
- Franka single-arm (MolmoAct Dataset, 93 tasks), Google Robot (SimplerEnv), LIBERO simulation (Franka, via LoRA… [1] [31]
- Source rights
- Named licenses are documented by part. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- Ecosystem
- LeRobot
Overview
MolmoAct (allenai/molmoact) is an open-weights action-reasoning Vision-Language-Action model family from the Allen Institute for AI. Rather than predicting actions directly, MolmoAct autoregressively generates depth perception tokens, a 2D end-effector visual trajectory trace, and a tokenized 7-DoF robot action. Its two primary variants use distinct backbones: MolmoAct-7B-D-0812 pairs Qwen/Qwen2.5-7B with google/siglip2-so400m-patch14-384 (8,119,417,808 F32 parameters, 30.67 GB safetensors on disk), whereas MolmoAct-7B-O-0812 pairs allenai/OLMo-2-1124-7B with openai/clip-vit-large-patch14-336 (7,656,630,272 F32 parameters, 28.53 GB safetensors on disk). [1] [30] [31] [32]
Model card summary
- Architecture
Autoregressive action-reasoning VLA generating depth perception tokens, 2D end-effector trajectory traces, and tokenized actions; 7B-D uses Qwen2.5-7B + SigLIP2-SO400M-384, while 7B-O uses OLMo-2-1124-7B + CLIP-ViT-L/14-336.
- Base model
Qwen/Qwen2.5-7B + google/siglip2-so400m-patch14-384 (7B-D); allenai/OLMo-2-1124-7B + openai/clip-vit-large-patch14-336 (7B-O)
- Parameter count
8,119,417,808 (~8.12B F32 for 7B-D variants); 7,656,630,272 (~7.66B F32 for 7B-O-0812) per HF safetensors metadata
Exact tensor counts verified via Hugging Face API.
- Input modalities
- rgblanguage
Multi-view RGB images (side + wrist) and structured reasoning prompt.
- Output modalities
- ee-pose-deltagripper-commandwaypoints
Generates depth perception tokens, 2D visual trajectory waypoints (parse_trace), and 7-DoF action vector (parse_action).
- Action space
Autoregressively generated depth tokens, 2D end-effector trajectory trace, and 7-DoF robot action vector unnormalized via dataset_statistics.json / unnorm_key (e.g., molmoact).
- Trained / supported embodiments
- Franka single-arm (MolmoAct Dataset, 93 tasks)
- Google Robot (SimplerEnv)
- LIBERO simulation (Franka, via LoRA post-training)
- Training datasets
- allenai/MolmoAct-Pretraining-Mixture (OXE subset)
- allenai/MolmoAct-Midtraining-Mixture (10k single-arm Franka trajectories across 93 tasks)
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| allenai/MolmoAct-7B-D-0812 | safetensors | 30.67 GB | open | 936bcc40a2d9 |
| allenai/MolmoAct-7B-O-0812 | safetensors | 28.53 GB | open | b614d9041b68 |
| allenai/MolmoAct-7B-D-Pretrain-0812 | safetensors | 30.26 GB | open | e97f9e3bb0b2 |
| allenai/MolmoAct-7B-D-Pretrain-RT-1-0812 | safetensors | 30.26 GB | open | 5eca3376a325 |
All four ungated FP32 safetensors checkpoints verified via Hugging Face API.
Is this for you?
- Use cases
- Downstream post-training / LoRA adaptation of 7B-D-0812 or 7B-O-0812 on custom manipulation datasets
- Inspectable spatial action reasoning (depth tokens + 2D trajectory trace before action execution)
- SimplerEnv (Google Robot) and LIBERO benchmark evaluation
- Known limitations
- 7B-O uses OLMo2/CLIP,7B-D uses Qwen2.5/SigLIP2; do not treat them as identical checkpoints.
- Model-card preview language and later repository training-release updates have different scopes/dates.
- Code/checkpoint declarations do not settle every OXE/data-mixture right; no model payload, inference or physical behavior verified.
Editorial scope assessment from the cited implementation/documentation; no execution or compatibility test.
- Safety and limitations
- 7B-O (OLMo-2/CLIP) and 7B-D (Qwen2.5/SigLIP2) use distinct backbones and parameter counts; select the checkpoint matching the intended adaptation or SimplerEnv evaluation scope.
- Inference quickstarts require trust_remote_code=True and a strict reasoning prompt; parsed actions require matching dataset_statistics.json / unnorm_key.
- Data mixtures containing Open X-Embodiment and MolmoAct datasets require per-dataset rights review beyond the Apache-2.0 code and model-card declarations.
Compute & fine-tuning
- Inference compute
- Documented hardware: CUDA GPU (FP32 checkpoints cast to BF16 at load in quickstart)
- Precision:
bf16 - Edge / consumer capable: No
Published in FP32 (28.5-30.7 GB on disk) and cast to BF16 for inference; exact minimum VRAM not stated.
- Fine-tuning
- Supported: Yes
- Methods: LoRA adaptation (8x A100/H100 example in README; r=32, alpha=16 in LIBERO recipe); Full-parameter fine-tuning (supported route; hardware example not assigned)
- Recipe: Upstream fine-tuning documentation
Adaptation preprocessing requires LeRobot 0.3.3, Depth Anything V2, and MolmoAct VQVAE checkpoints.
- Host computer
CUDA GPU (FP32 safetensors checkpoints cast to BF16 for inference; documented LoRA adaptation recipe illustrates 8x A100/H100 GPUs).
- Operating system
- Linux
Dockerfile and torchrun Linux training workflow documented.
[1] · Editorial assessment
- Dependencies
- Repository recommends its Dockerfile for reproducing training environment
- Inspected D/O quickstarts name transformers4.52, einops, torchvision and accelerate
- Model cards publish FP32 weights with BF16 loading illustrated; actual GPU-memory minimum not measured
Setup & usage
Open the upstream getting-started guide [1]
- Choose7B-O or7B-D and the intended adaptation/evaluation checkpoint; use its model card and pinned implementation together, then inspect parsed actions before any hardware integration.
- Preserve exact source/dependency/model versions and evaluate the selected workflow before connecting physical hardware or reusing data.
Editorial scope assessment from the cited implementation/documentation; no execution or compatibility test.
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| SimplerEnv (Google Robot) and LIBERO (arXiv:2508.07917) | See arXiv:2508.07917 (upstream provides 7B-D-Pretrain-0812 for zero-shot SimplerEnv, 7B-D-Pretrain-RT-1-0812 for RT-1 fine-tuned SimplerEnv, and LIBERO post-training recipes) | Upstream-reported evaluation claims in arXiv:2508.07917 and repository README; not independently verified by OSR. |
Project notes
Checkpoints, output contract, and preprocessing
Four ungated FP32 safetensors checkpoints are verified on Hugging Face Hub: 7B-D-0812 and 7B-O-0812 are intended for downstream fine-tuning, while 7B-D-Pretrain-0812 (30.26 GB) and 7B-D-Pretrain-RT-1-0812 (30.26 GB) target zero-shot and RT-1-fine-tuned SimplerEnv Google Robot evaluations respectively. The illustrated inference workflow runs on CUDA and installs einops, torchvision, and accelerate alongside transformers==4.52, loading the FP32 weights in BF16 with trust_remote_code=True. It feeds side and wrist RGB images with a prescribed reasoning prompt and decodes outputs via parse_depth, parse_trace, and parse_action(..., unnorm_key="molmoact"). For custom-data adaptation, the repository's manual training environment specifies Python 3.11 and PyTorch (with a Dockerfile recommended), and preprocessing requires lerobot 0.3.3, Depth Anything V2, and a MolmoAct VQVAE checkpoint to produce processed data and dataset_statistics.json; those statistics must be retained during checkpoint conversion (passing --norm_stats_path to add missing statistics or replace existing ones for the custom dataset) and matched via unnorm_key at inference. Inspecting parsed actions offline is recommended as a next step before any hardware integration. [1] [30] [31] [32]
Fine-tuning compute and upstream-reported benchmarks
Upstream's documented adaptation recipe enables LoRA (including an 8x A100/H100 example workload—not a measured minimum—and r=32, alpha=16 for LIBERO post-training), while full-parameter fine-tuning is also supported as a separate route. Minimum inference VRAM, inference latency, and control frequency are not established in this review, and the listed safetensors sizes (28.53–30.67 GB) are disk sizes rather than runtime memory requirements. Benchmark claims across SimplerEnv (Google Robot), LIBERO, and real-robot tasks in arXiv:2508.07917 are upstream-reported; Open Source Robotics has not downloaded checkpoint binaries or run training, fine-tuning, inference, or robot evaluations. [1] [31]
What we checked
Source research only. Selected implementation/model text, repository tree and license declarations were inspected. No upstream compilation, dependency installation, simulator, network/USB/device integration, data collection, benchmark or physical robot test was performed. Source files are linked rather than redistributed; external services and paid products are not endorsed.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
Separate license matrix
Distinct robot-facing model contribution with actual implementation/model content and a bounded documented next step. Review covers source artifacts, not operational performance or a complete tested deployment.
| Layer | Availability / license |
|---|---|
| Code | available · Apache-2.0 Declaration for the scoped source implementation/documentation; dependencies, external services, model/data assets and inherited file-specific terms require separate checks. No blanket redistribution or operational clearance. [1] [2] [3] |
| Model weights | available · Apache-2.0 All four inspected Hugging Face checkpoints (7B-D-0812, 7B-O-0812, 7B-D-Pretrain-0812, 7B-D-Pretrain-RT-1-0812) are ungated and declare Apache-2.0; weight binaries not downloaded or executed. [30] [31] [32] |
| Training datasets | available · Mixed (MolmoAct data mixtures and OXE constituent dataset licenses apply) Pretraining includes Open X-Embodiment subsets alongside the MolmoAct Dataset; constituent dataset licenses apply. [1] [31] |
| Documentation | available · Apache-2.0 Declaration for the scoped source implementation/documentation; dependencies, external services, model/data assets and inherited file-specific terms require separate checks. No blanket redistribution or operational clearance. [1] [2] [3] |
Separate code, weights, and data-mixture rights
Both the allenai/molmoact repository and all four inspected Hugging Face checkpoints declare Apache-2.0 licenses. Pretraining and mid-training draw on Open X-Embodiment subsets alongside the 10k-trajectory single-arm Franka MolmoAct Dataset; constituent dataset licenses must be reviewed separately before reuse. [1] [2] [30] [31] [32]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 Project README
Revision, date & where cited
allenai/molmoact contributors · github.com · revision
a5cf980· Project README at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Is this for you?, Compute & fine-tuning and 6 more - S002 License declaration
Revision, date & where cited
allenai/molmoact contributors · github.com · revision
a5cf980· License declaration at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Separate license matrix - S003 Selected implementation excerpt
Revision, date & where cited
allenai/molmoact contributors · github.com · revision
a5cf980· Selected implementation excerpt at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage, Separate license matrix - S004 Repository artifact tree
Revision, date & where cited
allenai/molmoact contributors · github.com · revision
a5cf980· Repository artifact tree at · Accessed Nov 1, 2026
Used for: Is this for you?, Compute & fine-tuning, Setup & usage - S030 Published checkpoint card and metadata
Revision, date & where cited
allenai/molmoact contributors · huggingface.co · Published checkpoint card and metadata at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 4 more - S031 Published checkpoint card and metadata
Revision, date & where cited
allenai/molmoact contributors · huggingface.co · Published checkpoint card and metadata at · Accessed Nov 1, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S032 Hugging Face API metadata: allenai/MolmoAct-7B-* checkpoints
7B-D variants: 8,119,417,808 F32 params (30.26-30.67 GB safetensors, base Qwen/Qwen2.5-7B + google/siglip2-so400m-patch14-384); 7B-O-0812: 7,656,630,272 F32 params (28.53 GB safetensors, base allenai/OLMo-2-1124-7B + openai/clip-vit-large-patch14-336); all ungated, license: apache-2.0.Revision, date & where cited
Hugging Face (allenai) · huggingface.co · HF API metadata for 7B-D-0812, 7B-O-0812, 7B-D-Pretrain-0812, 7B-D-Pretrain-RT-1-0812 · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Checkpoints, output contract, and preprocessing and 1 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Migrated 0.7.0-draft.5 -> 0.8.0-draft.1 (kind=model); added verified HF API metadata (S032) for 4 checkpoints while preserving historical source dates.