MolmoAct

Project notes

Keep the environment and output contract explicit

The repository recommends its Dockerfile for reproducing training environments. The inspected D/O cards illustrate an inference route with transformers 4.52, einops, torchvision and accelerate. They state that published checkpoints use FP32 weights while illustrating BF16 loading for inference. This is source guidance, not a GPU-memory minimum established by this catalog. [1] [30] [31]

The card quickstarts load processor/model code with trust_remote_code enabled and demonstrate parsing generated depth tokens, visual traces and actions. They pass a named action unnormalization key. Inspect that code and normalization contract before adapting the output to any robot interface; parsed numbers are not independently verified physical commands. No remote model code or inference example was executed here. [30] [31]

Those examples use side and wrist images and a prescribed reasoning prompt. The card action parser uses unnorm_key="molmoact"; that example key does not establish compatibility with arbitrary robot actions. For adaptation, the repository supplies dataset_statistics.json through --norm_stats_path. Keep the selected dataset statistics with the checkpoint and action convention. [1] [30] [31]

The repository's manual setup names Python 3.11 and PyTorch, and its adaptation workflow preprocesses data before fine-tuning. One documented fine-tuning configuration uses eight A100/H100 GPUs. Treat that as an upstream workload example, not a measured minimum; batch size must be chosen for available hardware. No training environment or preprocessing was run here. [1]

Adaptation preprocessing additionally requires a LeRobot-format dataset, lerobot 0.3.3, Depth Anything V2 with its checkpoint and a MolmoAct VQVAE checkpoint. README section 4.1.1 supplies the preprocessing command that produces the processed dataset and normalization statistics. These are separate prerequisites from the inference-only card example and were not downloaded or executed here. [1]

Separate release claims and reuse scopes

Both inspected checkpoint cards declare Apache-2.0, as does the code repository. Their preview wording and promises of later artifacts coexist with later training-release updates in the repository; neither text alone establishes complete reproduction of every result. Data mixtures that include OXE and other sources need their own rights review rather than inheriting one code or checkpoint declaration. [1] [2] [30] [31]

No weight binaries or training datasets were downloaded, and no policy adaptation, evaluation, inference quality or physical behavior was tested.