Open Source RoboticsRobots, software & learning

DexVLA

Robotics model

Vision-Language-Action architecture pairing an off-the-shelf Qwen2-VL-2B-Instruct backbone with a plug-in ScaleDP diffusion transformer action expert (~1B ScaleDP-H or ~400M/410M ScaleDP-L), releasing Stage-1 pretrained ScaleDP weights (CC-BY-NC-4.0) and three-stage training code under Attribution-NonCommercial terms. [1] [2] [3] [4]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Robotics and VLA researchers; Bimanual manipulation practitioners (ALOHA / AgileX-style HDF5 pipelines) [1]
Parameters
~3B combined (~2B Qwen2-VL-2B-Instruct VLM + ~1B ScaleDP-H policy head [1]
Embodiments
AgileX / ALOHA-style bimanual arms (evaluate/smart_eval_agilex.py, aloha_scripts/), Custom multi-camera HDF5 /… [1]
Source rights
Named licenses are documented by part. Includes noncommercial terms. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware

Requirements · Research details and dates

On this page

Overview

DexVLA is a Vision-Language-Action (VLA) architecture that combines an off-the-shelf Qwen/Qwen2-VL-2B-Instruct vision-language backbone with a plug-in ScaleDP diffusion transformer action expert (~1B ScaleDP-H or ~400M/410M ScaleDP-L), yielding roughly 3B combined parameters in the full configuration. Rather than releasing merged end-to-end robot weights, the authors publish the Stage-1 pretrained ScaleDP policy-head checkpoints (lesjie/scale_dp_h and lesjie/scale_dp_l) alongside the three-stage PyTorch training and evaluation codebase (juruobenruo/DexVLA at commit fc21a822f4c7). [1] [3] [4] [5]

Model card summary

Architecture

Qwen2-VL-2B-Instruct vision-language backbone (28 layers, hidden_size 1536, ViT depth 32) paired with a plug-in ScaleDP diffusion transformer action expert (1B ScaleDP-H or 410M ScaleDP-L) using DDIM denoising, plus optional substep reasoning tokens.

[1] [5]

Base model

Qwen/Qwen2-VL-2B-Instruct (VLM backbone) + ScaleDP-H / ScaleDP-L (Stage-1 diffusion policy head)

[1] [3] [4]

Parameter count

~3B combined (~2B Qwen2-VL-2B-Instruct VLM + ~1B ScaleDP-H policy head; or ~2.4B with ~400M/410M ScaleDP-L)

README reports ~2B Qwen2-VL, ~1B ScaleDP-H, and ~400M (table) / 410M (Smaller ScaleDP section) ScaleDP-L.

[1]

Input modalities
rgbproprioceptionlanguage

Multi-view RGB cameras (e.g., left, right, wrist at 480x640x3 in example HDF5), robot joint_positions/qpos/qvel proprioception, raw language instructions, and optional substep reasoning.

[1]

Output modalities
joint-positionsgripper-command

Continuous diffused action chunks (e.g., 10-D action vectors in example HDF5 schema; AgileX/ALOHA bimanual arm/gripper control).

[1]

Action space

Continuous diffusion-denoised action chunks generated by the ScaleDP transformer policy head (DDIMScheduler); example HDF5 schema uses 10-D action vectors with 7-D joint_positions/qpos.

[1]

Trained / supported embodiments
  • AgileX / ALOHA-style bimanual arms (evaluate/smart_eval_agilex.py, aloha_scripts/)
  • Custom multi-camera HDF5 / RLDS-converted embodiments

[1]

Training datasets
  • Multi-embodiment Stage-1 ScaleDP pretraining data (upstream details in arXiv:2502.05855)
  • Example dataset: lesjie/dexvla_example_data (Hugging Face)
  • User-supplied ACT-format HDF5 or RLDS-converted demonstrations for Stage 2 / Stage 3

[1] [5]

Interfaces
  • PyTorch / Hugging Face Transformers custom architecture (Qwen2VLForConditionalGenerationForVLA)
  • Shell training scripts (DeepSpeed / Python)
  • ACT-compatible HDF5 dataset format

[1]

Checkpoints

CheckpointFormatSizeAccessPinned revision
lesjie/scale_dp_h (Stage-1 ScaleDP-H ~1B policy head)pytorch3.8 GBopendbc18619a5c5
lesjie/scale_dp_l (Stage-1 ScaleDP-L ~400M policy head)pytorch1.83 GBopen52c2e81af0a5

Both checkpoints are ungated Stage-1 diffusion policy-head backbones (open_scale_dp_h_backbone.ckpt at 3.799 GB and open_scale_dp_l_backbone.ckpt at 1.827 GB) under CC-BY-NC-4.0; VLM backbone Qwen/Qwen2-VL-2B-Instruct is downloaded separately.

[3] [4]

Is this for you?

Use cases
  • Three-stage VLA training with a pretrained ScaleDP diffusion policy head
  • LoRA or full fine-tuning of Qwen2-VL-2B with a diffusion action expert on custom HDF5 robot demonstrations
  • Diffusion-VLA (DiVLA) UNet-head baseline training via scripts/train_divla.sh

[1]

Known limitations
  • Only Stage-1 pretrained ScaleDP policy-head checkpoints (ScaleDP-H and ScaleDP-L) are published; no merged Stage-2 or Stage-3 full DexVLA weights are released on Hugging Face.
  • Repository code and Stage-1 ScaleDP checkpoints are restricted to non-commercial use under CC-BY-NC-4.0.
  • Requires replacing Qwen2-VL-2B-Instruct's official config.json with docs/config.json and manually patching diffusers DDIMScheduler to float32 to avoid NaN actions during inference.

[1] [2] [3] [4]

Safety and limitations
  • CC-BY-NC-4.0 license on both the DexVLA repository and Stage-1 ScaleDP checkpoints prohibits commercial use.
  • No ready-to-deploy full Stage-2/Stage-3 DexVLA checkpoint is published; users must assemble Qwen2-VL-2B-Instruct with Stage-1 ScaleDP weights and train on target demonstrations.
  • Known precision overflow in diffusers DDIMScheduler can emit NaN robot actions unless self.betas is cast to float32.

[1] [2] [3] [4]

Compute & fine-tuning

Inference compute
  • Documented hardware: NVIDIA A6000 (46 GB VRAM, documented for training benchmarks)
  • Precision: bf16
  • Edge / consumer capable: No

docs/config.json specifies torch_dtype bfloat16; README documents 20-38 GB VRAM for training on an A6000, while exact minimum inference VRAM and latency_ms are not reported.

[1] · Editorial assessment

Fine-tuning
  • Supported: Yes
  • Methods: Stage-2 multi-task fine-tuning (scripts/train_dexvla_stage2.sh); Stage-3 target dexterous post-training (scripts/train_dexvla_stage3.sh); LoRA on VLM backbone with full policy-head training; DeepSpeed ZeRO-2 with optional CPU optimizer offload
  • Minimum VRAM: 20 GB
  • Recipe: Upstream fine-tuning documentation

README reports 20-29 GB CUDA memory with DeepSpeed ZeRO-2 CPU offload, 24 GB with LoRA VLM, 33 GB with ScaleDP-L, and 38 GB full on a single A6000 (batch_size 2).

[1]

Host computer

README documents single NVIDIA A6000 (46 GB VRAM) training at batch size 2 requiring 38 GB VRAM (full ScaleDP-H), 33 GB with smaller ScaleDP-L, 24 GB with LoRA VLM, or 20-29 GB with DeepSpeed ZeRO-2 CPU offload.

[1]

Operating system
  • Linux

CUDA, flash-attn, GCC > 9 DeepSpeed offload, and shell scripts target Linux.

[1] · Editorial assessment

Dependencies
  • Python 3.10
  • PyTorch
  • transformers (4.41.2 in docs/config.json)
  • diffusers
  • flash-attn
  • DeepSpeed (optional offload)
  • Qwen/Qwen2-VL-2B-Instruct

[1]

Skills
  • Python
  • PyTorch
  • DeepSpeed
  • HDF5 / RLDS data preprocessing

Inferred from documented conda, PyTorch, DeepSpeed ZeRO-2, and h5py pipeline.

[1] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [1] · Model card / documentation [1]

  • Clone juruobenruo/DexVLA, create a Python 3.10 conda environment, install requirements.txt, editable-install policy_heads, and install flash-attn.
  • Download Qwen/Qwen2-VL-2B-Instruct and replace its config.json with docs/config.json.
  • Download Stage-1 ScaleDP-H or ScaleDP-L weights from Hugging Face (lesjie/scale_dp_h or lesjie/scale_dp_l).
  • Convert robot demonstrations to ACT-style h5py files, register paths in aloha_scripts/constants.py, and run scripts/train_dexvla_stage2.sh and scripts/train_dexvla_stage3.sh.

[1]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
DexVLA paper evaluations (arXiv:2502.05855; bimanual and dexterous manipulation tasks)See arXiv:2502.05855 (no numeric task table in repository README)Upstream-reported paper evaluations; not independently verified or reproduced by OSR.

[1] [5]

Project notes

Architecture, inputs, and three-stage training recipe

In docs/config.json, DexVLA registers a custom Qwen2VLForConditionalGenerationForVLA architecture (hidden_size: 1536, 28 layers, torch_dtype: bfloat16) with policy_head_type: "scale_dp_policy". Data pipelines follow the ACT-compatible HDF5 structure (with an RLDS-to-HDF5 conversion script provided), ingesting multi-view RGB images (for example left, right, and wrist views at 480x640x3), proprioceptive state (joint_positions, qpos, qvel), raw language instructions, and optional substep reasoning strings to generate continuous diffusion-denoised action chunks. Training proceeds from Stage-1 ScaleDP pretraining into Stage-2 multi-task training (scripts/train_dexvla_stage2.sh) and Stage-3 target dexterous post-training (scripts/train_dexvla_stage3.sh); the repository also includes a script for the UNet-head Diffusion-VLA (DiVLA) predecessor. [1]

Compute footprint, caveats, and upstream-reported evaluations

The README benchmarks single-GPU training on an NVIDIA A6000 (46 GB VRAM, batch size 2): full training consumes 38 GB VRAM (1.1s/iter), switching to ScaleDP-L lowers usage to 33 GB, LoRA fine-tuning on the VLM backbone (with full policy-head training) uses 24 GB, and DeepSpeed ZeRO-2 CPU optimizer offload brings usage to 20–29 GB (6.56s/iter). The README does not establish minimum inference-only VRAM or a fixed closed-loop control frequency. Users must replace Qwen2-VL-2B-Instruct's config.json with docs/config.json, copy preprocessor_config.json and chat_template.json into trained checkpoints, and apply a documented float32 cast in diffusers's DDIMScheduler to avoid NaN actions during denoising. Benchmark evaluations are reported in arXiv:2502.05855 (no numeric task table appears in the README); Open Source Robotics has not run training, fine-tuning, inference, or robot evaluations for this profile. [1] [5]

What we checked

None. Prepared from public upstream sources without hands-on training, fine-tuning, or robot execution.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Apr 15, 2025 [1]
Latest release
Feb 28, 2025 [4]
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
Feb 19, 2027
Reviewed revision
juruobenruo/DexVLA@fc21a822f4c774e242eb6f1ab4a235788de7aba9; HF lesjie/scale_dp_h@dbc18619a5c575488c1c0f09d0ced11e01853d3c; HF lesjie/scale_dp_l@52c2e81af0a5f45bf286be0fe0fc10bfdac636c1 · combined hardware and software repository

Separate license matrix

Publishes reproducible VLA training/evaluation code and ungated Stage-1 ScaleDP diffusion policy-head checkpoints (ScaleDP-H ~1B and ScaleDP-L ~400M), but both code and weights carry a non-commercial CC-BY-NC-4.0 restriction and full Stage-2/Stage-3 merged weights are not published.

LayerAvailability / license
Codeavailable · Creative Commons Attribution-NonCommercial summary terms (version not stated in root LICENSE; non-OSI)
Root LICENSE contains the Creative Commons Attribution-NonCommercial deed summary (Share, Adapt, Attribution, NonCommercial, No additional restrictions) without a version number; GitHub API returns NOASSERTION. [1] [2]
Model weightsavailable · CC-BY-NC-4.0 (ScaleDP-H and ScaleDP-L); separate upstream terms apply to Qwen/Qwen2-VL-2B-Instruct
Stage-1 ScaleDP-H and ScaleDP-L weights are ungated on Hugging Face under cc-by-nc-4.0; full Stage-2/Stage-3 merged DexVLA checkpoints are not released. [1] [3] [4]
Training datasetsavailable · Not specified in README for example dataset; full Stage-1/Stage-2 training corpus is not bundled
An example HDF5 dataset is linked on Hugging Face for pipeline testing, while full pretraining data is not released. [1]
Documentationavailable · Creative Commons Attribution-NonCommercial summary terms (version not stated in root LICENSE)
[1] [2]

Non-commercial licensing and partial weight availability

The juruobenruo/DexVLA repository root LICENSE contains Creative Commons Attribution-NonCommercial summary terms (Share, Adapt, Attribution, NonCommercial, No additional restrictions) without stating a version identifier, while the two ungated Hugging Face Stage-1 checkpoints (lesjie/scale_dp_h at 3.80 GB and lesjie/scale_dp_l at 1.83 GB) explicitly declare cc-by-nc-4.0. The separately downloaded Qwen/Qwen2-VL-2B-Instruct VLM backbone carries its own upstream license terms that must be verified separately. Because no Stage-2 or Stage-3 merged DexVLA weights are published, practitioners need suitable demonstrations for target-task adaptation, while the linked example HDF5 dataset (lesjie/dexvla_example_data) can support pipeline testing subject to separate verification of its usage rights (the README does not specify its license) alongside the non-commercial restrictions on both code and Stage-1 weights. [1] [2] [3] [4]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model profile prepared from juruobenruo/DexVLA@fc21a822f4c7 and HF Stage-1 ScaleDP checkpoints.