SpatialVLA
Robotics model4B-parameter spatial-enhanced Vision-Language-Action model built on PaLiGemma2-3B with Ego3D Position Encoding (via ZoeDepth) and Adaptive Action Grids, pretrained on 1.1M real robot episodes and loadable directly via Hugging Face Transformers. [1] [2] [3]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Robotics researchers; VLA fine-tuning engineers; 3D-aware manipulation developers [2]
- Parameters
- 4,027,854,731 (~4.03B BF16 parameters per safetensors metadata) [2]
- Embodiments
- WidowX 250 (Bridge/SimplerEnv), Google Robot (SimplerEnv), Franka Panda (LIBERO / Open X-Embodiment), … [2] [4]
- Source rights
- Named licenses are documented by part. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- MIT (declared on HF model card; no root LICENSE file in GitHub repo), MIT (HF model card declaration; separate upstream terms of base model google/paligemma2-3b-pt-224 require independent verification), Mixed upstream (Open X-Embodiment + RH20T; individual dataset terms not excerpted in this review), MIT (card declaration) · Details by part
Overview
SpatialVLA (IPEC-COMMUNITY/spatialvla-4b-224-pt) is a 4,027,854,731-parameter (~4.03B) Vision-Language-Action model built on google/paligemma2-3b-pt-224 (SigLIP vision encoder and Gemma2-2B causal LM) augmented with a ZoeDepth monocular depth module (use_vision_zoe: true), Ego3D Position Encoding, and Adaptive Action Grids. It is pretrained on 1.1M real robot episodes from Open X-Embodiment and RH20T and loads through Hugging Face Transformers. [2] [3] [4]
Model card summary
- Architecture
PaLiGemma2-3B (SigLIP vision encoder + Gemma2-2B causal LM) augmented with ZoeDepth monocular depth module (use_vision_zoe: true), Ego3D Position Encoding, and Adaptive Action Grid discrete spatial action tokens (spatial_token_num: 8194).
- Base model
google/paligemma2-3b-pt-224
- Parameter count
4,027,854,731 (~4.03B BF16 parameters per safetensors metadata)
Verified via Hugging Face API safetensors metadata.
- Input modalities
- rgblanguage
Single RGB image (224x224; depth estimated internally via ZoeDepth) + English language prompt. Multi-view and state inputs are noted as under development on the lerobot branch.
- Output modalities
- ee-pose-deltagripper-command
Discrete Adaptive Action Grid spatial action tokens decoded via dataset unnorm_key into continuous end-effector deltas and gripper commands.
- Action space
Discrete Adaptive Action Grid tokens (spatial_token_num: 8194 starting at vocab index 257153) decoded to continuous 7-DoF end-effector delta + gripper actions using per-dataset unnormalization keys (e.g., bridge_orig/1.0.0).
- Trained / supported embodiments
- WidowX 250 (Bridge/SimplerEnv)
- Google Robot (SimplerEnv)
- Franka Panda (LIBERO / Open X-Embodiment)
- RH20T multi-robot embodiments
- Training datasets
- open-x-embodiment
- rh20t
Pretrained on 1.1M real robot episodes from Open X-Embodiment and RH20T.
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| IPEC-COMMUNITY/spatialvla-4b-224-pt | safetensors | 7.54 GB | open | 886c4557dfde |
Ungated BF16 safetensors checkpoint (7.54 GB) verified via Hugging Face API.
Is this for you?
- Use cases
- Zero-shot evaluation on pretraining embodiments (e.g., Bridge/SimplerEnv)
- Fine-tuning on custom robot manipulation datasets
- Research on 3D spatial tokenization for VLAs
- Known limitations
- Does not zero-shot generalize to unseen robot embodiments not present in the pretraining mixture
- Requires trust_remote_code=True in Hugging Face Transformers
- GitHub repository root lacks a standalone LICENSE file (MIT is declared on the Hugging Face model card, while underlying PaLiGemma2-3B carries Gemma Terms of Use)
- Safety and limitations
- Does not zero-shot generalize to unseen robot embodiments; requires fine-tuning on target hardware demonstrations.
- Loading weights in Hugging Face Transformers requires trust_remote_code=True (config.json defines custom auto_map implementations); inspect repository code at the pinned revision before enabling.
- GitHub repository lacks a root LICENSE file; separate upstream terms for google/paligemma2-3b-pt-224 and training datasets (Open X-Embodiment, RH20T) require independent verification.
Compute & fine-tuning
- Inference compute
- Minimum VRAM: 8.5 GB
- Precision:
bf16 - Edge / consumer capable: No
Model card explicitly reports 8.5 GB GPU memory requirement for BF16 inference; latency_ms not stated in model card.
- Fine-tuning
- Supported: Yes
- Methods: full fine-tune / LoRA via SpatialVLA repository
- Recipe: Upstream fine-tuning documentation
- Host computer
Requires 8.5 GB of GPU memory for BF16 inference per model card and README.
- Operating system
- Linux
README documents Linux apt/tcmalloc tip and CUDA inference.
[4] · Editorial assessment
- Dependencies
- Python
- PyTorch
- transformers >= 4.47.0
- PIL
- Skills
- Python
- PyTorch
- Hugging Face Transformers
Inferred from documented Python/PyTorch/CLI setup and training dependencies.
[2] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [2] · Model card / documentation [1] [2]
- Install transformers >= 4.47.0 and PyTorch
- Load IPEC-COMMUNITY/spatialvla-4b-224-pt with AutoModel/AutoProcessor (trust_remote_code=True, bfloat16)
- Call model.predict_action(inputs) and processor.decode_actions(outputs, unnorm_key=...)
- Clone SpatialVLA/SpatialVLA for pretraining or fine-tuning
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| LIBERO (Spatial, Object, Goal, Long — fine-tuned) | 88.2% Spatial, 89.9% Object, 78.6% Goal, 55.5% Long (78.1% avg) | Upstream-reported simulation scores in SpatialVLA README; not independently verified by OSR. |
Project notes
Spatial tokenization and runtime requirements
At pinned Hub revision 886c4557dfde, config.json defines SpatialVLAForConditionalGeneration with spatial_token_num: 8194 starting at vocabulary index 257153, decoding discrete spatial tokens into continuous 7-DoF end-effector delta and gripper commands using per-dataset unnormalization keys (such as bridge_orig/1.0.0). The model card documents a PyTorch/CUDA BF16 inference path requiring transformers >= 4.47.0 and 8.5 GB of GPU memory (separate from the repository's Python >= 3.10 training environment prerequisites). Because config.json registers custom configuration and model implementations via auto_map, loading requires trust_remote_code=True, which executes repository-provided Python code; as editorial guidance, inspect that code at the pinned revision before enabling the flag. Unseen robot embodiments require fine-tuning on target demonstrations rather than zero-shot execution. [2] [3] [4]
Upstream-reported simulation evaluations
The pinned GitHub README reports fine-tuned LIBERO simulation success rates of 88.2% Spatial, 89.9% Object, 78.6% Goal, and 55.5% Long (78.1% average), alongside links to SimplerEnv evaluations. All reported metrics are upstream-reported simulation figures; Open Source Robotics has not run training, fine-tuning, inference, or robot evaluations for SpatialVLA and reports no independently measured latency or task accuracy. [4]
What we checked
None. Prepared from public upstream sources without hands-on hardware execution.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Jun 23, 2025 [1]
- Latest release
- Mar 16, 2025 [2]
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- May 21, 2027
- Reviewed revision
- SpatialVLA/SpatialVLA@18fd74b2a633ec8d9ec7aadcd803969555cc9fbd; HF IPEC-COMMUNITY/spatialvla-4b-224-pt@886c4557dfde53b44c0f74fde7d89bf83a26c3cd · software repository
Separate license matrix
Inspectable 4B-parameter 3D-spatial VLA with ungated safetensors weights and Hugging Face Transformers integration; marked include_partial because GitHub repo lacks a root LICENSE file and weights inherit PaLiGemma2/Gemma Terms.
| Layer | Availability / license |
|---|---|
| Code | available · MIT (declared on HF model card; no root LICENSE file in GitHub repo) Verified via GitHub API that SpatialVLA/SpatialVLA has no root LICENSE file (license_spdx: null); HF model card states 'All SpatialVLA checkpoints, as well as our training codebase are released under an MIT License.' [1] [2] |
| Model weights | available · MIT (HF model card declaration; separate upstream terms of base model google/paligemma2-3b-pt-224 require independent verification) HF model card declares license: mit; base_model is google/paligemma2-3b-pt-224, whose upstream terms were not excerpted in this review and require verification. [2] |
| Training datasets | available · Mixed upstream (Open X-Embodiment + RH20T; individual dataset terms not excerpted in this review) [2] |
| Documentation | available · MIT (card declaration) [2] |
Separate code, weights, base-model, and dataset licensing
The Hugging Face model card declares MIT licensing for both checkpoints and training code, but the SpatialVLA/SpatialVLA GitHub repository root lacks a LICENSE file (license_spdx: null via GitHub API). In addition, weights derive from google/paligemma2-3b-pt-224 and pretraining draws on Open X-Embodiment and RH20T; those upstream base-model and dataset license terms were not excerpted in this review and require independent verification (without assuming that dataset terms automatically transfer to model weights). [1] [2]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 SpatialVLA/SpatialVLA GitHub repository metadata and HEAD commit
Default branch main HEAD is 18fd74b2a633ec8d9ec7aadcd803969555cc9fbd (2025-06-23T08:14:12Z); license_spdx is null and no LICENSE file exists in repository root.Revision, date & where cited
SpatialVLA (GitHub) · github.com · revision
18fd74b· GitHub API repos/SpatialVLA/SpatialVLA & contents?ref=18fd74b2a633 · Accessed Nov 21, 2026
Used for: Is this for you?, Setup & usage, What we checked, Separate license matrix - S002 Hugging Face model card and API metadata: IPEC-COMMUNITY/spatialvla-4b-224-pt
4,027,854,731 BF16 parameters, 7.54 GB safetensors, ungated, license: mit, base_model: google/paligemma2-3b-pt-224, requires 8.5 GB GPU memory.Revision, date & where cited
IPEC-COMMUNITY (Hugging Face) · huggingface.co · HF API & README.md at revision 886c4557dfde · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 5 more - S003 IPEC-COMMUNITY/spatialvla-4b-224-pt config.json at pinned revision
Confirms SpatialVLAForConditionalGeneration, Gemma2ForCausalLM + SigLIP + ZoeDepth (use_vision_zoe: true), spatial_token_num: 8194, action_token_begin_idx: 257153.Revision, date & where cited
IPEC-COMMUNITY (Hugging Face) · huggingface.co · config.json at revision 886c4557dfde · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Spatial tokenization and runtime requirements - S004 SpatialVLA/SpatialVLA README.md at pinned commit
Documents RSS 2025 acceptance, LIBERO fine-tuned table (78.1% avg), SimplerEnv eval link, and in-development lerobot branch.Revision, date & where cited
SpatialVLA (GitHub) · github.com · revision
18fd74b· README.md at commit 18fd74b2a633 · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Compute & fine-tuning, Upstream-reported benchmarks and 2 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model draft prepared from pinned GitHub commit 18fd74b2a633 and HF revision 886c4557dfde.