Diffusion Policy
Robotics modelFoundational visuomotor policy architecture and reference implementation framing robot action generation as conditional denoising diffusion over action sequences, with CNN and Transformer backbones and pretrained benchmark checkpoints. [1] [2]
Setup & usage →CheckpointsModel hub / homepageGetting started
- For
- Robotics researchers; Imitation learning engineers; Students reproducing manipulation benchmarks [2]
- Embodiments
- Push-T (2D sim), Robomimic Franka Panda (sim), UR5 (real-robot tasks in paper), … [2]
- Source rights
- Some licenses are unresolved. Review licenses
Source researched · Not tested by us
Sources checked · Research details
Licenses & related hardware
- License
- MIT · license not fully assessed for model weights, datasets · Details by part
Overview
Diffusion Policy is a visuomotor policy architecture and reference PyTorch implementation that frames robot action generation as conditional denoising diffusion over receding-horizon action sequences. The repository provides both 1D temporal U-Net (CNN) and time-series Transformer noise predictors conditioned on visual observations and proprioceptive state, alongside Hydra task configs and a public Columbia web archive of per-task experiment checkpoints. [1] [2] [4]
Model card summary
- Architecture
Conditional denoising diffusion policy with 1D temporal U-Net (CNN) or time-series Transformer noise predictor conditioned on ResNet visual encoder + proprioceptive state history (DDPM/DDIM).
Exact architecture depends on selected Hydra workspace (e.g., TrainDiffusionUnetHybridWorkspace for image Push-T).
- Parameter count
Varies by task config (CNN U-Net vs. Transformer, low-dim vs. image hybrid; no single scalar published)
Varies by task config (CNN U-Net vs. Transformer, low-dim vs. image hybrid); no single canonical parameter count is published across all task checkpoints.
- Input modalities
- rgbproprioception
Supports both low-dim state-only and multi-camera RGB + proprioception hybrid observation histories (dataset_obs_steps=2 in image Push-T config).
- Output modalities
- ee-pose-absee-pose-deltajoint-positionsgripper-command
Outputs continuous action chunks (e.g., 2D position in Push-T, 6-DoF end-effector position/velocity or joint actions + gripper depending on task config).
- Action space
Task-dependent continuous receding-horizon action chunks predicted via iterative denoising (e.g., 2D planar EE target for Push-T; 7-DoF / bimanual continuous EE pose + gripper for Robomimic and real-robot tasks).
- Trained / supported embodiments
- Push-T (2D sim)
- Robomimic Franka Panda (sim)
- UR5 (real-robot tasks in paper)
- Franka Panda (real-robot tasks in paper)
Not a cross-embodiment model; each checkpoint is trained for a specific robot/task environment.
- Training datasets
- Push-T demonstrations
- Robomimic (Lift, Can, Square, Transport, Tool Hang)
- Block Pushing / Franka Kitchen benchmarks
Experiment replay buffers hosted under https://diffusion-policy.cs.columbia.edu/data/.
Checkpoints
| Checkpoint | Format | Size | Access | Pinned revision |
|---|---|---|---|---|
| Per-task experiment checkpoints (image & low_dim, CNN & Transformer) | pytorch | Not stated | open | unversioned-http-archive |
Individual .ckpt files (best epoch and latest.ckpt across 3 seeds per task) are hosted on Columbia web directory tree rather than Hugging Face Hub; total archive size not established.
Is this for you?
- Use cases
- Single-task visuomotor imitation learning
- Multimodal manipulation baseline benchmarking
- Real-robot and simulation policy training from teleoperated demos
- Known limitations
- Task-specific training per benchmark/environment rather than zero-shot language-conditioned generalization
- No unified Hugging Face Hub model card; checkpoints are hosted as per-experiment directory trees on Columbia web servers
- Safety and limitations
- Inspected releases are task- and environment-specific; supplied evidence does not establish zero-shot transfer across unseen embodiments or language instructions.
- Iterative denoising (e.g., num_inference_steps: 100, horizon: 16, n_action_steps: 8, n_obs_steps: 2 in the inspected Push-T CNN config) must be reconciled with target hardware control-loop timing.
- Explicit license terms for hosted checkpoints and replay buffers on diffusion-policy.cs.columbia.edu/data/ were not established in this review and require upstream confirmation.
Synthesized from README architecture and missing standalone license declaration on hosted data directory.
Compute & fine-tuning
- Inference compute
- Precision:
fp32 - Edge / consumer capable: Yes
Exact minimum VRAM and latency_ms are not specified in README; small per-task CNN/Transformer policies run on consumer GPUs and Colab notebooks.
[2] · Editorial assessment
- Precision:
- Fine-tuning
- Supported: Yes
- Methods: from-scratch task training via Hydra configs
- Recipe: Upstream fine-tuning documentation
Designed for per-task training from demonstrations rather than foundation-model fine-tuning.
- Host computer
CUDA-capable GPU recommended for training and vision-based policy inference; Colab notebooks provided for state and vision tasks.
- Operating system
- Linux
Standard Linux conda/CUDA setup documented in README; other OS support not explicitly documented.
[2] · Editorial assessment
Setup & usage
Open the upstream getting-started guide [2] · Model card / documentation [2]
- Clone real-stanford/diffusion_policy
- Create conda environment from conda_environment.yaml
- Download task dataset or experiment checkpoint from diffusion-policy.cs.columbia.edu/data/
- Run Hydra train.py or eval.py
Upstream-reported benchmarks
Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.
| Benchmark | Upstream-reported score | Scope & caveats |
|---|---|---|
| Robomimic, Push-T, Block Pushing, Franka Kitchen (Table I/II/IV) | See per-experiment logs.json.txt in the upstream archive (README illustrates checkpoint filenames including 1.000 without tying that tree to a specific task) | Upstream-reported simulation & real-robot benchmark results across 3 seeds per task; not independently verified by OSR. |
Project notes
Task-specific checkpoints rather than one generalist model
Unlike multi-embodiment VLA base models, Diffusion Policy releases separate task- and environment-specific checkpoints across simulation and paper benchmarks (such as Push-T, Robomimic, Block Pushing, and Franka Kitchen) rather than a single universal weight file. Parameter count, observation history, and action dimensions depend on the selected Hydra workspace—for example, TrainDiffusionUnetHybridWorkspace in the inspected Push-T image CNN config sets n_obs_steps: 2 (dataset_obs_steps: 2) and outputs continuous 2D targets, whereas Robomimic and real-robot tasks use higher-dimensional end-effector or joint actions. The inspected releases are task-specific; the supplied evidence does not establish zero-shot transfer to unseen embodiments or language-conditioned operation. [2] [4]
Closed-loop inference and benchmark scope
Iterative denoising (DDPM/DDIM) couples denoising step count to closed-loop control timing. In the inspected Push-T image CNN config (config.yaml), upstream sets num_inference_steps: 100, horizon: 16, n_action_steps: 8, and n_obs_steps: 2; these are configuration settings rather than measured timing results, and the selected checkpoint's observation history, prediction horizon, and executed action chunk must be reconciled with the target control loop before physical execution. Upstream hosts per-experiment logs.json.txt and .ckpt files across three seeds, and the README's illustrative directory tree shows checkpoint filenames including 1.000 without tying that tree to a specific task. All benchmark claims are upstream-reported; Open Source Robotics has not run training, inference, or hardware control for this profile and reports no measured VRAM or latency numbers. [2] [4]
What we checked
None. Prepared from public upstream sources without hands-on hardware execution.
Not yet established
- Documented closed-loop control frequency (Hz)
Updates
- Upstream last updated
- Dec 24, 2024 [1]
- Latest release
- Not established
- Profile updated
- Nov 21, 2026
- Sources last checked
- Nov 21, 2026
- Next review
- May 21, 2027
- Reviewed revision
- real-stanford/diffusion_policy@5ba07ac6661db573af695b419a7947ecb704690f · software repository
Separate license matrix
Foundational visuomotor policy implementation with MIT-licensed source code and publicly downloadable checkpoints/configs; marked include_partial pending explicit license confirmation for hosted checkpoint weights and bundled benchmark datasets.
| Layer | Availability / license |
|---|---|
| Code | available · MIT MIT License (Copyright 2023 Columbia Artificial Intelligence and Robotics Lab) verified in root LICENSE. [1] [3] |
| Model weights | available · License not fully assessed Pretrained .ckpt weights are publicly downloadable from the Columbia web archive, but this review did not establish an explicit separate license for the hosted checkpoints; confirm terms upstream before assuming MIT applies. [2] [4] |
| Training datasets | available · License not fully assessed Task replay buffers (Push-T, Robomimic, etc.) are publicly downloadable, but this review did not establish a unified data license and upstream third-party benchmark datasets carry their own terms. [2] |
| Documentation | available · MIT README and Hydra configs in repo are covered by root MIT LICENSE; academic paper rights remain separate. [2] [3] |
Separate code, checkpoint, and dataset rights
The GitHub repository carries a verified root MIT license covering the code, README, and Hydra configurations. Pretrained .ckpt weights and task replay buffers are publicly downloadable from diffusion-policy.cs.columbia.edu/data/, but this review did not establish explicit licensing for the hosted checkpoints and replay buffers, and upstream third-party benchmark datasets carry their own terms. Confirm the selected artifacts' terms upstream rather than assuming MIT covers hosted weights automatically. [1] [2] [3]
References
Each source links to the exact revision we read. “Used for” shows where this page relies on it.
- S001 real-stanford/diffusion_policy GitHub repository metadata and HEAD commit
Default branch main HEAD is 5ba07ac6661db573af695b419a7947ecb704690f (2024-12-24T19:47:59Z); SPDX license MIT; homepage https://diffusion-policy.cs.columbia.edu/.Revision, date & where cited
real-stanford (GitHub) · github.com · revision
5ba07ac· GitHub API repos/real-stanford/diffusion_policy & commits/main · Accessed Nov 21, 2026
Used for: Overview, What we checked, Separate license matrix - S002 real-stanford/diffusion_policy README.md at pinned commit
Documents Colab notebooks, experiment archive structure (config.yaml, logs.json.txt, best/latest .ckpt across 3 seeds) at diffusion-policy.cs.columbia.edu/data/experiments/.Revision, date & where cited
real-stanford (GitHub) · github.com · revision
5ba07ac· README.md (pinned commit 5ba07ac6661d) · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 6 more - S003 real-stanford/diffusion_policy LICENSE at pinned commit
MIT License, Copyright (c) 2023 Columbia Artificial Intelligence and Robotics Lab.Revision, date & where cited
Columbia Artificial Intelligence and Robotics Lab · github.com · revision
5ba07ac· LICENSE (pinned commit 5ba07ac6661d) · Accessed Nov 21, 2026
Used for: Is this for you?, Separate license matrix - S004 Diffusion Policy Push-T Image CNN experiment config.yaml
Confirms hosted experiment directory is live and specifies TrainDiffusionUnetHybridWorkspace, dataset_obs_steps: 2, and .ckpt checkpointing.Revision, date & where cited
Columbia University · diffusion-policy.cs.columbia.edu · HTTP 200 config.yaml (3743 bytes) · Accessed Nov 21, 2026
Used for: Overview, Model card summary, Checkpoints, Is this for you? and 4 more
Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1
Change history
2026-11-21 — Initial 0.8.0-draft.1 model draft prepared from pinned GitHub commit 5ba07ac6661d and Columbia experiment archive.