Open Source RoboticsRobots, software & learning

Diffusion Policy

Robotics model

Foundational visuomotor policy architecture and reference implementation framing robot action generation as conditional denoising diffusion over action sequences, with CNN and Transformer backbones and pretrained benchmark checkpoints. [1] [2]

Setup & usage →CheckpointsModel hub / homepageGetting started

For
Robotics researchers; Imitation learning engineers; Students reproducing manipulation benchmarks [2]
Parameters
Varies by task config (CNN U-Net vs [2] [4]
Embodiments
Push-T (2D sim), Robomimic Franka Panda (sim), UR5 (real-robot tasks in paper), … [2]
Source rights
Some licenses are unresolved. Review licenses

Source researched · Not tested by us
Sources checked · Research details

Licenses & related hardware
License
MIT · license not fully assessed for model weights, datasets · Details by part

Requirements · Research details and dates

On this page

Overview

Diffusion Policy is a visuomotor policy architecture and reference PyTorch implementation that frames robot action generation as conditional denoising diffusion over receding-horizon action sequences. The repository provides both 1D temporal U-Net (CNN) and time-series Transformer noise predictors conditioned on visual observations and proprioceptive state, alongside Hydra task configs and a public Columbia web archive of per-task experiment checkpoints. [1] [2] [4]

Model card summary

Architecture

Conditional denoising diffusion policy with 1D temporal U-Net (CNN) or time-series Transformer noise predictor conditioned on ResNet visual encoder + proprioceptive state history (DDPM/DDIM).

Exact architecture depends on selected Hydra workspace (e.g., TrainDiffusionUnetHybridWorkspace for image Push-T).

[2] [4]

Parameter count

Varies by task config (CNN U-Net vs. Transformer, low-dim vs. image hybrid; no single scalar published)

Varies by task config (CNN U-Net vs. Transformer, low-dim vs. image hybrid); no single canonical parameter count is published across all task checkpoints.

[2] [4]

Input modalities
rgbproprioception

Supports both low-dim state-only and multi-camera RGB + proprioception hybrid observation histories (dataset_obs_steps=2 in image Push-T config).

[2] [4]

Output modalities
ee-pose-absee-pose-deltajoint-positionsgripper-command

Outputs continuous action chunks (e.g., 2D position in Push-T, 6-DoF end-effector position/velocity or joint actions + gripper depending on task config).

[2]

Action space

Task-dependent continuous receding-horizon action chunks predicted via iterative denoising (e.g., 2D planar EE target for Push-T; 7-DoF / bimanual continuous EE pose + gripper for Robomimic and real-robot tasks).

[2] [4]

Trained / supported embodiments
  • Push-T (2D sim)
  • Robomimic Franka Panda (sim)
  • UR5 (real-robot tasks in paper)
  • Franka Panda (real-robot tasks in paper)

Not a cross-embodiment model; each checkpoint is trained for a specific robot/task environment.

[2]

Training datasets
  • Push-T demonstrations
  • Robomimic (Lift, Can, Square, Transport, Tool Hang)
  • Block Pushing / Franka Kitchen benchmarks

Experiment replay buffers hosted under https://diffusion-policy.cs.columbia.edu/data/.

[2] [4]

Interfaces
  • Python API
  • Hydra CLI
  • PyTorch checkpoint (.ckpt)

[2] [4]

Checkpoints

CheckpointFormatSizeAccessPinned revision
Per-task experiment checkpoints (image & low_dim, CNN & Transformer)pytorchNot statedopenunversioned-http-archive

Individual .ckpt files (best epoch and latest.ckpt across 3 seeds per task) are hosted on Columbia web directory tree rather than Hugging Face Hub; total archive size not established.

[2] [4]

Is this for you?

Use cases
  • Single-task visuomotor imitation learning
  • Multimodal manipulation baseline benchmarking
  • Real-robot and simulation policy training from teleoperated demos

[2]

Known limitations
  • Task-specific training per benchmark/environment rather than zero-shot language-conditioned generalization
  • No unified Hugging Face Hub model card; checkpoints are hosted as per-experiment directory trees on Columbia web servers

[2] [4]

Safety and limitations
  • Inspected releases are task- and environment-specific; supplied evidence does not establish zero-shot transfer across unseen embodiments or language instructions.
  • Iterative denoising (e.g., num_inference_steps: 100, horizon: 16, n_action_steps: 8, n_obs_steps: 2 in the inspected Push-T CNN config) must be reconciled with target hardware control-loop timing.
  • Explicit license terms for hosted checkpoints and replay buffers on diffusion-policy.cs.columbia.edu/data/ were not established in this review and require upstream confirmation.

Synthesized from README architecture and missing standalone license declaration on hosted data directory.

[2] [3] [4] · Editorial assessment

Compute & fine-tuning

Inference compute
  • Precision: fp32
  • Edge / consumer capable: Yes

Exact minimum VRAM and latency_ms are not specified in README; small per-task CNN/Transformer policies run on consumer GPUs and Colab notebooks.

[2] · Editorial assessment

Fine-tuning

Designed for per-task training from demonstrations rather than foundation-model fine-tuning.

[2]

Host computer

CUDA-capable GPU recommended for training and vision-based policy inference; Colab notebooks provided for state and vision tasks.

[2]

Operating system
  • Linux

Standard Linux conda/CUDA setup documented in README; other OS support not explicitly documented.

[2] · Editorial assessment

Dependencies
  • Python
  • PyTorch
  • Hydra
  • MuJoCo / Robomimic (for simulation benchmarks)

[2] [4]

Skills
  • Python
  • PyTorch
  • Hydra configuration
  • Conda environment management

Inferred from repository README and Hydra YAML experiment configs.

[2] [4] · Editorial assessment

Setup & usage

Open the upstream getting-started guide [2] · Model card / documentation [2]

  • Clone real-stanford/diffusion_policy
  • Create conda environment from conda_environment.yaml
  • Download task dataset or experiment checkpoint from diffusion-policy.cs.columbia.edu/data/
  • Run Hydra train.py or eval.py

[2]

Upstream-reported benchmarks

Upstream-reported only: Open Source Robotics has not run training, inference, or hardware evaluations for these benchmarks.

BenchmarkUpstream-reported scoreScope & caveats
Robomimic, Push-T, Block Pushing, Franka Kitchen (Table I/II/IV)See per-experiment logs.json.txt in the upstream archive (README illustrates checkpoint filenames including 1.000 without tying that tree to a specific task)Upstream-reported simulation & real-robot benchmark results across 3 seeds per task; not independently verified by OSR.

[2]

Project notes

Task-specific checkpoints rather than one generalist model

Unlike multi-embodiment VLA base models, Diffusion Policy releases separate task- and environment-specific checkpoints across simulation and paper benchmarks (such as Push-T, Robomimic, Block Pushing, and Franka Kitchen) rather than a single universal weight file. Parameter count, observation history, and action dimensions depend on the selected Hydra workspace—for example, TrainDiffusionUnetHybridWorkspace in the inspected Push-T image CNN config sets n_obs_steps: 2 (dataset_obs_steps: 2) and outputs continuous 2D targets, whereas Robomimic and real-robot tasks use higher-dimensional end-effector or joint actions. The inspected releases are task-specific; the supplied evidence does not establish zero-shot transfer to unseen embodiments or language-conditioned operation. [2] [4]

Closed-loop inference and benchmark scope

Iterative denoising (DDPM/DDIM) couples denoising step count to closed-loop control timing. In the inspected Push-T image CNN config (config.yaml), upstream sets num_inference_steps: 100, horizon: 16, n_action_steps: 8, and n_obs_steps: 2; these are configuration settings rather than measured timing results, and the selected checkpoint's observation history, prediction horizon, and executed action chunk must be reconciled with the target control loop before physical execution. Upstream hosts per-experiment logs.json.txt and .ckpt files across three seeds, and the README's illustrative directory tree shows checkpoint filenames including 1.000 without tying that tree to a specific task. All benchmark claims are upstream-reported; Open Source Robotics has not run training, inference, or hardware control for this profile and reports no measured VRAM or latency numbers. [2] [4]

What we checked

None. Prepared from public upstream sources without hands-on hardware execution.

Not yet established

  • Documented closed-loop control frequency (Hz)

Updates

Upstream last updated
Dec 24, 2024 [1]
Latest release
Not established
Profile updated
Nov 21, 2026
Sources last checked
Nov 21, 2026
Next review
May 21, 2027
Reviewed revision
real-stanford/diffusion_policy@5ba07ac6661db573af695b419a7947ecb704690f · software repository

Separate license matrix

Foundational visuomotor policy implementation with MIT-licensed source code and publicly downloadable checkpoints/configs; marked include_partial pending explicit license confirmation for hosted checkpoint weights and bundled benchmark datasets.

LayerAvailability / license
Codeavailable · MIT
MIT License (Copyright 2023 Columbia Artificial Intelligence and Robotics Lab) verified in root LICENSE. [1] [3]
Model weightsavailable · License not fully assessed
Pretrained .ckpt weights are publicly downloadable from the Columbia web archive, but this review did not establish an explicit separate license for the hosted checkpoints; confirm terms upstream before assuming MIT applies. [2] [4]
Training datasetsavailable · License not fully assessed
Task replay buffers (Push-T, Robomimic, etc.) are publicly downloadable, but this review did not establish a unified data license and upstream third-party benchmark datasets carry their own terms. [2]
Documentationavailable · MIT
README and Hydra configs in repo are covered by root MIT LICENSE; academic paper rights remain separate. [2] [3]

Separate code, checkpoint, and dataset rights

The GitHub repository carries a verified root MIT license covering the code, README, and Hydra configurations. Pretrained .ckpt weights and task replay buffers are publicly downloadable from diffusion-policy.cs.columbia.edu/data/, but this review did not establish explicit licensing for the hosted checkpoints and replay buffers, and upstream third-party benchmark datasets carry their own terms. Confirm the selected artifacts' terms upstream rather than assuming MIT covers hosted weights automatically. [1] [2] [3]

References

Each source links to the exact revision we read. “Used for” shows where this page relies on it.

Correction guidance · Discussion is not enabled · Profile format 0.8.0-draft.1

Change history

2026-11-21 — Initial 0.8.0-draft.1 model draft prepared from pinned GitHub commit 5ba07ac6661d and Columbia experiment archive.