JUMP-HOWL and FlexiCup contributors Asset: /assets/projects/flexicup/upstream-image.png Source: https://github.com/JUMP-HOWL/FlexiCup/blob/0a23196e7b3afa5f6bdf7738558609f956016b69/static/images/image/FigSuctionPhotos.png Original image SHA-256: 1a6d77afb9b1d2ac9fa7cc492965594d06eaec6d8f7b1aab2d469407f525379b Pinned README/repository revision: 0a23196e7b3afa5f6bdf7738558609f956016b69 Reuse basis: CC-BY-SA-4.0 Upstream rights statement: https://github.com/JUMP-HOWL/FlexiCup/blob/0a23196e7b3afa5f6bdf7738558609f956016b69/README.md Changes: Original image unchanged; displayed responsively. Media review date: 2026-10-25 No endorsement or hardware testing is claimed. UPSTREAM RIGHTS STATEMENT β€” README.md # FlexiCup: Wireless Multimodal Suction Cup with Dual-Zone Vision-Tactile Sensing This repository contains the complete project files for FlexiCupβ€”a multimodal suction cup with wireless electronics that integrates dual-zone vision-tactile sensing. The modular mechanical design supports both vacuum and Bernoulli actuation modes while maintaining the identical sensing architecture, demonstrating sensing-actuation decoupling. Included are hardware designs (CAD/STEP, PCB schematics/layout/BOM/Gerber), ESP32-S3 firmware, the full learning pipeline, and documentation. ## πŸ“ Repository Structure ``` FlexiCup/ β”œβ”€β”€ Hardware/ # Hardware design and firmware β”‚ β”œβ”€β”€ Fabrication/ # Physical hardware design files β”‚ β”‚ β”œβ”€β”€ CAD/ # 3D CAD models and mechanical designs β”‚ β”‚ β”‚ β”œβ”€β”€ FlexiCup_Vacuum.STEP # Vacuum mode assembly (109MB) β”‚ β”‚ β”‚ β”œβ”€β”€ FlexiCup_Bernoulli.STEP # Bernoulli mode assembly (109MB) β”‚ β”‚ β”‚ β”œβ”€β”€ suction cup bottom I.STEP # Bottom configuration I β”‚ β”‚ β”‚ β”œβ”€β”€ suction cup bottom II.STEP # Bottom configuration II β”‚ β”‚ β”‚ β”œβ”€β”€ suction cup bottom III.STEP # Bottom configuration III β”‚ β”‚ β”‚ β”œβ”€β”€ suction cup bottom IV.STEP # Bottom configuration IV β”‚ β”‚ β”‚ β”œβ”€β”€ suction cup top.STEP # Top assembly β”‚ β”‚ β”‚ └── FlexiCup_CAD_BOM.xlsx # CAD Bill of Materials β”‚ β”‚ β”‚ β”‚ β”‚ └── PCB/ # PCB design files β”‚ β”‚ β”œβ”€β”€ FlexiCup_Schematic.pdf # Circuit schematic β”‚ β”‚ β”œβ”€β”€ FlexiCup_PCB.pdf # PCB layout β”‚ β”‚ β”œβ”€β”€ FlexiCup_PCB_BOM.xlsx # PCB Bill of Materials β”‚ β”‚ └── Gerber_PCB.zip # Manufacturing files β”‚ β”‚ β”‚ └── Firmware/ # Embedded firmware code β”‚ └── ESPCAM/ # ESP32S3 camera firmware β”‚ β”œβ”€β”€ main/ # Main application code β”‚ β”‚ β”œβ”€β”€ inc/ # Header files β”‚ β”‚ β”‚ β”œβ”€β”€ camera.h β”‚ β”‚ β”‚ β”œβ”€β”€ httpServer.h β”‚ β”‚ β”‚ β”œβ”€β”€ led.h β”‚ β”‚ β”‚ └── wifiConnect.h β”‚ β”‚ β”œβ”€β”€ src/ # Source files β”‚ β”‚ β”‚ β”œβ”€β”€ camera.c β”‚ β”‚ β”‚ β”œβ”€β”€ httpServer.c β”‚ β”‚ β”‚ β”œβ”€β”€ led.c β”‚ β”‚ β”‚ └── wifiConnect.c β”‚ β”‚ β”œβ”€β”€ html/ # Web interface β”‚ β”‚ └── main.c # Main entry point β”‚ β”‚ β”‚ β”œβ”€β”€ managed_components/ # ESP32-Camera library β”‚ β”œβ”€β”€ CMakeLists.txt β”‚ β”œβ”€β”€ sdkconfig β”‚ └── README.md β”‚ β”œβ”€β”€ Software/ # High-level software and algorithms β”‚ └── diffusion_policies/ # Diffusion policy implementation β”‚ β”œβ”€β”€ data_collect.py # Data collection pipeline β”‚ β”œβ”€β”€ train.py # Training script β”‚ β”œβ”€β”€ deploy.py # Deployment script β”‚ β”œβ”€β”€ environment.yml # Conda environment β”‚ └── README.md # Software documentation β”‚ β”œβ”€β”€ PDF/ # Documentation β”‚ └── paper.pdf # Research paper β”‚ β”œβ”€β”€ static/ # Website assets β”‚ β”œβ”€β”€ css/ # Stylesheets β”‚ β”œβ”€β”€ js/ # JavaScript files β”‚ β”œβ”€β”€ images/ # Images and figures β”‚ └── video/ # Video demonstrations β”‚ β”œβ”€β”€ optimized/ # Optimized videos for web (12.2MB total) β”‚ └── websitevideo/# Original high-quality videos β”‚ └── index.html # Project website ``` ## πŸ”§ Hardware Files ### Fabrication Design Files All mechanical and electronic designs are provided for complete system reproduction: **CAD Models (STEP Format)** All mechanical designs are provided in STEP format for maximum compatibility: - **Main Assemblies**: - `FlexiCup_Vacuum.STEP` (109MB): Complete vacuum suction mode assembly - `FlexiCup_Bernoulli.STEP` (109MB): Complete Bernoulli suction mode assembly - **Modular Components**: - `suction cup top.STEP`: Top housing with camera and electronics - `suction cup bottom I-IV.STEP`: Four interchangeable bottom configurations - I & II: For vacuum mode operation - III & IV: For Bernoulli mode operation - **Bill of Materials**: - `FlexiCup_CAD_BOM.xlsx`: Complete list of mechanical components **PCB Design Files** Complete electronics design for the FlexiCup controller: - **Schematic**: `FlexiCup_Schematic.pdf` - Circuit diagram - **Layout**: `FlexiCup_PCB.pdf` - PCB layout design - **BOM**: `FlexiCup_PCB_BOM.xlsx` - Electronic components list - **Manufacturing**: `Gerber_PCB.zip` - Gerber files for PCB fabrication **Key Components**: - ESP32S3 microcontroller with Wi-Fi - OV5640 camera interface - LED driver circuits - Power management (3.7V LiPo battery) - Wireless charging circuit ### Firmware Code **ESP32S3 Camera Firmware** Located in `Hardware/Firmware/ESPCAM/`, built with ESP-IDF framework. **Main Features**: - **Camera Control** (`camera.c/h`): OV5640 camera configuration and image capture - **LED Control** (`led.c/h`): Illumination switching for vision-tactile sensing - **HTTP Server** (`httpServer.c/h`): Web interface and video streaming - **Wi-Fi** (`wifiConnect.c/h`): Wireless communication **Key Specifications**: - Image Resolution: 640Γ—480 @ 30 Hz - Streaming: Real-time video over Wi-Fi (UDP) - Power: 3.7V 300mAh LiPo battery with wireless charging - Runtime: ~30 minutes continuous operation **Building the Firmware** ```bash cd Hardware/Firmware/ESPCAM idf.py build idf.py flash ``` ## πŸ’» Software ### Diffusion Policy Implementation Located in `Software/diffusion_policies/`, this contains the complete machine learning pipeline for contact-aware manipulation. **Main Components**: - **Data Collection** (`data_collect.py`): Automated data collection pipeline for demonstration gathering - **Training** (`train.py`): Diffusion policy training with multimodal observations - **Deployment** (`deploy.py`): Real-time deployment for robot control - **Environment** (`environment.yml`): Conda environment with all dependencies **Key Features**: - Multimodal fusion of dual-zone vision-tactile data via multi-head attention (8 heads, 512-d) - Diffusion policy with action chunking (8-step history, 48-step horizon) - AdamW optimizer with cosine annealing, 500 epochs training - Real-time inference for robot control at 10 Hz - Contact-aware manipulation with illumination switching and valve control **Setup and Usage** ```bash cd Software/diffusion_policies conda env create -f environment.yml conda activate flexicup python train.py --config configs/flexicup_config.yaml ``` ## πŸ“Š System Specifications ### Mechanical - **Normal Force**: 41.5 N (mean, at βˆ’80 kPa vacuum) - **Shear Force**: 8.34 N (mean) - **Modes**: Vacuum (sustained-contact adhesion) and Bernoulli (contactless lifting) - **Configurations**: 4 modular bottom designs (I–II vacuum, III–IV Bernoulli) - **Material**: Dual-layer PDMS membrane (30:1 base + Ag:PDMS 100:1 reflective layer) ### Electronics - **Controller**: ESP32-S3 (dual-core, Wi-Fi enabled) - **Camera**: OV5640 with 180Β° fisheye lens - **Resolution**: 640Γ—480 @ 30 Hz - **Connectivity**: Wi-Fi 802.11 b/g/n (UDP streaming) - **Power**: 3.7V 300mAh LiPo with wireless charging (12.5 ΞΌH coil, 200 mA) - **Runtime**: ~30 minutes continuous operation ### Sensing - **Dual-Zone**: Central (switchable vision-tactile via LED control) + Peripheral (continuous spatial awareness) - **Modality Switching**: Real-time illumination control by ESP32-S3 - **Multimodal Recognition**: 100% accuracy (vs. vision-only 82.5%, tactile-only 46.7%) ## πŸ“„ Documentation - **Research Paper**: `PDF/paper.pdf` - **Tutorial**: `TUTORIAL.pdf` - Complete fabrication and deployment guide - **Companion Website**: `index.html` (view at https://jump-howl.github.io/FlexiCup/) - **Hardware README**: `Hardware/README.md` - **Software README**: `Software/diffusion_policies/README.md` ## πŸŽ₯ Video Demonstrations Optimized videos are available in `static/video/optimized/`: - `overview.mp4`: System overview - `integrated_show.mp4`: Hardware integration and assembly - `multimodal_performance.mp4`: Vision-tactile sensing demonstration - `modular_task.mp4`: Modular perception-driven grasping (vacuum & Bernoulli) - `dptask1.mp4`: Inclined transport task (diffusion policy) - `dptask2.mp4`: Orange extraction task (diffusion policy) - `baseline1.mp4` / `baseline2.mp4`: BC-RNN baseline comparisons - `Wafer_Bernoulli.mp4` / `Wafer_Vaccum.mp4`: Wafer handling comparison - `Move_Orange.mp4` / `Move_Bottle.mp4`: Dynamic performance evaluation ## πŸš€ Getting Started ### Hardware Assembly 1. Review CAD models in `Hardware/Fabrication/CAD/` 2. Fabricate PCB using files in `Hardware/Fabrication/PCB/` 3. Follow BOM files for component sourcing 4. Assemble according to CAD models ### Firmware Setup 1. Install ESP-IDF framework 2. Navigate to `Hardware/Firmware/ESPCAM/` 3. Configure Wi-Fi settings in `main/src/wifiConnect.c` 4. Build and flash firmware ### Software Setup 1. Set up the diffusion policy environment: ```bash cd Software/diffusion_policies conda env create -f environment.yml conda activate flexicup ``` 2. Collect training data or use provided datasets 3. Train the diffusion policy model 4. Deploy for real-time robot control ### Testing 1. Power on the device 2. Connect to FlexiCup Wi-Fi network 3. Access web interface via browser 4. Test vision and tactile sensing modes 5. Run software algorithms for manipulation tasks ## πŸ“§ Contact For questions or collaboration inquiries, please refer to the paper for contact information. ## πŸ“œ License This project is licensed under Creative Commons Attribution-ShareAlike 4.0 International License. --- **Note**: Large CAD files (*.STEP) are managed with Git LFS. Clone with `git lfs clone` to download all files.