Abstract
Learning agile robotic policies requires large-scale demonstrations, but translating abundant human motion data to robots is bottlenecked by the embodiment gap and missing dynamic information. To bridge this gap, we propose Scalable Physics-Informed DExterous Retargeting (SPIDER), a physics-based retargeting framework to transform and augment kinematic-only human demonstrations into dynamically feasible robot trajectories at scale. Our key insight is that human demonstrations should provide global task structure and objective, while a sampling-based solver can be used to find the feasible solution given the physics constraints. As a general framework, SPIDER is an efficient physics-based retargeting method that can be applied to both humanoid whole-body loco-manipulation and dexterous manipulation across 9 humanoid/dexterous hand embodiments, 6 datasets and 3 simulators. By bypassing the need for policy optimization, it achieves state-of-the-art performance while being 10x faster than reinforcement learning (RL) baselines. Furthermore, SPIDER enables physics-based data augmentation—such as imposing external payloads, perturbations, and new contact patterns—to generate diverse data. We demonstrate that the retargeted motion can be executed on the robot directly open-loop or serve as feasible reference motion for efficient RL policy learning. Our pipeline enables large scale robot demonstration trajectory generation from human and we release full dataset with 6523 demonstrations across 4 hands and 191 distinct objects to assist future research.
Framework Overview
SPIDER converts human demonstrations into dynamically feasible
robot trajectories across 9 embodiments, 6 datasets, and 3 simulators using a training-free
sampling-based solver.
Full-Scale Dataset Release
We are releasing the full-scale SPIDER dataset: 6,523 demonstrations across 4 robot hands and 191 objects. Explore the currently published examples below, with videos streamed directly from Hugging Face.
Explore Retargeted Trajectories in 3D
Loading trajectory catalog…
Video Examples
Loading published examples…
Videos replay retargeted motion with orange-red robot–object contact points and available object textures. Contact points are recomputed from the saved poses.
Pipeline
Extract hand motion and object geometry from human data, retarget kinematic motion with physics-based sampling, then robustify and diversify the trajectories for deployment and learning.
Direct Deployment on Robots
Being dynamically feasible, the generated trajectories can be directly executed on the robot. The rollout data is augmented with domain randomization.
Pick Spoon from Bowl
Play Guitar
Rotate Bulb
Unplug
Pick Cup
Pick Spoon
Rotate Cube
Pick Duck
Pick Lego
Pick Toy
Retargeting for Dexterous Hands
Interactive Visualization
Choose a demo and open its interactive Viser scene. Drag to orbit, scroll to zoom, and use the playback timeline to explore the motion. Planned traces remain visible throughout playback.
Simulation Videos
Allegro Hand
Inspire Hand
Schunk Hand
XHand
Ability Hand - Tea
Ability Hand - Board Wiping
Inspire Hand - Board Lifting
Retargeting for Humanoid Robots
Interactive Visualization
Choose a demo and open its interactive Viser scene. Drag to orbit, scroll to zoom, and use the playback timeline to explore the motion. Planned traces remain visible throughout playback.
Data Augmentation
As a physics-based retargeting method, SPIDER can diversify single demonstration into multiple feasible trajectories with
new objects and environments.
Contact Guidance
Contact guidance is used to ensure desired contact sequences is achieved.
With Contact Guidance - Allegro
Without Contact Guidance - Allegro
With Contact Guidance - Constraint G1
Without Contact Guidance - Constraint G1
BibTeX
@misc{pan2025spiderscalablephysicsinformeddexterous,
title={SPIDER: Scalable Physics-Informed Dexterous Retargeting},
author={Chaoyi Pan and Changhao Wang and Haozhi Qi and Julen Urain and Zixi Liu and Homanga Bharadhwaj and Akash Sharma and Tingfan Wu and Guanya Shi and Jitendra Malik and Francois Hogan},
year={2025},
eprint={2511.09484},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2511.09484},
}