Building a Sim-to-Real Robotics Pipeline: Imitation Learning, Digital Twins, and Synthetic Training for the SO-101

This project develops a complete robotics learning pipeline using the 6-DOF LeRobot SO-101 arm with a gripper, beginning with a USB control interface for robot calibration and basic grasping operations. A digital twin is then created in NVIDIA Omniverse Isaac Sim to mirror and control the physical robot, while scripted tools generate batches of simulated manipulation tasks for imitation-learning experiments. The project ultimately investigates sim-to-real transfer by training policies in Isaac Lab and evaluating their deployment on the physical robot.

Intern: Arvin Kandasamy

Mentors: Homer Li (AMDS/A4I), Carlos Barajas (AMDS/A4I), Stephen Vance (FPS/KVM)