LeRobot hackathon: real-world RL on a robotic arm
RoboticsReinforcement LearningI took part in the LeRobot hackathon in Toulouse, organized by the Hugging Face team 🤗. It was my first step into real-world reinforcement learning and robotics.
We went from a very messy table full of hardware parts to a fully assembled robot in a few hours: a Moss v1 robotic arm with a gripper, which we then tried to teach to manipulate a cube.
We chose the Reinforcement Learning track. We calibrated the arm with a second "leader" arm we could control by hand, set up a camera, and built a simple real-world RL environment: a tray with a cube, where the goal is to push the cube to the other side. We recorded a dataset of "expert" demonstrations (us) with the LeRobot library, and trained the arm with offline RL using TD-MPC.
We then kept training with online RL, with the arm collecting its own trajectories in the real world. The result was fun: the robot became great at pushing the cube from right to left, but struggled with left to right.
This sparked my interest in real-world robotics: there is something mesmerizing about watching algorithms come to life outside a simulator. Thanks to my teammates Waris Radji, Anthony Kobanda, Mathieu Petitbois, Théo Michel and Mohamed Yassine Kabour, and to the LeRobot team for organizing the event!