Humanoid-Robot-Imitation-of-Human-Motion-from-Instructional-Videos

Topic :

With the fast emergence of robots in our era, the biggest challenge nowadays is to train these robots for different tasks. Inspired from the human learning from example process, interesting methods could be used to reach this goal. We choose to work on the “Topic L - Humanoid Robot Imitation of Human Motion from Instructional Videos” as it combines two interesting Machine Learning fields, Image Recognition and Reinforcement Learning.

The plan of work :

a. Read and understand the main ideas of the papers. (Fatine and Hamza)

b. Recover human shape and model from the “handtools dataset” ( Short video clips collected on Youtube). Apply the pretrained HMR model to the dataset of videos to get 3D motion trajectories from human demonstrators. (Fatine)

c. Use the simple 3D human motions post-process described in the paper “ SFV: Reinforcement Learning of Physical Skills from Videos “ of the datasets to get 3D motion trajectories from human demonstrators. (Hamza)

d. Take a humanoid robot model and build a mapping from a Human Mesh Recovery human pose vector to the humanoid robot configuration vector. (Fatine)

e. Apply the Reinforcement Learning approach to map the human motions recovered from the dataset videos to the motions of a humanoid robot. (Hamza)