Mohammadreza Kasaei

• September 2021: Our paper Robust Biped Locomotion Using Deep ReinforcementLearning on Top of an Analytical Control Approach got accepted at Robotics and Autonomous Systems.

• April 2021:  We proposed a hybrid biped stabilizer system based on analytical control and learning of symmetrical residual physics.   The paper is available online at Here.

• February 2021:  We designed a CPG-based agile and versatile locomotion framework using proximal symmetry loss.   The paper is available online at Here.

• December 2020:  We used the Linear Inverted Pendulum(LIP) and Divergent Component of Motion(DCM) concepts to formulate the biped locomotion and stabilization as an analytical control framework. On top of that, a neural network with symmetric partial data augmentation learns residuals to adjust the joint's position and thus improving the robot's stability when facing external perturbations.   The paper is available online at Here.

• September 2020:  We propose a modular framework to generate robust biped locomotion using a tight coupling between an analytical walking approach and deep reinforcement learning.  The paper is available online at Here.

• July 2020:  We have deployed our framework on a simulated torque-controlled humanoid to verify its performance and robustness and performed a set of simulations. We extended our ICARSC2020 paper and submitted it to SN Applied Sciences.

• April 2020:  I have presented my paper at ICARSC2020.

• March 2020: Our paper A Robust Model-Based Biped Locomotion Framework Based on Three-Mass Model: From Planning to Control got accepted at ICARSC2020.

• February 2020: Our hierarchical walk engine has been successfully ported on a simulated COMAN humanoid robot. A video of this simulation is available online here!!!

• January 2020: Proximal Policy Optimization(PPO) approach is used on top of analytical control approach to improve the stability of a simulated humanoid robot while it is walking. The interesting results will be published soon.

• November 2019: I have presented my paper at IROS2019.

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains, or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This paper tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. The Linear Inverted Pendulum Model and Central Pattern Generator concepts are used to develop a closed-loop walk engine that is combined with a reinforcement learning module. This module learns to regulate the walk engine parameters adaptively and generates residuals to adjust the robot's target joint positions (residual physics). Additionally, we propose a proximal symmetry loss to increase the sample efficiency of the Proximal Policy Optimization algorithm by leveraging model symmetries. The effectiveness of the proposed framework was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.

• Publications:

¤  Paper (Under-Review)

• Videos:

¤ Video

Although humanoid robots are made to resemble humans, their stability is not yet comparable to ours. When facing external disturbances, humans efficiently and unconsciously combine a set of strategies to regain stability. This work deals with the problem of developing a robust hybrid stabilizer system for biped robots. The Linear Inverted Pendulum(LIP) and Divergent Component of Motion(DCM) concepts are used to formulate the biped locomotion and stabilization as an analytical control framework. On top of that, a neural network with symmetric partial data augmentation learns residuals to adjust the joint's position and thus improving the robot's stability when facing external perturbations. The performance of the proposed framework was evaluated across a set of challenging simulation scenarios. The results show a considerable improvement over the baseline in recovering from large external forces. Moreover, the produced behaviors are human-like and robust to considerably noisy environments.

• Publications:

¤ paper (Under-Review)

• Videos:

¤ Video

This paper proposes a modular framework to generate robust biped locomotion using a tight coupling between an analytical walking approach and deep reinforcement learning. This framework is composed of six main modules which are hierarchically connected to reduce the overall complexity and increase its flexibility. The core of this framework is a specific dynamics model which abstracts a humanoid's dynamics model into two masses for modeling the upper and lower body. This dynamics model is used to design an adaptive reference trajectories planner and an optimal controller which are fully parametric. Furthermore, a learning framework is developed based on Genetic Algorithm (GA) and Proximal Policy Optimization~(PPO) to find the optimum parameters and to learn how to improve the stability of the robot by moving the arms and changing its center of mass (COM) height. A set of simulations are performed to validate the performance of the framework using the official RoboCup 3D League simulation environment. The results validate the performance of the framework, not only in creating a fast and stable gait but also in learning to improve the upper body efficiency.  

• Publications:

¤ To appear in Robotics and Autonomous Systems (Under-Review)

• Videos:

¤ Demo1: Omnidirectional Walk

¤ Demo2: Optimizing the Walking Parameters

¤ Demo3: Learning to Improve the Upper Body Efficiency

Biped robots are inherently unstable because of their complex kinematics as well as dynamics. Despite the many research efforts in developing biped locomotion, the performance of biped locomotion is still far from the expectations. This paper proposes a model-based framework to generate stable biped locomotion. The core of this framework is an abstract dynamics model which is composed of three masses to consider the dynamics of stance leg, torso, and swing leg for minimizing the tracking problems. According to this dynamics model, we propose a modular walking reference trajectories planner which takes into account obstacles to plan all the references. Moreover, this dynamics model is used to formulate the controller as a Model Predictive Control (MPC) scheme which can consider some constraints in the states of the system, inputs, outputs and also mixed input-output. The performance and the robustness of the proposed framework are validated by performing several numerical simulations using MATLAB. Moreover, the framework is deployed on a simulated torque-controlled humanoid to verify its performance and robustness. The simulation results show that the proposed framework is capable of generating biped locomotion robustly.

• Publications:

¤ To appear in SN Applied Sciences (Under-Review)

• Videos:

¤ Demo0: Overall performance

¤ Demo1: Walking around a disk

¤ Demo2: Omnidirectional walking

¤ Demo3: Push recovery

¤ Demo4: Walking on uneven terrains

Biped robots are inherently unstable because of their complex kinematics as well as dynamics. Despite types of research in developing biped locomotion, the performance of biped locomotion is still far from the expectations. This paper proposes a model-based framework to generate stable biped locomotion. The core of this framework is an abstract dynamics model which is composed of three masses to consider the dynamics of stance leg, torso and swing leg for minimizing the tracking problems. According to this dynamics model, we propose a modular walking reference trajectories planner which takes into account obstacles to plan all the references. Moreover, this dynamics model is used to formulate the controller as a Model Predictive Control (MPC) scheme which can consider some constraints in the states of the system, inputs, outputs and also mixed input-output. The performance and the robustness of the proposed framework are validated by performing several simulations using MATLAB. The simulation results show that the proposed framework is capable of generating the biped locomotion robustly.

• Publications:

¤ paper (ICARSC2020)

• Videos:

¤ Demo1: Path planning scenario scenario

To get out of the laboratory and work in a real environment, humanoids robots must be able to keep their stability under harsh conditions. Since humanoid robots have similar kinematic to a human, humans expect these robots to be robustly capable of stabilizing even in a challenging situation as when a severe push is applied. This paper presents a robust walking framework which not only takes into account the traditional push recovery approaches (e.g., ankle, hip and step strategies) but also uses the concept of Divergent Component of the Motion (DCM) to adjust the next step timing and location. The control core of the proposed framework is composed of a Linear-Quadratic-Gaussian (LQG) controller and two proportional controllers. In this framework, the LQG controller tries to track the reference trajectories and the proportional controllers are designed to adjust the next step timing and location that allow the robot to recover from a severe push. The robustness and the performance of the proposed framework have been validated by performing a set of simulations, including walking and push recovery, using MATLAB. The simulation results verified that the proposed framework is capable of providing robust walking even in very challenging situations.

• Publications:

¤ To appear in Advanced Robotics

Developing robust locomotion for humanoid robots is a complex task due to the unstable nature of these robots and also to the unpredictability of the terrain. A robust locomotion planner is one of the fundamental components for generating stable biped locomotion. This paper presents an optimal closed-loop biped locomotion planner which can plan reference trajectories even in challenging conditions. The proposed planner is designed based on a Time-Varying Model Predictive Control (TVMPC) scheme to be able to consider some constraints in the states, inputs and outputs of the system and also mixed input-output. Moreover, the proposed planner takes into account the vertical motion of the Center of Mass (COM) to generate walking with mostly stretched knees which is more human-like. Additionally, the planner uses the concept of Divergent Component of Motion (DCM) to modify the reference ZMP online to improve the withstanding level of the robot in the presence of severe disturbances. The performance and also the robustness of the proposed planner are validated by performing several simulations using MATLAB. The simulation results show that the proposed planner is capable of generating the biped locomotion robustly.

• Publications:

¤ arxiv.org

• Videos:

¤ Demo1: Diagonal walking on a flat terrain scenario

¤ Demo2: Stair climbing scenario

This paper proposes a framework designed to generate a closed-loop walking engine for a humanoid robot. In particular, the core of this framework is an abstract dynamics model which is composed of two masses that represent the lower and the upper body of a humanoid robot. Moreover, according to the proposed dynamics model, the low-level controller is formulated as a LinearQuadratic-Gaussian (LQG) controller that is able to robustly track the desired trajectories. Besides, this framework is fully parametric which allows using an optimization algorithm to find the optimum parameters. To examine the performance of the proposed framework, a set of simulations using a simulated Nao robot in the RoboCup 3D simulation environment has been carried out. Simulation results show that the proposed framework is capable of providing fast and reliable omnidirectional walking. After optimizing the parameters using the genetic algorithm (GA), the maximum forward walking velocity that we have achieved was 80:5cm=s.

• Publications:

¤ RoboCup 2019

• Videos:

¤ Demo1: Omnidirectional walking scenario

¤ Demo2: Maximum walking speed scenario

Generating robust locomotion for a humanoid robot in the presence of disturbances is difficult because of its high degrees of freedom and its unstable nature. In this paper, we used the concept of Divergent Component of Motion (DCM) and propose an optimal closed-loop controller based on Linear-Quadratic-Gaussian to generate a robust and stable walking for humanoid robots. The biped robot dynamics has been approximated using the Linear Inverted Pendulum Model (LIPM). Moreover, we propose a controller to adjust the landing location of the swing leg to increase the withstanding level of the robot against a severe external push. The performance and also the robustness of the proposed controller are analyzed and verified by performing a set of simulations using MATLAB. The simulation results showed that the proposed controller is capable of providing robust walking even in the presence of disturbances and in challenging situations.

• Publications:

¤ IROS 2019

This paper presents a model-based balance stabilization system that takes into account not only the stable part of COM dynamics but also the unstable part. In this system, the overall dynamics of a humanoid robot is approximated using a Linear Inverted Pendulum Plus Flywheel Model (LIPPFM). Moreover, Divergent Component of Motion (DCM) is used to define when and where a robot should take a step to prevent falling. The proposed system has been successfully tested by performing several simulations using MATLAB. The simulation results show this system is capable of stabilizing the balance of the robot in various conditions.

• Publications:

¤ WS-IROS2018

Biped robots have high degrees of freedom and they are naturally unstable, hence design and develop a reliable walking controller is a complex subject that is one of the interesting topics in the robotic community. In this paper, we proposed a model-based walking controller which is able to negate the effect of external impacts not only by applying compensating torques but also by adjusting the landing location of the swing leg. This controller composed of two levels of control which takes into account the stable and unstable dynamics parts of the center of mass (COM). In the proposed controller, the overall dynamics of a humanoid robot is approximated using an enhanced version of the Linear Inverted Pendulum Plus Flywheel Model (ELIPPFM) and according to this dynamics model, an optimal controller is designed to track the reference trajectories. Moreover, Divergent Component of Motion (DCM) is used to define when and where a robot should take a step to prevent falling. The proposed controller has been successfully tested by performing several simulations using MATLAB. The results showed that the proposed controller is capable of controlling the balance of a simulated robot in presence of severe disturbances.

• Publications:

¤ ICARSC2019

Bipedal robots are essentially unstable because of their complex kinematics as well as high dimensional state space dynamics, hence control and generation of stable walking is a complex subject and still one of the active topics in the robotic community. Nowadays, there are many humanoids performing stable walking, but fewer show effective push recovery under pushes. In this paper, we firstly review more commonly used abstract dynamics models for a humanoid robot which are based on the inverted pendulum and show how these models can be used to provide walking for a humanoid robot and also how a hierarchical control structure could fade the complexities of a humanoid walking. Secondly, the reviewed models are compared together not only in an analytical manner but also by performing several numerical simulations in a push recovery scenario using MATLAB. These theoretical and simulation studies quantitatively compare these models regarding regaining balance. The results showed that the enhanced version of Linear Inverted Pendulum Plus Flywheel is the ablest dynamics model to regain the stability of the robot even in very challenging situations.

• Publications:

¤ MAPi Seminar(MAPiS)

¤ arxiv.org

Bipedal robots are essentially unstable because of their complex kinematics as well as high dimensional state space dynamics, hence control and generation of stable walking is a complex subject that is still one of the active topics in the robotic community. This paper proposes a closed-loop model-based walk engine that takes into account push recovery strategies. In this paper, Linear Inverted Pendulum Plus Flywheel Model (LIPPFM) is extended and used to approximate the overall dynamics of a humanoid robot. We extended this model by releasing the height constraint of the center of mass (COM) as well as by considering the mass of the pendulum to increase the accuracy of the model. In this framework, a step is composed of a double support phase in addition to a single support phase. Moreover, ZMP and reference trajectory generators are formulated based on the input parameters, and tracking problems are formulated as a finite-time horizon linear quadratic regulator (LQR) problem. The proposed framework has been successfully tested by performing several simulations using MATLAB. The simulation results show this framework is capable to provide stable walking on uneven terrain.

• Publications:

¤ ICARSC2018

Developing an optimized omnidirectional walking for biped robots is a challenging task due to their complex dynamics and kinematics. This paper proposes a hierarchical walk engine structure to generate fast and stable walking. In particular, this structure provides a closed-loop CPG-based omnidirectional walking that takes into account two human-inspired push recovery strategies. In addition, this structure is fully parametric and allows using a policy search algorithm to find the optimum parameters for walking. To show the performance of the proposed structure, a set of experiments on a simulated NAO robot has been carried out. Experimental results demonstrate that the proposed structure is able to generate fast and stable omnidirectional walking. The maximum speed of forward walking that we have achieved was 59cm=s.

• Publications:

¤ ROBOT2017

• Videos:

¤ Demo1: Omnidirectional walking scenario

¤ Demo1: Maximum walking speed scenario

According to the similarity in kinematic architecture, bipedal robots are the most appropriate type of robot to operate in humanoid environments. Most of the humanoid robots have more than 20 degrees of freedom (DoF), therefore they have complex kinematics and dynamics. Due to these complexities, developing a stable walking engine is a difficult subject which is still one of the main challenges. In this paper, a hierarchical walking engine is presented which tries to fade the complexities and increases flexibility and portability. To generate the reference trajectories of walking, Linear Inverted Pendulum Plus Flywheel Model is used. We enhanced this model to release the height constraint of the Center of Mass (CoM). This enhancement not only provides more natural motion but also it provides a larger stride. The reliability of the proposed structure is verified through real experiments for a 110cm bipedal robot. The experimental results show the performance of this controller to keep the robot’s stability during walking. The average speed of walking that we have achieved was 20cm=sec.

• Publications:

¤ CLAWAR2017

• Videos:

¤ Demo1: Real Robot Experiment

According to the similarity in kinematic architecture, bipedal robots are the most appropriate type of robot to operate in humanoid environments. Most of the humanoid robots have more than 20 degrees of freedom (DoF), therefore they have complex kinematics and dynamics. Due to these complexities, developing a stable walking engine is a difficult subject which is still one of the main challenges. In this paper, a hierarchical walking engine is presented which tries to fade the complexities and increases flexibility and portability. To generate the reference trajectories of walking, Linear Inverted Pendulum Plus Flywheel Model is used. We enhanced this model to release the height constraint of the Center of Mass (CoM). This enhancement not only provides more natural motion but also it provides a larger stride. The reliability of the proposed structure is verified through real experiments for a 110cm bipedal robot. The experimental results show the performance of this controller to keep the robot’s stability during walking. The average speed of walking that we have achieved was 20cm=sec.

• Publications:

¤ ICARSC2017

• Videos:

¤ Demo1: Real Robot Experiment

Avoiding a fall after strong collisions between two players is an important capability for an adult-size humanoid robot. Particularly in the RoboCup competitions, matches are really competitive and collisions between players occur frequently. In the adult-size humanoid league, robots are tall and heavy. Whenever robots contact each other during moving, several unpredicted non-linear forces are applied into the robots. As a consequence, the stability of robots goes out of control and they fall down. In order to maintain and recover the balance of an adult-size humanoid robot against external disturbances, a Neural Network is used for learning from past experiments to reduce the effect of disturbances forces by providing proper step sizes and joint angles to the robot. In our approach, the robot’s controller is learned using several empirical experiments and tested on a real adult-size humanoid robot namely Ariana from BehRobot humanoid team. Experiments demonstrate after receiving strong pushes during walking, Ariana can efficiently recover its stability in the real environment.

• Publications:

¤ ICARSC2016

• Videos:

¤ Demo1: Real Robot Experiment