Abbas Abdolmaleki
From IEETA
Subject | Information theoretic stochastic search | |
---|---|---|
Advisor | Luis Paulo Reis, Nuno Lau | |
Group | Intelligent Robotics and Systems | |
Status | PhD student | |
Starts | 2011/09/01 | |
Ends | 2018/12/31 | |
Country | Portugal | |
Projects | ||
Past projects | EUROC, FCPortugal | |
Events | GECCO'2017 | |
Proposals | ||
Personal page |
Abbas Abdolmaleki joined the IEETA and IRIS Labs as a Ph.D. student in September 2011 to join the Robot Soccer Project under the supervision of Luis Paulo Reis and Nuno Lau. During his PhD, Abbas is working on stochastic search methods with robotics application in mind. Abbas also has participated in several robotic projects, and has achieved several awards in international competitions, as for example RoboCup. Currently he is the team leader of FC Portugal soccer humanoid robots simulation project.
Before his Ph.D, Abbas completed his Master Degree in Computer Engineering filed of Artificial Intelligence at the University of Isfahan. His thesis entitled “Trajectory generation for omnidirectional biped robot walking using machine learning
methods" was written under the supervision of Nasser Ghasem-Aghaee. Besides, he worked on rescue robots and humanoid soccer robots. Current version of his Curriculum Vitae is available here, and his full list of publications and corresponding bibtex files can be found on his Google scholar account. A few source codes of his projects can be accessed from his Github account. And some of the videos of his projects can be found here.
Research Interests
My research is inspired by the aim of creating intelligent systems, with an emphasis on self-learning, which gives adaptation capabilities in unknown environments. In particular I investigate the challenges behind matching the capabilities of theoretically-grounded algorithms with the practical demands of real-world applications. I have applied my research in domains such as humanoid soccer robots and rescue robots. I actively participate and contribute to events such as Robocup. My research is summarized by the following projects and videos:
Stochastic Search For Motor Learning
Stochastic search algorithms are general black-box optimizers. Due to their ease of use and their generality, they have recently also gained a lot of attention in operations research, machine learning and policy search. Yet, these algorithms require a lot of evaluations of the objective, scale poorly with the problem dimension, are affected by highly noisy objective functions and may converge prematurely. To alleviate these problems, we introduce a new surrogate-based stochastic search approach. We learn simple, quadratic surrogate models of the objective function. As the quality of such a quadratic approximation is limited, we do not greedily exploit the learned models. The algorithm can be misled by an inaccurate optimum introduced by the surrogate. Instead, we use information theoretic constraints to bound the `distance' between the new and old data distribution while maximizing the objective function. Additionally the new method is able to sustain the exploration of the search distribution to avoid premature convergence. A video of this project is available at: [ https://www.youtube.com/watch?v=DB93_ojldKo] . In this video we used our method along with imitation learning to learn a table tennis stroke.
Contextual Stochastic Search For Multi Task Learning
Many stochastic search algorithms require relearning if the task changes slightly to adapt the solution to the new situation or the new context. In this project we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective might change slightly for each parameter vector evaluation. Therefore we we investigated the contextual stochastic search algorithms that can learn from multiple tasks simultaneously. Two videos of this project are available at [ https://www.youtube.com/watch?v=eMTr4hA37Dw ] and at [ https://www.youtube.com/watch?v=bKNBMPIEWQw ].
Humanoid Robot Omnidirectional Walking
In this project, we propose a novel omnidirectional walking engine that achieves human like, stable and fast walking. We augment the 3D inverted pendulum with a spring model to implement a height change in the robot’s center of mass trajectory. This model is used as simplified model of the robot and the zero moment point (ZMP) criterion is used as the stability indicator. Two videos of the project are available at [ https://www.youtube.com/watch?v=jHTba15b01M ] and at [ https://www.youtube.com/watch?v=z_0Udxk0VUg ].
Robocup Humanoid Soccer Simulation
In this project we build a team of simulated humanoid robots in context of FCPortugal project. We actively participate in Robocup competitions and we have won several titles. A video of this project is available at: [ https://www.youtube.com/watch?v=EwYBBUOF6Ao ]
Intelligent Wheelchair Simulator
In this project we developed a virtual intelligent wheelchair with multimodal interface. A video of this project is available at: [ https://www.youtube.com/watch?v=1enZjUB90eU ]
Publications
Articles in international journals listed in the ISI
- Abbas Abdolmaleki, David Simões, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Contextual Direct Policy Search: With Regularized Covariance Matrix Estimation. Journal of Intelligent & Robotic Systems, vol. 96, no. 2, p. 141-157, November 2019 [IF 2018: 2.020 (Q3, 17/26)]
- Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis, Jan Peters, Gerhard Neumann. Contextual policy search for linear and nonlinear generalization of a humanoid walking controller. Journal of Intelligent & Robotic Systems, vol. 83, no. 3, p. 393-408, September 2016 [IF 2016: 1.512 (Q3, 17/26)] [IF 2018: 2.020 (Q3, 17/26)]
- Nima Shafii, Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis. Development of an Omnidirectional Walk Engine for Soccer Humanoid Robots. International Journal of Advanced Robotic Systems, vol. 12, no. 193, December 2015 [IF 2015: 0.615 (Q4, 19/25)]
Articles in conference proceedings
- Abbas Abdolmaleki, David Simões, Nuno Lau, Luis Paulo Reis, Bob Price, Gerhard Neumann. Stochastic Search In Changing Situations. The AAAI-17 Workshop on Human-Machine Collaborative Learning, p. 636-642, 2017
- Abbas Abdolmaleki, Bob Price, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Contextual Covariance Matrix Adaptation Evolutionary Strategies. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), Melbourne, Australia, p. 1378-1385, August 2017
- Abbas Abdolmaleki, Bob Price, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Deriving and Improving CMA-ES with Information Geometric Trust Regions. GECCO - The Genetic and Evolutionary Computation Conference, Berlin, Germany, July 2017
- Abbas Abdolmaleki, David Simoes, Nuno Lau and Luís Paulo Reis. Learning a Humanoid Kick With Controlled Distance. RoboCup 2016: Robot World Cup XX, vol. LNCS, 2016
- Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Non-parametric contextual stochastic search. (2016) IEEE International Conference on Intelligent Robots and Systems, South Korea, p. 2643-2648, November 2016
- Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Contextual Stochastic Search. Genetic and Evolutionary Computation Conference - GECCO, Denver, July 2016
- Abbas Abdolmaleki, David Simões, Nuno Lau, Luis Paulo Reis. Learning a Humanoid Kick With Controlled Distance. RoboCup 2016: Robot World Cup XX, July 2016
- Abbas Abdolmaleki, David Simoes, Nuno Lau, Luís Reis, Gerhard Neumann. Contextual Relative Entropy Policy Search with Covariance Matrix Adaptation. IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2016, Bragança, May 2016
- Abbas Abdolmaleki, Rudolf Lioutikov, Jan Peters, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Model-Based Relative Entropy Stochastic Search. Advances in Neural Information Processing Systems (NIPS), MIT Press, p. 3523-3531, 2015
- Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis, Gerhard Neumann. Regularized Covariance Estimation for Weighted Maximum Likelihood Policy Search Methods. Proceedings of the International Conference on Humanoid Robots, 2015
- Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis, Jan Peters, Gerhard Neumann. Contextual policy search for generalizing a parameterized biped walking controller. Proceedings - 2015 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2015, p. 17-22, April 2015
- Abbas Abdolmaleki, Nima Shafii, Luis Paulo Reis, Nuno Lau, Jan Peters, Gerhard Neunann. Omnidirectional Walking with a Compliant Inverted Pendulum Model. Advances in Artificial Intelligence -- IBERAMIA 2014, Santiago de Chile, Chile, vol. Lecture Notes in Computer Science 2014, p. 481-493, November 2014
- Abbas Abdolmaleki, Jan Peters, Luis Paulo Reis, Nuno Lau, Gerhard Neumann. Contextual Policy Search for Learning a Flexible Bipedal Robot Locomotion Controller. Dynamic Walking Conference, Zürich, Switzerland, June 2014
- Nima Shafii, Abbas Abdolmaleki, Rui Ferreira, Nuno Lau, Luis Paulo Reis. Omnidirectional Walking and Active Balance for Soccer Humanoid Robot. Progress in Artificial Intelligence, 16th Portuguese Conference on Artificial Intelligence, EPIA 2013, Angra do Heroísmo, Azores, Portugal, September 9-12, 2013. Proceedings, Angra do Heroismo, Portugal, vol. Lecture Notes in Computer Science, Volume 8154, Springer, p. 283-294, September 2013
- Abbas Abdolmaleki, Mostafa Movahedi, Nuno Lau, Luis Paulo Reis. A Distributed Cooperative Reinforcement Learning Method for Decision Making in Fire Brigade Teams. RoboCup-2012: Robot Soccer World Cup XVI, Mexico City, Mexico, June 2013
- Rui Ferreira, Nima Shafii, Nuno Lau, Luis Paulo Reis, Abbas Abdolmaleki. Diagonal Walk Reference Generator based on Fourier Approximation of ZMP Trajectory. Proc. of 13th International Conference on Autonomous Robot Systems and Competitions, Lisbon, Portugal, p. 94-99, April 2013
- Nima Shafii, Abbas Abdolmaleki, Nuno Lau, Luis Paulo Reis. A Robust Closed-Loop Gait for Humanoid Clock-Turning. Proceedings of Workshop on Optimality Principle and Adaptation in Humanoid Robotic Control in conjunction with the 2012 IEEE-RSJ International Conference on Intelligent Robots and Systems (IROS 2012), Vilamoura, Portugal, October 2012
- Abbas Abdolmaleki, Mostafa Movahedi, Sajjad Salehi, Nuno Lau, Luis Paulo Reis. A Reinforcement Learning Based Method for Optimizing the Process of Decision Making in Fire Brigade Agents. Progress in Artificial Intelligence, 15th Portuguese Conference on Artificial Intelligence, EPIA 2011, Lisbon, Portugal, vol. LNAI 7026, p. 340-351, October 2011
PhD thesis
- Abbas Abdolmaleki. Information Theoretic Stochastic Search. PhD Thesis, MAP-i - Universidades de Minho, Aveiro e Porto, December 2018