Learning to control a robot can be a difficult endeavor, but there are simpler approaches and strategies that can be used, particularly for beginners or those seeking a clear approach.
MIT and Stanford University researchers have developed a novel machine-learning approach that might be used to drive a robot, such as a drone or autonomous vehicle, more effectively and efficiently in dynamic environments where conditions can change quickly. This technology could assist an autonomous car in learning to correct for slick road conditions in order to prevent a skid, enabling a robotic free-flyer to tow different items in space, or allow a drone to closely follow a downhill skier despite being buffeted by strong winds.
The researchers’ method combines specific structure from control theory into the process of learning a model, resulting in an effective method of regulating complex dynamics, such as those induced by wind influences on the trajectory of a flying vehicle. This structure can be thought of as a tip that can help advise how to control a system.
“The focus of our work is to learn intrinsic structure in the dynamics of the system that can be leveraged to design more effective, stabilizing controllers,” says Navid Azizan, the Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), and a member of the Laboratory for Information and Decision Systems (LIDS). “By jointly learning the system’s dynamics and these unique control-oriented structures from data, we’re able to naturally create controllers that function much more effectively in the real world.”
This work tries to strike a balance between identifying structure in your system and just learning a model from data. Our method is inspired by how roboticists use physics to create simpler robot models.
Spencer M. Richards
The focus of our work is to learn intrinsic structure in the dynamics of the system that can be leveraged to design more effective, stabilizing controllers,” says Navid Azizan, the Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), and a member of the Laboratory for Information and Decision Systems (LIDS). “By jointly learning the system’s dynamics and these unique control-oriented structures from data, we’re able to naturally create controllers that function much more effectively in the real world.”
The researchers’ strategy, which uses this structure in a learnt model, quickly pulls an efficient controller from the model, as opposed to other machine-learning methods, which require a controller to be created or taught separately with additional steps. Their approach, with this structure, can also learn an effective controller with less input than other approaches. This could help their learning-based control system operate better in quickly changing conditions.
Additional authors of the paper are Jean-Jacques Slotine, professor of mechanical engineering and of brain and cognitive sciences at MIT, and Marco Pavone, associate professor of aeronautics and astronautics at Stanford. The research will be presented at the International Conference on Machine Learning (ICML).
Learning a controller
Even when researchers know how to describe everything about the system, determining the optimum approach to direct a robot to do a specific task can be a difficult problem. A controller, for example, is the logic that allows a drone to follow a specific trajectory. This controller would instruct the drone on how to change its rotor forces to compensate for the effect of winds, which could cause it to deviate from a steady path to its destination.
This drone is a dynamical system, which means it is a physical system that changes over time. As it flies through the surroundings, its position and velocity fluctuate. Engineers can create a controller by hand if the system is simple enough.
Hand modeling a system captures a certain structure based on the physics of the system. For example, if a robot were manually modeled using differential equations, the relationship between velocity, acceleration, and force would be captured. The rate of change in velocity over time is dictated by the mass of the robot and the forces applied to it.
However, the system is frequently too complicated to be precisely modeled by hand. Aerodynamic effects, such as how swirling wind propels a flying aircraft, are notoriously difficult to calculate manually, according to Richards. Instead, researchers would collect data on the drone’s position, velocity, and rotor speeds over time and apply machine learning to fit a model of this dynamical system to the data.
However, these techniques do not often learn a control-based structure. This structure is useful in knowing how to configure the rotor speeds to best steer the drone’s motion over time. Many existing approaches employ data to learn a separate controller for the system after they have modeled the dynamical system.
“Other approaches that attempt to learn dynamics and a controller from data as separate entities are philosophically disconnected from how we normally do it for simpler systems. Our approach is more akin to manually deriving models from physics and connecting them to control,” Richards explains.
Identifying structure
The MIT and Stanford teams created a technique that employs machine learning to learn the dynamics model, but in such a way that the model has some prescribed structure that may be used to regulate the system. They can extract a controller straight from the dynamics model using this structure, rather than utilizing data to learn an entirely different model for the controller.
“We discovered that, in addition to learning the dynamics, it is critical to learn the control-oriented structure that supports effective controller design.” “In terms of data efficiency and tracking capability, our approach of learning state-dependent coefficient factorizations of the dynamics outperformed the baselines, proving to be successful in efficiently and effectively controlling the system’s trajectory,” Azizan explains.
When they tested this approach, their controller closely followed desired trajectories, outpacing all the baseline methods. The controller extracted from their learned model nearly matched the performance of a ground-truth controller, which is built using the exact dynamics of the system.
“By making simpler assumptions, we got something that actually worked better than other complicated baseline approaches,” Richards adds.
The researchers also discovered that their strategy was data-efficient, which indicates that it performed well even with limited data. It could, for example, effectively represent a highly dynamic rotor-driven vehicle with only 100 data points. With smaller datasets, methods that used numerous learnt components saw their performance degrade significantly faster.
Their technique could be especially valuable in circumstances when a drone or robot needs to learn quickly in rapidly changing conditions due to its efficiency. Furthermore, their method is broad and might be applied to a wide range of dynamical systems, from robotic arms to free-flying spacecraft in low-gravity situations.
