Training a machine learning model in space would be an incredible achievement. On board a satellite in space, scientists trained a machine learning model. This accomplishment has the potential to transform the capabilities of remote-sensing satellites by enabling real-time monitoring and decision making for a variety of applications.
For the first time, a project led by the University of Oxford trained a machine learning model on board a satellite in space. This accomplishment has the potential to transform the capabilities of remote-sensing satellites by enabling real-time monitoring and decision making for a variety of applications.
Data collected by remote-sensing satellites is critical for many critical activities, including aerial mapping, weather prediction, and deforestation monitoring. Most satellites can only collect data passively right now because they lack the capability to make decisions or detect changes. Instead, data must be relayed to Earth for processing, which can take hours or even days. This reduces the ability to detect and respond to rapidly developing events, such as a natural disaster.
Machine learning has a huge potential for improving remote sensing. The ability to push as much intelligence into satellites as possible will make space-based sensing increasingly autonomous.
Professor Andrew Markham
To overcome these restrictions, a group of researchers led by DPhil student Vít Růžička (Department of Computer Science, University of Oxford), took on the challenge of training the first machine learning program in outer space. During 2022, the team successfully pitched their idea to the Dashing through the Stars mission, which had issued an open call for project proposals to be carried out on board the ION SCV004 satellite, launched in January 2022. During the autumn of 2022, the team uplinked the code for the program to the satellite already in orbit.
In contrast to training on the ground, the researchers trained a simple model to detect changes in cloud cover from aerial images directly onboard the satellite. The model was built using a technique known as few-shot learning, which allows a model to learn the most important features to look for when it only has a few samples to train on. The ability to compress data into smaller representations makes the model faster and more efficient.
Vít Růžička explained: ‘The model we developed, called RaVAEn, first compresses the large image files into vectors of 128 numbers. During the training phase, the model learns to keep only the informative values in this vector; the ones that relate to the change it is trying to detect (in this case, whether there is a cloud present or not). This results in extremely fast training due to having only a very small classification model to train.’
While the first part of the model was trained on the ground to compress newly seen images, the second part (which determined whether the image contained clouds or not) was trained directly on the satellite.
Developing a machine learning model would normally necessitate several rounds of training, utilizing the power of a cluster of linked computers. The team’s tiny model, on the other hand, finished the training phase (using over 1300 images) in about one and a half seconds.
When the team tested the model’s performance on new data, it detected whether the cloud was present or not in less than a tenth of a second. This entailed encoding and analyzing a scene the size of 4.8×4.8 km2 (equivalent to nearly 450 football fields).
According to the researchers, the model could be easily adapted to perform different tasks and use different types of data. ‘We now intend to develop more advanced models that can automatically differentiate between changes of interest (such as flooding, fires, and deforestation) and natural changes (such as natural changes in leaf color across the seasons,’ Vt Rika added. Another goal is to create models for more complex data, such as hyperspectral satellite images. This could allow for the detection of methane leaks, for example, and has important implications for combating climate change.’
Performing machine learning in outer space could also help overcome the problem of on-board satellite sensors being affected by the harsh environmental conditions, so that they require regular calibration. Vít Růžička said: ‘Our proposed system could be used in constellations of non-homogeneous satellites, where reliable information from one satellite can be applied to train the rest of the constellation. This could be used, for instance, to recalibrate sensors that have degraded over time or experienced rapid changes in the environment.’
‘Machine learning has a huge potential for improving remote sensing,’ said Professor Andrew Markham, who supervised Vt’s DPhil research. ‘The ability to push as much intelligence into satellites as possible will make space-based sensing increasingly autonomous. Allowing the satellite to learn from data on board would help to overcome the issues with the inherent delays between acquisition and action. Vt’s work is an intriguing proof-of-principle.’
