Intelligent microscopes use advanced imaging techniques and computer algorithms to analyze and detect rare biological events. These events can include cellular processes, changes in gene expression, and the presence of specific biomolecules. Examples of intelligent microscope technologies include super-resolution microscopy, fluorescence lifetime imaging, and single-molecule imaging. These tools have the potential to greatly enhance our understanding of biological systems and improve the diagnosis and treatment of diseases.
Control software developed by biophysicists optimizes how fluorescence microscopes collect data on living samples. Their control loop, which was used to image mitochondrial and bacterial division sites in great detail, has been released as an open-source plug-in and could inspire a new generation of intelligent microscopes.
Assume you’re a Ph.D. student with a fluorescent microscope and a live bacteria sample. How should these resources be used to obtain detailed observations of bacterial division from the sample?
You may be tempted to forgo food and rest, to sit at the microscope non-stop and acquire images when bacterial finally division starts. (It can take hours for one bacterium to divide!) It’s not as crazy as it sounds, since manual detection and acquisition control is widespread in many of the sciences.
An intelligent microscope is kind of like a self-driving car. It needs to process certain types of information, subtle patterns that it then responds to by changing its behavior. By using a neural network, we can detect much more subtle events and use them to drive changes in acquisition speed.
Suliana Manley
Alternatively, you could program the microscope to take images in any order and as frequently as possible. Excessive light, on the other hand, depletes the fluorescence of the sample faster and can prematurely destroy living samples. Furthermore, you’d generate a lot of uninteresting images because only a few would contain images of dividing bacteria.
Another approach would be to use artificial intelligence to detect bacterial division precursors and use these to automatically update the microscope’s control software to capture more images of the event.
Drum roll… yes, EPFL biophysicists have indeed found a way to automate microscope control for imaging biological events in detail while limiting stress on the sample, all with the help of artificial neural networks. Their technique works for bacterial cell division, and for mitochondrial division. The details of their intelligent microscope are described in Nature Methods.
“An intelligent microscope is kind of like a self-driving car. It needs to process certain types of information, subtle patterns that it then responds to by changing its behavior,” explains principal investigator Suliana Manley of EPFL’s Laboratory of Experimental Biophysics. “By using a neural network, we can detect much more subtle events and use them to drive changes in acquisition speed.”
Manley and her colleagues first figured out how to detect mitochondrial division, which was more difficult than it was for bacteria like C. crescentus. Mitochondrial division is unpredictable because it occurs infrequently and can occur at any time within the mitochondrial network. However, scientists solved the problem by training the neural network to detect mitochondrial constrictions, a change in the shape of mitochondria that leads to division, in conjunction with observations of a protein known to be enriched at division sites.
When both constrictions and protein levels are high, the microscope switches to high-speed imaging to capture a large number of detailed images of division events. To avoid exposing the sample to too much light, the microscope switches to low-speed imaging when constriction and protein levels are low.
The scientists demonstrated that by using this intelligent fluorescent microscope, they could observe the sample for longer periods of time than standard fast imaging. Despite the fact that the sample was more stressed than in standard slow imaging, they were able to obtain more meaningful data.
“The potential of intelligent microscopy includes measuring what conventional acquisitions would miss,” explains Manley. “We can track each division in greater detail because we capture more events, measure smaller constrictions, and measure smaller constrictions.”
The control framework will be made available as an open-source plug-in for the open microscope software Micro-Manager, allowing other scientists to integrate artificial intelligence into their own microscopes.
