Participants in neuropsychological assessments complete tasks designed to study memory and thinking. Participants receive a score based on their performance, which researchers use to assess how well specific domains of their cognition are functioning.
Consider two people who both score the same on one of these paper-and-pencil neuropsychological tests. One completed the task in 60 seconds and spent the entire time writing; the other took three minutes and alternated between writing answers and staring off into space. Would researchers be missing something important if they only looked at the overall score of these two participants?
“By focusing solely on the outcome, or what score someone receives, we miss out on a lot of important information about how the person performed the task that could help us better understand the underlying problem,” says lead author Stacy Andersen, Ph.D., assistant professor of medicine at Boston University School of Medicine (BUSM).
During neuropsychological assessments, participants complete tasks designed to study memory and thinking. Based on their performance, the participants receive a score that researchers use to evaluate how well specific domains of their cognition are functioning.
The Long Life Family Study (LLFS) researchers used digital pens and digital voice recorders to capture differences in study participants’ performance while completing a cognitive test and discovered that differences in ‘thinking’ versus ‘writing’ time on a symbol coding test could act as clinically relevant, early biomarkers for cognitive/motor decline.
Participants in the LLFS were chosen because they had multiple siblings who lived to be very old. Longevity has long been associated with an increased health span, so these families are being studied to better understand the factors that contribute to healthy aging. The participants were evaluated on a variety of physical and cognitive measures, including the Digit Symbol Substitution Test, a symbol coding test.
This timed test requires participants to fill in numbered boxes with symbols from a given key and assesses both cognitive (attention and processing speed) and non-cognitive factors (motor speed and visual scanning). The participants used a digital pen to complete the test, allowing researchers to collect data on how they went about completing the task. A small camera was attached to the tip of this pen, which tracked what and when a participant wrote. The LLFS researchers divided the output of this digital pen into ‘writing time’ (the time the participant spent writing) and ‘thinking time’ (the time the participant did not spend writing) and examined how these changed over the course of the 90-second test.
The researchers then identified groups of participants who displayed similar patterns of writing and thinking time throughout the test. They discovered that, while the majority of participants had consistent writing and thinking times, there were groups of participants who sped up or slowed down.
“This method of clustering allowed us to look at other similarities in terms of health and function among the participants in each group that may be related to differences in writing and thinking time patterns,” said co-author and lead biostatistician Benjamin Sweigart, a biostatistics doctoral student at Boston University School of Public Health.
The researchers discovered that those who wrote the symbols slower during the test had a poorer physical function on grip strength and walking speed tests. Those who changed their thinking time speed had lower scores on memory and executive function tests, indicating that writing time and thinking time capture different contributors to overall test performance.
These findings, according to the researchers, highlight the importance of capturing aspects of test performance other than test scores. “Determining whether poor test performance is due to impaired cognitive function versus impaired motor function is critical for selecting the appropriate treatment for an individual patient,” Andersen adds. “The use of digital technologies improves our ability to detect subtle differences in test behavior and functional abilities, even on brief cognitive function tests. Furthermore, these metrics have the potential to be early indicators of dysfunction.”
