To improve the capabilities of deep learning transcription, researchers at the University of Notre Dame are developing an artificial neural network based on human perception to read complex ancient handwriting.
The Abbey Library of St. Gall in Switzerland houses approximately 160,000 volumes of literary and historical manuscripts dating back to the eighth century, all of which are written by hand, on parchment, in languages rarely spoken in modern times.
Such texts, numbering in the millions, have been safely stored away in libraries and monasteries all over the world to preserve these historical accounts of humanity. A significant portion of these collections are accessible to the general public via digital imagery, but experts say there is an extraordinary amount of material that has never been read – a treasure trove of insight into world history hidden within.
Researchers are developing an artificial neural network to read complex ancient handwriting based on human perception to improve the capabilities of deep learning transcription.
“We’re dealing with historical documents written in styles that have long fallen out of favor, dating back many centuries, and in languages like Latin that are rarely ever used anymore,” said Walter Scheirer, the Dennis O. Doughty Collegiate Associate Professor in Notre Dame’s Department of Computer Science and Engineering. “You can get beautiful photos of these materials, but our goal is to automate transcription in a way that mimics the perception of the page through the eyes of an expert reader and provides a quick, searchable reading of the text.”
In a paper published in the journal Transactions on Pattern Analysis and Machine Intelligence of the Institute of Electrical and Electronics Engineers, Scheirer describes how his team combined traditional machine learning methods with visual psychophysics — a method of measuring the connections between physical stimuli and mental phenomena such as the amount of time it takes an expert reader to read a sentence.
Scheirer’s team examined digitized Latin manuscripts written in the ninth century by scribes in the Cloister of St. Gall. Readers manually transcribed their work into a specially designed software interface. The team then measured reaction times during transcription to determine which words, characters, and passages were easy or difficult to understand. According to Scheirer, including this type of data resulted in a network that was more consistent with human behavior, reduced errors, and provided a more accurate, realistic reading of the text.
“It’s a strategy that isn’t commonly used in machine learning,” Scheirer explained. “We’re labeling the data using psychophysical measurements derived directly from psychological studies of perception – by taking behavioral measurements. We then notify the network of any common difficulties in perceiving these characters and can make corrections based on the measurements.”
Humanities scholars are very interested in using deep learning to transcribe ancient texts. “There’s a difference between just taking photos and reading them and having a program that provides a searchable reading,” said Hildegund Müller, an associate professor in Notre Dame’s Department of Classics. “When you consider the texts used in this study – ninth-century manuscripts – you’re looking at a very early stage of the Middle Ages. It’s a long time before the printing press was invented. At the time, an enormous number of manuscripts were produced. There is a wealth of information hidden in these manuscripts, including unidentified texts that have never been seen before.”
According to Scheirer, challenges remain. His team is working to improve transcription accuracy, particularly in the case of damaged or incomplete documents, as well as how to account for illustrations or other aspects of a page that may be confusing to the network.
The team was able to modify the program to transcribe Ethiopian texts, adapting it to a language with a completely different set of characters – a first step toward developing a program that can transcribe and translate information for users.
“It could be very useful in the literary field.” Every good literary work is surrounded by a plethora of historical documents, but where it will be most useful is in historical archival research,” Müller explained. “Advancing the digital humanities is critical.” When discussing the Middle Ages and early modern times, if you want to understand the details and consequences of historical events, you must consult written material, and these texts are the only ones we have.
Outside of the Western world, the problem may be even worse. Consider the disappearance of languages in threatened cultures. We must first preserve these works, make them accessible, and eventually incorporate translations to make them a part of ongoing cultural processes – and we are racing against time.”
