Can computers really read manuscripts?
Using computers to read handwritten symbols has been done before. But now, researchers in Barcelona, Spain, have developed a system more efficient and reliable than currently existing ones. Their BSM system -- an acronym for 'Blurred Shape Model' -- has been designed to work with ancient, damaged or difficult to read manuscripts, handwritten scores and architectural drawings. And apparently, it works. It has been able to recognize musical notes in handwritten partitions with an exactness of over 98%.
You can see on the left some classes of handwritten symbols recognized by this new software (Credit: Universitat Autonoma de Barcelona). Surprisingly, this diagram comes from a document released by the University on July 18, 2007, and which is only now released worldwide. This project was led by Josep Lladós, an Associate Professor at the Computer Sciences Department of the Universitat Autònoma de Barcelona and a staff researcher of the Computer Vision Center.
So how this BSM system works? "The BSM differs from other existing systems which follow the same process when deciphering different types of symbols, since a standard process makes it more difficult to recognise the symbols after they have been introduced. In contrast, the methodology developed by the Computer Vision Centre can be adapted to each of the areas it is applied to. To be able to analyse and recognise symbols, the system divides image regions into sub regions - with the help of a grid - and saves the information from each grid square, while registering even the smallest of differences (e.g. between p and b). Depending on the shape introduced, the system undergoes a process to distinguish the shape and also any possible deformations (the letter P for example would be registered as being rounder or having a shorter or longer stem, etc.). It then stores this information and classifies it automatically."
And is this BSM system accurate? Apparently yes, according to the researchers. "Researchers decided to test the efficiency of the system by experimenting with two application areas. They created a database of musical notes and a database of architectural symbols. The first was created from a collection of modern and ancient musical scores (from the 18th and 19th centuries) from the archives of the Barcelona Seminary, which included a total of 2,128 examples of three types of musical notes drawn by 24 different people. The second database included 2,762 examples of handwritten architectural symbols belonging to 14 different groups. Each group contained approximately 200 types of symbols drawn by 13 different people. In order to compare the performance and reliability of the BSM, the same data was introduced into other similar systems. The BSM was capable of recognising musical notes with an exactness of over 98% and architectural symbols with an exactness of 90%."
This research work received the price for the best paper at the Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA 2007) which took place in Girona, Spain, on June 6-8, 2007. The title of the paper is "Handwritten Symbol Recognition by a Boosted Blurred Shape Model with Error Correction." It has been included in a book published by the Springer group, "Pattern Recognition and Image Analysis." Here is a link to the abstract of this paper (available for US$ 25.00).
This paper will also be presented at the 9th International Conference on Document Analysis and Recognition (ICDAR), sponsored by the International Association for Pattern Recognition (IAPR), which will be held in Curitiba, Brazil, on September 23-26, 2007. The lead author of the paper, Josep Lladós, will receive the IAPR/ICDAR Young Investigator Award for his innovative research in graphics recognition. Here are some research projects where Lladós is also involved.
- DocMining: A Framework of Intelligent Management of Document Contents
- Recognition of newborn fingerprints
- PVPC: Virtual Prototyping for Architectural Projects
Sources: Universitat Autonoma de Barcelona news release, via EurekAlert!, September 6, 2007; and various websites
You'll find related stories by following the links below.