5 of 6Image
Foreign languages and characters can be translated into the user's language, so a user can find out what an object is. Search results can be personalised, so the user can be alerted if a foodstuff contains a certain allergen, for example.
The phone takes a video of the object at 10 frames per second, and the images are sent to a database in real-time using "video calling", a low-latency communications medium.
The database that is used for video search can be built automatically; Accenture has written spiders to crawl the web and download images on a specific theme such as Asian food.
Linaker, pictured above, explained that to pinpoint the features necessary to identify an object, the image is run through an algorithm called Scale-Invariant Feature Transform, or SIFT, a technology developed by academic David Lowe.
The software extracts feature points from a jpeg, according to Linaker, and makes a match against images in the database at the full frame rate of the camera, which is 10 frames per second. If a match exists then the software on the server retrieves information and sends it back to the user's phone.
The advantage with video is that if you have a "bad angle" you have another image for comparison supplied "a few milliseconds later", according to Linaker.
The researchers are currently in the process of adding features to the Pocket Supercomputer. Linaker said he would like to add optical character recognition, because once the system can recognise text, it can be linked to the internet.
Character searches online can be combined with video searches of sites such as YouTube, Linaker added.
Accenture is also studying how to add speech-recognition software to the system. This would enable the user to speak a word or phrase into the phone in one language, and have the phone "speak" back that word or sentence in a different language. Video and audio can also be synchronised, so a user could speak the name of a subject or object into the phone, and receive a video containing relevant information back.
The researchers also plan to study how a user could do a video search that returns images of objects that have similar features, rather than the current search, which returns images of objects with exactly the same features. For example, a user could video a certain type of CPU, and in turn receive information not only about that particular CPU, but also CPUs that have a similar specifications
Researchers hope to have a product ready for the market within three years.