Hakia adds PubMed, enhances medical search

"For the first time, users will start to get credibility-stamped search results in full view, instead of just popularity-ranked results.

Health related searches on the Web are always important, and can even be a matter of life and death. More than 100 million people search online for health-related information. Not surprisingly [according to a 2005 Forrester Research report], a staggering 30 percent of searchers have concerns about the quality of the information."

Hakia today strengthened their offering in the health domain, adding more than 10 million abstracts from PubMed to their index and setting up a new site dedicated to searching this content at pubmed.hakia.com. The PubMed content will also be visible in search results on the hakia medical search site and via the main hakia search page.

PubMed includes a freely available search engine, funded by the National Library of Medicine and National Institutes of Health in the United States, and provides access to over 17 million citations and abstracts from scholarly articles in medicine and related fields. The full text of articles described in PubMed is not always available without an expensive subscription to the journal in which a particular article appears... or a visit to your local library.

Despite being freely searchable elsewhere, Dr. John Boockvar, Alvina and Willis Murphy Assistant Professor of Neurological Surgery at Weill Cornell Medical College notes that;

"hakia's PubMed search can bring results that you didn't know existed. Hakia's addition of PubMed articles is the right direction toward more efficient search of medical information, which is critical to timely progress in medical research."

In a simple test, my search for "german measles" on PubMed itself retrieved 183 hits. Hakia pulled back just 17. There could be various reasons for that, including the fact that hakia currently indexes a subset of the content that PubMed itself can reach. Behind-the-scenes hakia magic may even have decided that 166 of the articles were irrelevant... although that seems unlikely. A decision as to which interface best displays the results probably comes down to personal preference, and the nature of the task being performed. For me hakia came out just ahead on appearance, and I can't really comment as to which results were better. The inflexible way in which most journal content is corralled and guarded by mainstream publishers is certainly not hakia's fault, but being limited to working with the stilted text of a short abstract must surely diminish the company's ability to extract significant meaning. It would be interesting to see the extent to which the results changed if hakia were able to read the full text of articles such as the freely available ones in PubMed Central.

Engines such as hakia are rarely optimised for the simple keyword search, and tend to excel when given richer queries. Asking "can HIV be cured?" should better put hakia through its paces, and returns a set of results more relevant and more intelligibly (to me, at least) related to the question than PubMed's 327 hits. The text snippets and highlighting employed by hakia also make it easier to work out whether or not to look at any of the results in more detail.

Talking to hakia's Chief Scientific Officer, Dr. Christian F. Hempelmann, at the Semantic Technology Conference last month, he was keen to stress the effort invested in ensuring that hakia understood queries and the documents being searched. Unlike competitors in the market that Hempelmann characterised as taking a statistical and computationally intensive approach to 'understanding' text, hakia employs the same linguistic cues as humans in interpreting the meaning behind a query or the words of a document. Hempelmann suggested that the more brute-force approaches will continue to face scalability issues, whilst asserting that there is 'no ceiling' to hakia's potential growth. Hempelmann himself has been at hakia for two years, building upon a research background in semantic linguistics and (strange but true!) the little-known field of humour research. Talking to him, it is easy to see an evolution from understanding how humour works and how we construct and interpret metaphor to teaching machines to interpret language.

Quoted in the hakia press release, CEO Dr Riza C. Berkan commented,

"PubMed documents, like many other credible databases, represent a unique challenge for search engines like Google, because popularity is simply not the right criteria for retrieving medical information. Hakia's semantic search technology is devoid of all such limitations that come with statistical methods. Instead, hakia’s algorithms search for the best contextual matches of the search query by using medical ontologies."

Medicine is often cited as one of those disciplines in which effective interpretation of professional ontologies will be important in increasing access to information. End user terms such as 'heart attack,' it is claimed, rarely appear in medical literature that limits itself to more precise terminology. As such, I was keen to see how hakia "used medical ontologies" to "search for the best contextual matches."

Unfortunately, every result in a heart attack search included the term 'heart attack' somewhere in the abstract. Are medical researchers beginning to use the same language as the rest of us, or did hakia miss - or ignore - all those papers discussing myocardial infarctions?

Hakia is intriguing and clearly has potential. I am left wondering, though, if this particular data set was a good one to show off its capabilities.