The Semantic wedge: Freebase, Powerset and Twine

Danny Hillis (Metaweb), Barney Pell (Powerset) and Nova Spivack (Radar Networks) shared the stage for a session on the semantic Web at the Web 2.0 Summit.

Danny Hillis (Metaweb), Barney Pell (Powerset) and Nova Spivack (Radar Networks) shared the stage for a session on the semantic Web at the Web 2.0 Summit.

All three are developing services that bake more intelligence into the Web. And, the companies are all well funded, hyped and loaded with experience and talent. In fact, Hillis, Pell and Spivack have been part of the Silicon Valley AI community for the last dozen years. Danny Hillis was a founder of Thinking Machines, where Spivack worked for a period.

However, funding and talent are no guarantees of success, especially when plowing new ground with technology and product concepts that are on the edge of innovation, and have yet to proven useful to consumers.

Danny Hillis, Barney Pell, Nova Spivack, Tim O'Reilly

Each of the entrepreneurs demoed their products, and their different approaches to making the Web smarter.

Hillis showed applications built on Metaweb's Freebase. "It more valuable to operate across multiple systems, opening up silos of data and creating interconnections between them," Hillis said. The Freebase database allows users to add data to the system, filling out preset fields, and builds a "web" of connections. Freebase catalogs people, geographies, products, services, sports, media, technology and other topics, and has an open API for others to use the data. Hillis said Freebase was in "solid alpha" state.

Powerset is building a new search engine based on natural language technology, reading every word of every page to extract semantic meaning (I wrote about it here). Powerset has also imported Freebase data to improve its knowledge base. The company is signing up people for Powerlabs, which gives people access to the search engine, different interest groups, such as travel, and the capability to participate in idea and mashup competitions.

Spivack gave the first public preview of Twine (I posted about it here). Twine is about knowledge networking, rather than social networking, Spivack said. "The semantic graph is a superset of the social graph." Rather than just relationships, Twine makes connections between everything based on underlying semantic Web technologies. Twine is currently in the invite-only beta phase.

But the three companies are using various semantic techniques to extract meaning and entities from information sources. Twine and Powerset are leaning more on machine learning to extract semantic structure and Freebase relies on people to provide the metadata. But all three are applying some degree of user-contributed content--user-generated metadata--not just robots crawling the Web.

Freebase is more about a public database on the Web and Twine is more about an individual's data," Lew Tucker, CTO of Radar Networks and a Thinking Machines alumni, told me. "The underlying semantic Web technologies are similar, but we have different implementations. But, we are facing the same technical issues--how to handle the data because traditional databases don't handle what we are doing well and the access patterns, dealing with data that is interconnected and linked. We are in an early stage of figuring out how to optimize the systems based on usage patterns. The exciting part is that we are trying to capture meaning, information in context."

Hillis preferred not to characterize Freebase as a semantic Web application, to distinguish it from the semantic Web standards work that have been underway for several years by the W3C.

Spivack didn't shy away from associating Twine with semantic Web standards. "Open, semantic Web standards are important to get the network effect. Otherwise you end up with different platforms," Spivak said. "With open semantic Web standards, the data can become portable, whether in Freebase, Twine or Powerset," he said. It's the "hyperdata....the Web is the platform, the Web is the database...we are different services in the platform."

Hillis seemed to agree. "I think there will eventually be one Web of data, with all the data."

The three companies represent more than semantic technologies lurking at the edge of the Web, waiting to merge into the core of the Internet--it's more like a semantic wedge pushing the Web forward to radically improve the experience and utility of the Web.