An afternoon at Yahoo Labs
Fayyad told me that Yahoo Labs is about announce some major additions, what he called “big surprises” (this is change of direction based on my interview in August with newly appointed lab director Prabhakar Raghavan, who said that Yahoo would stay away from high-profile hires). Fayyad said that Yahoo will hire either well known, accomplished researchers and professors or the most talented students, and nothing in between. “We will maintain that standard—and we have no pressure to grow faster,” he added. I would guess a big surprise would be someone possibly from rival Google.
Marc Davis, who heads up Yahoo’s Berkeley Lab (a collaboration with the University of California, Berkeley), described his charter as enabling billions of consumers to become producers and to reshape culture. A professor in the School of Information Management and on leave from UC Berkeley to run the facility, Davis also wants to revolutionize the research space. Instead of 60 or 70 people in a study, he can tap into Yahoo’s 400 million users and 10 terabytes of data (with proper permission) that is spawned every day. “We are focused on socio-technical design. We have to think about how technology, media and people come together,” Davis said. Video, photos, audio, messaging and even tagging are becoming a part of the daily experience, and Davis said research will focus on how content connects with community and context.
“No one understands the phenomenon with millions of people interacting online, trust, rating systems that can’t be scammed and spam,” Fayyad told me. “We want to understand what makes a community work, how do you measure what is working, what makes people stay or break apart in a community on a large scale. We are building a new science, a mixture of math and philosophy. It’s not going to be a fast process.” He doesn’t plan to hire any philosophers, but the best researchers are partly philosophers.
I'm not sure if it's a new science. Cynically, it's the science of getting people to use more of the services and stay as a loyal member of the Yahoo tribe and extended community as the online world evolves. But scientists do tend to be less business and more philosophically oriented, and want to create a better world, bridging the digital divide and inventing new models for richer online interaction across all kinds of boundaries. Yahoo says that the research will be made public and others can gain insight into how the virtual world will take shape, and in the spirit of collaboration develop new models for communities with dozens or millions of members. And, just like the real world, mutiple models for online communities will co-exist.
It will be interesting to find out if the social dynamics of a small village in the Himalayas or even ant colonies is that much different from millions of people interacting within online communities. I sense that the notion of "human scale" will be debated and tested as collective input and output is harnessed and ad hoc communities dynamically coalese and break apart. Developing the tools and environments for large scale online communities, however, will be like transitioning from telegraph and to the cell phone. It will take time and lots of experimentation by many players.
Raghavan spoke at the luncheon, reprising what I wrote in the interview about research areas (information retrieval, computational linguistics, machine learning, matrix and graph algorithms, unsupervised clustering, data mining and related areas), turning research into products and engaging with the academic community. He also spoke about a focus on microeconomics. “We have the opportunity to address a billion consumers who are producing and consuming content. We want to figure out how to get in midst of all that and connect the two groups and provide them with incentives,” Raghavan said.Jeff Weiner,, senior vice president for search and marketplace at Yahoo, categorized the incentives as personal (offering utilities for people, such as desktop search), social (blogging, for example, to share info and get recognition from the community) and economic (getting paid). Yahoo is clearly looking at incentives as a way to gain deeper engagement with its users and monetize far more than search pages, as Yahoo CEO Terry Semel implied during his Web 2.0 interview.
Weiner went on about democratizing information, pointing out that search today is not very democratic. Results are delivered by link structures isn't even representative democracy, he said. By his calculations, about three objects per person are available online (20 billion objects, 7 billion people). He is suggesting that in the future each individual will have thousands, if not tens of thousands of objects digitized and capable of being "fused" into the online world. That's for sure, and it's the kind of problem that researchers love to work on, and they will be for decades.