I spent part of the afternoon at one of Yahoo Lab's facilities, a nondescript building on University Ave. in Berkeley, CA., for a press luncheon with the company’s research wonks. Prior to the presentations I talked to Usama Fayyad, Yahoo’s data chief and senior vice president who sits above the labs in the org chart, about staffing Yahoo Labs, which is currently about 60 people. “We have unrestricted head count for our labs. The issue is finding talent,” Fayyad said. Welcome to the search and social media talent arms race.
Fayyad told me that Yahoo Labs is about announce some major additions, what he called “big surprises” (this is change of direction based on my interview in August with newly appointed lab director Prabhakar Raghavan, who said that Yahoo would stay away from high-profile hires). Fayyad said that Yahoo will hire either well known, accomplished researchers and professors or the most talented students, and nothing in between. “We will maintain that standard—and we have no pressure to grow faster,” he added. I would guess a big surprise would be someone possibly from rival Google.
Marc Davis, who heads up Yahoo’s Berkeley Lab (a collaboration with the University of California, Berkeley), described his charter as enabling billions of consumers to become producers and to reshape culture. A professor in the School of Information Management and on leave from UC Berkeley to run the facility, Davis also wants to revolutionize the research space. Instead of 60 or 70 people in a study, he can tap into Yahoo’s 400 million users and 10 terabytes of data (with proper permission) that is spawned every day. “We are focused on socio-technical design. We have to think about how technology, media and people come together,” Davis said. Video, photos, audio, messaging and even tagging are becoming a part of the daily experience, and Davis said research will focus on how content connects with community and context.
“No one understands the phenomenon with millions of people interacting online, trust, rating systems that can’t be scammed and spam,” Fayyad told me. “We want to understand what makes a community work, how do you measure what is working, what makes people stay or break apart in a community on a large scale. We are building a new science, a mixture of math and philosophy. It’s not going to be a fast process.” He doesn’t plan to hire any philosophers, but the best researchers are partly philosophers.
I'm not sure if it's a new science. Cynically, it's the science of getting people to use more of the services and stay as a loyal member of the Yahoo tribe and extended community as the online world evolves. But scientists do tend to be less business and more philosophically oriented, and want to create a better world, bridging the digital divide and inventing new models for richer online interaction across all kinds of boundaries. Yahoo says that the research will be made public and others can gain insight into how the virtual world will take shape, and in the spirit of collaboration develop new models for communities with dozens or millions of members. And, just like the real world, mutiple models for online communities will co-exist.
It will be interesting to find out if the social dynamics of a small village in the Himalayas or even ant colonies is that much different from millions of people interacting within online communities. I sense that the notion of "human scale" will be debated and tested as collective input and output is harnessed and ad hoc communities dynamically coalese and break apart. Developing the tools and environments for large scale online communities, however, will be like transitioning from telegraph and to the cell phone. It will take time and lots of experimentation by many players.
Raghavan spoke at the luncheon, reprising what I wrote in the interview about research areas (information retrieval, computational linguistics, machine learning, matrix and graph algorithms, unsupervised clustering, data mining and related areas), turning research into products and engaging with the academic community. He also spoke about a focus on microeconomics. “We have the opportunity to address a billion consumers who are producing and consuming content. We want to figure out how to get in midst of all that and connect the two groups and provide them with incentives,” Raghavan said.
Jeff Weiner,, senior vice president for search and marketplace at Yahoo, categorized the incentives as personal (offering utilities for people, such as desktop search), social (blogging, for example, to share info and get recognition from the community) and economic (getting paid). Yahoo is clearly looking at incentives as a way to gain deeper engagement with its users and monetize far more than search pages, as Yahoo CEO Terry Semel implied during his Web 2.0 interview.
Weiner went on about democratizing information, pointing out that search today is not very democratic. Results are delivered by link structures isn't even representative democracy, he said. By his calculations, about three objects per person are available online (20 billion objects, 7 billion people). He is suggesting that in the future each individual will have thousands, if not tens of thousands of objects digitized and capable of being "fused" into the online world. That's for sure, and it's the kind of problem that researchers love to work on, and they will be for decades.
Weiner talks about finding, using, sharing and expanding all human knowledge as Yahoo's goal, and that the company "will amass the infrastructure and talent, and the only limit is our imagination." Despite the unbridled evangelistic fervor, Yahoo is on to something (it's about people) and is systematically moving on multiple fronts. All of the major portal players have similar aspirations--to make a better online world and bring billions into their tribes. Let's hope the tribes don't go to war with each other...