Supernova 2007: The Internet's black hole

Supernova 2007: The Internet's black hole

Summary: KC Claffy spends her days trying to figure out what is really going on inside the Internet. She is a researcher with the Cooperative Association for Internet Data Analysis (CAIDA), which provides tools and analysis in service of making Internet infrastructure more robust and scalable.

SHARE:
6

KC Claffy spends her days trying to figure out what is really going on inside the Internet. She is a researcher with the Cooperative Association for Internet Data Analysis (CAIDA), which provides tools and analysis in service of making Internet infrastructure more robust and scalable.

ciaffy.jpg

The only problem, according to Claffy, is that the data required to study the live, operational Internet is not available. The Interent is a black hole for researchers, with the data in the hands of incumbents--telcos, ISPs, etc.--who have economic incentives to keep it out of the hands of researchers, she said. Nor does she give the software security vendors dealing in anti-virus and anti-spam product the benefit of the doubt.

"The research community has its hands tied because we don't have access to real data to study. The problem is getting access to the atoms you want to measure, so you can't figure out the real cost of infrastructure.," she said. "We need the numbers on spam, but we get them from the anti-spam vendors who have an incentive to have them look a certain ways. It's the same [problem] for viruses, the cost of broadband and file sharing."

She alleged that Hollywood, for example, uses studies that haven't been peer reviewed or properly sourced and lack a published methodology to make claims about piracy and file sharing usage. And, based on research with available data, CAIDA concluded that the incumbent players hide the fact that they lose money on non-IP revenue to subsidize other services, such as voice, and spend money on lobbying efforts rather than addressing major problems.

"There is no culture of getting and curating of Internet data, and the data is kept hidden from researchers under the rubric of privacy and data sharing," Claffy added.

In a recent published report, Claffy cited 16 problems that would be more easily solved if researchers had the data:

security • authentication • spam • scalable configuration management • robust scalability of routing system • compromise of e2e principle • dumb network • measurement • patch management • “normal accidents” • growth trends in traffic and user expectations • time management and prioritization of tasks • stewardship vs governance • intellectual property and digital rights • interdomain qos/emergency services • inter-provider vendor/business coordination broader impact Red: persistently unsolved problems for 10+ years

"The problems on the list are not about technology--they are are rooted in economics, ownership, and trust." She recommended that the FCC offer up a part of spectrum and donate it to the commons so that researchers could see what is going on in the public network. The crowd at Supernova 2007 cheered.

Topics: Browser, Data Centers

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

6 comments
Log in or register to join the discussion
  • Excellent Analysis

    Make a lot of sense to me but trying to change the status will require a tremendous public push from a largely uneducated (in this area) public.
    Wayne Reid
  • lol

    "...data in the hands of people who have economic incentives to keep the information out of the mainstream"

    Don't forget the oodles of false data, intentionally injected into the mainstream!

    :o)
    Jack-Booted EULA
    • Oodles of false data? You mean...

      the propaganda from the likes of D'uh'bya, Dick (Head) Cheney, and Senior Gonzalez to justify NSAT&T rerouting our net traffic to themselves and sniffing it like cocaine addicts, all in the name of "national security." No reason to let the TRUTH escape to the mainstream, when a lie in the name of "national security" will suffice.

      Not to mention Google, whose sole purpose of existence is to control all the net's data, including our private info, and make the net one big revenue generating adbot that pours money into their pockets. Yeah, no economic incentive there. :p
      Mr. Roboto
      • It's nice to read a ...

        ...reasoned, articulate comment. Glad to see you aren't overly political!
        handydan918
  • I've said it before...we do not

    have an Internet as it was invented. Instead of many small networks we now have a few large networks.

    Until the Internet becomes open and not at the mercy of a few large corporations, it will never live up to its potential.

    The truth is that the Internet is unmanagable by humans. Greed will take hold of whoever or whatever is in control and the Internet will suffer.
    bjbrock
  • Internet shortcomings

    It's a bit like democracy guys. It's NOT perfect, just a long way ahead of the alternatives.

    BobJ
    plumnilly