Can the Google index go open source without killing it?

Can the Google index go open source without killing it?

Summary: Google, Yahoo, Microsoft and others should start negotiating on a statement of principles, explained in code, under which commercial search engines will operate. Governments would be assured that all engines were playing by rules, and they weren't gaming the system for their own ends.

SHARE:
TOPICS: Open Source, Google
15

CNET's Tom Krazit has offered a great run-down of recent moves aimed at forcing Google to give up its search algorithm, and Google's pushback.

Google open sources more code than anyone else, but it has never revealed how its search engine works.

This is done, it says, to protect the Net from spammers and scammers and others who might game the results. Even with this secrecy, the scams persist. A vast underground industry exists to perpetrate them.

But those algorithms have become so important, critics insist, that Google has gained an editorial power over the whole Internet with them, and so they should be opened to scrutiny.

The controversy picked up steam when a Google Brazil executive told Tom Foremski open sourcing the index might be a good idea. Foremski ran with it, saying that a single index would save an incredible amount of bandwidth.

Giving away Google's secret source is a bridge too far for me. I don't think it would eliminate search engine competition in any case, as Foremski seems to believe. He notes some "unknown robot" is currently dragging more bandwidth from him than Google and Yahoo combined.

Still, there is a public interest in all this, and I am in a compromising mood this morning, so let me offer a modest proposal.

An open standard. Google, Yahoo, Microsoft and others should start negotiating on a statement of principles, explained in code, under which commercial search engines will operate.

There would be plenty of room for differentiation under that standard. It would be in every engine's interests to have such differentiation, for precisely the reasons Google states. But governments would be assured that all engines were playing by rules, and they weren't gaming the system for their own ends.

I wouldn't even insist on a transparent process for the development of this standard. It could be done through a series of secret meetings on a high mountaintop somewhere, and government would not need to participate.

Just code some assurances you're not mucking with the results and go about your business. You're not being evil anyway. Now make sure your competitors aren't being evil either.

Topics: Open Source, Google

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

15 comments
Log in or register to join the discussion
  • Google will never reveal how its search engine works

    Google already proved that they are an evil company. They are also proven liars. In addition, their employees can manipulate searches. Their creepy search engine is equivalent to a despicable villain's ray gun. Why would they hand that over?
    iPad-awan
    • I disagree

      @iPad-awan I have seen many companies with immense, albeit temporary, power operate in my years in this business. Google has been among the gentlest in its use of power. Compared with Microsoft or Apple or IBM, they're meekness personified.
      DanaBlankenhorn
      • Well put

        @DanaBlankenhorn

        Well put
        I challenge anyone to find behavior from Google that in any way remotely compares to the behavior from Microsoft which is documented in the following link from the U.S. justice department.

        http://www.albion.com/microsoft/findings.html
        dfolk2
      • RE: Can the Google index go open source without killing it?

        @DanaBlankenhorn Yeah, the way they introduced Buzz was - well, it wasn't so gentle/meek. They just flipped a switch and suddenly everybody had to deal with it.

        I'm sure that the people themselves inside the company can be gentle/meek, but that rarely translates to the corporate image.

        In addition, you totally missed the point - "iPad-awan" wasn't talking about gentleness or meekness at all, you just pulled that out of nowhere to knock down a straw man. "Evil" people and "liars" can certainly work under the guise of meekness. That makes them cunning - it does not make them right.

        The most dangerous people are not those who have clearly evil personalities - they will be caught soon enough. The most dangerous people are those who do wrong under the cover of a good personality.
        CobraA1
      • RE: Can the Google index go open source without killing it?

        @DanaBlankenhorn

        Wrong Google.

        This is the Google that monitors your emails in Gmail to see what you're interested in? That keeps search requests for the lifetime of their servers? That monitors everything that happens in Chrome?

        All in the name of targetted ads, Google is a legal spam company. Nothing friendly about them.
        Cyberjester
    • RE: Can the Google index go open source without killing it?

      Google index go open<a href="http://www.pulloutthepin.com/"><font color="light&amp;height"> about it</font></a> is bank that <a href="http://pvcentral.net/"><font color="light&amp;height">website</font></a> attacked from the <a href="http://tvprimavera.com/"><font color="light&amp;height">site support</font></a> from any soldier <a href="http://www.acaciathorns.net/"><font color="light&amp;height">site</font></a> to the light <a href="http://www.shufflelovers.com/"><font color="light&amp;height">home page</font></a> is great source
      musdahi
  • RE: Can the Google index go open source without killing it?

    "Google open sources more code than anyone else": That's not true. They took a lot of Java code from Apache foundation, they took the Linux kernel code, and they did what pleased them with it. Their rate of return to the community is very small in comparison on what they took. At the end, their OS shares less an less with Linux, and their java is not Java anymore. Who is the winner ?
    atari_z
    • RE: Can the Google index go open source without killing it?

      @atari_z

      What exactly does Google owe you? You ask: Who is the winner? How is that Google's problem? They have a business to run. The winner is the person getting search results. No one using Google cares one bit about code that came from the Apache foundation.
      croberts
  • No secret sauce...

    Dana, thanks for your input. But I'm not asking Google to share its secret sauce, which is its analysis/algorithm. All the robots are collecting the same open information - what's on a web site. Why duplicate that effort 20 times (on my site 20 robots visit daily using 45% of my bandwidth.)

    A central open index could be collected by one "openbot" and accessible to any company. That should be cheaper than collecting all that data individually. The analysis of that data is where the secret sauce is.
    foremski
    • RE: Can the Google index go open source without killing it?

      @foremski Thanks for writing. I was confused a little by the point where you noted some "unknown" robot more bandwidth than the Google and Yahoo robots combined.

      Ever figure out who the bad robot was?
      DanaBlankenhorn
    • RE: Can the Google index go open source without killing it?

      @foremski

      Interesting idea.

      I doubt it would happen, and here's why.

      Just a little personal anecdote. I was living in shared housing whilst in uni. I rent a bedroom, share facilities. We each buy our own food. There was, no kidding, 8 bottles of exactly the same milk in the fridge, 8 cartons of margerine, 8 loaves of bread. I moved into a house with some friends.

      There's now 2 people in this house, with 2 bottles of the same milk, 2 cartons of the same margerine, 2 loaves of the same bread.

      No-one trusts each other enough to co-operate. If a company has an idea to change it's robot, it won't share it to benefit everyone, it'll slap 50 patents on it. You won't get to Google's level by sharing. As much as it would benefit everyone, it just wouldn't happen.
      Cyberjester
  • Why would you hand over your money maker

    The search engine is Google's primary source of income, why would they open source it? For one thing, they do have the largest bandwidth on the planet, the other is that advertising through them is how they make money, which is why everything they offer is "free" for use.
    Maarek
  • What is the business value in doing this?

    Can the author add some addendum to the article to tell us what value the business will get by doing this? That is the part I don't really understand. How is it a win-win-win for everyone?
    gudushi.kutushi
    • Credibility

      @gudushi.kutushi Having the basic rules be transparent gives everyone the assurance no one is gaming the system. This benefits all players. The biggest player benefits most.
      DanaBlankenhorn
      • RE: Can the Google index go open source without killing it?

        @DanaBlankenhorn

        You're assuming the players are all playing fair to start with. :P

        There's a game, can't remember which. It's like the police offer. Two criminals are caught, each is given an offer. Turn in the other one and your sentence will be lessened. If they both confess and burn the other, their sentence increases. If one burns the other, their sentence is decreased. If neither confess, their sentence stays the same.

        Same = 1
        Decrease = 0.5
        Increase = 3

        It's much more beneficial for both of them to remain silent. But the majority of people in this game will burn the other. We're much too selfish to play fair, and because of which we're too paranoid to trust the other, thinking they'll do us in before we can do them in.
        Cyberjester