The magic that makes Google tick

The magic that makes Google tick

Summary: Google's vice-president of engineering was in London this week to talk to potential recruits about just what lies behind that search page. ZDNet UK snuck in to listen

TOPICS: Servers
The process
Obviously it would be impractical to run the algorithm once every page for every query, so Google splits the problem down.

When a query comes in to the system it is sent off to index servers, which contain an index of the Web. This index is a mapping of each word to each page that contains that word. For instance, the word 'Imperial' will point to a list of documents containing that word, and similarly for 'College'. For a search on 'Imperial College' Google does a Boolean 'AND' operation on the two words to get a list of what Hölzle calls 'word pages'.

"We also consider additional data, such as where in the page does the word occur: in the title, the footnote, is it in bold or not, and so on.

Each index server indexes only part of the Web, as the whole Web will not fit on a single machine - certainly not the type of machines that Google uses. Google's index of the Web is distributed across many machines, and the query gets sent to many of them - Google calls each on a shard (of the Web). Each one works on its part of the problem.

Google computes the top 1000 or so results, and those come back as document IDs rather than text. The next step is to use document servers, which contain a copy of the Web as crawled by Google's spiders. Again the Web is essentially chopped up so that each machine contains one part of the Web. When a match is found, it is sent to the ad server which matches the ads and produces the familiar results page.

Google's business model works because all this is done on cheap hardware, which allows it to run the service free-of-charge to users, and charge only for advertising.

Topic: Servers

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Hello
    One thing that ticks me off about google is that it desplays multiple results on a search query from the same web site.
    How hard can it be to limit the exposer to one url per query?
    Is it logical to have a website show up hundreds of times on a search query?
    Try searching for the word "algarve" (region in Portugal).
    I think it is googles mission to deliver me to a website where I can find the content I am looking for, and not to worry about holding me up to reach the cookie jar and sink it down my mouth.
    Show me where the supermarket is, I will find the shelf!
    Thank you.
    Keep clicking...

    Hilton Santos
  • Dude... don't know what you're talking about. Tried your search and got 10 results for 10 different websites listed. Google DOES eliminate results that are alike.
  • Hi Carlos
    Thank you for your feedback.
    Search for "algarve car hire"
    You will notice as I have (saved the results in the case you would like me to send them to you)
    that the same website:
    comes in at 2nd and 3rd position, if you continue down the list you will find it time and time again.
    Are the results relevant?
    Well... I guess one of the results is, all others are repeats, that point to the same web page...
  • Open letter to Google.

    Hello my dear Googlers

    My comment has nothing to do with google suggest, I find it very useful.
    Now what really squeezes the sh.. out of me is ONE THING and this is it:

    Why the hell do you allow a website to show up multiple times on one search query?

    Why GOOGLE? Why?

    It seams so simple to deal with, and trust me when I say, that I AM NOT A GENIUS!

    Is there not a way of eliminating, the multiple exposer of a given website by means of it

    "Now please excuse me if I sound dramatic, but, HOW HARD can it be to eliminate the following results after the first one has been shown?"

    typical question from someone who has no idea what they are asking for.

    HOW HARD can it be?
    have you tried doing this? Have you tried to put up thousands of computers and make a "slight" adjustment on a program running on the whole network?
    If you have not tried doing it, dont pretend to understand how it works. You have no idea what you are asking for. You can probably send some money to Google so they can have the resources to process your request... Its not like you actually paid Google to do the search for you right? You want a personalized search engine? Go make one yourself. It's relatively easy to do... well, that is, if you have the brain for it.
  • Very dear Frustrated Research and Development specialist

    I am sorry you feel this way...
    Why don
  • Oh Wilton, Alem de parvo es muito burro!!!!
    Epa why Google?? mesmo de puto.

  • And he calls himself development specialist ...
    Some guys are so stupid !!!
  • Sou puto mas tenho colh... para assinar o que escrevo...
  • Dear Hilton,

    First of all, please forgive me for the bad english.
    Good afternoon. I don't think very ethical from your part to criticize other's websites listed on google. If you want to criticize google then e-mail them with general matters. Never with particular cases. I think that might not be the best way to gain customers or positions on google. I know several websites, experts in many matters that only show up on page 14 on google and on yahoo thay're on page 1. Can you imagine. I hope you try to compete with other companies in defferent ways then criticizing their marketing on the internet. Criticize Google if you want, but never pointing your finger to a particular website.
  • I do not have time for people who do not assume there identity....
    You are either born with balls or they grow on you...
  • anonymous
  • anonymous
  • October 27 Today's Hand-picked Gallery (Must See!)
    October 27 : cute 19 years old aileen showing her natural boobs (Amateurs)
    October 27 : watch as she gets her breasts examined (Amateurs)
    October 27 : the doctor then gives her a gyno and rectal exam (Amateurs)
    October 27 : sativa rose screwing two well hung men (Blowjob)
    October 27 : dirty housewife desirae stripping from her jeans (Amateurs)
    October 27 : little freaky chick plays with her delicious body parts solo (Hairy)
    October 27 : very hairy blonde chick exposes her fury one in the tub (Hairy)
    October 27 : milf's in a lil'cheating (Housewifes)
    October 27 : flower panties amateur stripping (Blonde)
    October 27 : sexy black slut banged by two big white cocks (Black and Ebony)
    October 27 : petite brunette invites her friend over for a little didldo fun (Dildos)
    October 27 : lucky guy fucking two busty blondes (Gangbangs)
    October 27 : blonde and redhead lesbians fucking (Babes)
    October 27 : lays on the table and touches herself (Pussy)
    October 27 : horny busty granny getting her pussy pounded in a threesome (Gangbangs)
    October 27 : gorgeous big tit latina babe loves strip teasing in bikini (Babes)
    October 27 : big tits blonde babe in blue underwear stripping (Babes)
    October 27 : lucky guy fucking a girls ass with a dildo (Anal)
    October 27 : kinky contortionist using an anal sex toy (Anal)
    October 27 : two sexy lesbians playing with toy (Dildos)
    October 27 : two teens getting pierced and tortured (Amateurs)
    October 27 : sexy brunette teen shows her pussy (Brunettes)
    October 27 : naughty blonde gets fucked hard and squirts her cum everywhere (Amateurs)
    October 27 : sexy goth babe with a black umbrella (Amateurs)
    October 27 : crissy moran taking off sexy dress (Beautiful)
    October 27 : insatiable housewife takes on three guys (Gangbangs)
    October 27 : teen redhead slut gives a deep throat blowjob (Blowjob)
    October 27 : amateur teen sucking n gets hard fucking in studio (Amateurs)
    October 27 : gorgeous big breasted milf stripping naked on the couch (Amateurs)
    October 27 : anita dark dressed like a sexy pirate (Beautiful)
    October 27 : all natural busty babe roxy carter spreads her pussy (Babes)
    October 27 : busty natulal teen showing virgin pussy in forest (Beautiful)
    October 27 : teen babe in hard sex act (Blowjob)
    October 27 : emily18 looking mighty hot (Beautiful)
    October 27 : purrfect sadie in panties (Beautiful)
    October 27 : busty teen dildoing her pussy hard with a big toy (Dildos)
    October 27 : blonde tanlined mature housewife doing your laundry (Blonde)
    October 27 : carmel moore gets cum on her tits (Hardcore)
    October 27 : blue eyed mature vixen stuffs dildos into both her mature holes (Dildos)
    October 27 : two seductress want to whack some balls (Babes)
    October 27 : blonde with glass toy (Babes)
    October 27 : helpless gagged girl bondaged and fucked with device in maledom (BDSM)
    October 27 : two sexy lady playing with waiter slave game (Babes)
    October 27 : cute sexy babe does handjob sex hard and blowjob (Amateurs)
    October 27 : busty blonde marketa showing her pussy and tits (Pussy)
    October 27 : hottest bare chick hard bondaged and fucked by master (BDSM)
    October 27 : sexy naked phone call (Babes)
    October 27 : penny flame exposes her tits and ass (Babes)
    October 27 : anime church orgies with priests and nums sex (Anime)
    October 27 : brunette teen takes off panties and spreads hairy pussy (Hairy)
    October 27 : cute brunette babe sucks huge cock and gets fucked on bed (Babes)
    October 27 : attractive brunette with jumbo juggs strips out of lingerie (Beautiful)
    October 27 : wild hardcore groupsex party (Anal)
    October 27 : toying cute lesbian couple (Babes)
    October 27 : slim and smalltittied indian teen (Brunettes)
    October 27 : lesbian anal sex with strapon dildo shoved in asshole (Anal)
    October 27 : sexy skillful feet making a guy cum all over them (Babes)
    October 27 : busty asian babe in pink panties shawing off ripe ass (
  • Welcome to

    You will find everything you need RIGHT HERE, and probably on this page.!
    Don't go bookmarking 20 different sites. Bookmark this one!

    Why? Simple! because we have it all... TGP Galleries, Free Sites, AVS Sites, Movie Galleries, Sex Contacts, E-cards and every bloody thing else.

    You will find all the new stuff on this page and the rest in the categories.

    Dave's Bullshit of the Day/Week/Month/Year
    Sep 24th - News - Server Fixed - Everything is now running fast and smoothly
    Sep 22/23 - News - Slow Server - Network Down - Part of the network that serves and our host is on the fritz
    Sep 9th - Cards - New DangerCards added - Kissing Girls - Send one to a friend today
    Sep 8th - News - New Categories added - Lesbian Strapon, Girls Kissing - Amy Reid
    Aug 9th - Blog/News - Threats to your internet - Burglars! - Porn at "work" and Would you believe
    Jul 25th - Blog/News - New Categories - New categories include - College Coed Sex, Nude College Coeds. We have a new daily fix too.. our Babe of the Day. For those that like porn trailers, we have a new category just for trailers - Reality Porn Trailers
    July 18th - Blog/News - Dell Porn - Dell's new blog has a clone!
    Apr 22nd - Cards - New DangerCards added - Nude Cheerleaders - Send one to a friend today
    Apr 12th - News - Yep we turned the bullshit upside down.. to match the rest of the site :)
    Mar 30th - Blog/News - New Categories - New categories include:- Pussy Closeups, Thai Girls, and Doggy Style.

    Note - Gay links tagged as - Gay, Shemale links tagged as She, For Women links as fW, and Bisexual links as Bi
  • Hello One thing that ticks me off about google is that it desplays...

    agreed can be a bit of a pain
    Trust me i can help