Apple's Spotlight: keywords are so '90s

Is Vista any better? With massive storage comes massive, unstructured data: needle in a cybernetic haystack. Search technology is the limiting factor for human access of massive data.

Keyword search works if there aren't too many words Personally, the whole keyword search thing is losing its charm, not to mention utility, as there are more and more things I know are out there - or on my 120 GB notebook drive - that I can't find with any of the half dozen keyword-based search tools I commonly use. Today, it's about just one of them.

The search function on Mac OS X is called Spotlight. It is supposed to create a index of every word of every document so you can quickly find content - .doc files, pdf's, mail messages, applications, presentations, even images - just by typing a keyword.

Except it is a real disappointment.

First, the whole "it just works" thing got left out I buy software on line and from sad experience know it is wise to keep all registration info in one place. A text file created with Apple's TextEdit named "Serial Numbers Software" keeps it all together. When I type "serial" into spotlight, that file never shows up, even after scrolling through the 362 files it does find. A Word doc with serial in the name does. If it weren't so irritating it would be spooky.

The UI needs work What is so hard about a text box? Type in the word, hit return. But Spotlight starts searching with the first letter and the UI gets distracting as new results appear with each new letter. Useless, showy virtuosity.

Slow learner Spotlight gives equal weight to a word no matter if it is buried deep within the file or is in the name. So instead of thinking of my keywords, I have think of a word within the document unique enough to narrow down the search. Who is supposed to be helping who?

Nor does it keep track of favorites, or recently opened files, even though it has all the metadata. I use the excellent freeware EasyFind utility from Devontechnologies to actually find files. Now if they'd just stick it on the menubar!

Cutting edge or cranky? Cranky for sure. I spend hours on the web each day, writing and researching (as I should be right now). For a recent project I've collected about 100 pdf's, many with titles like 5056004309.pdf so, searching within documents is undeniably helpful. What I've realized though is that keyword searches aren't enough.

The Storage Bits take Search is the biggest problem in a world of unstructured storage, especially personal storage. Google's Page Rank algorithm seems smart because it harnesses the judgments of people to create better search results. But no one is linking to my software serial numbers file, so Page Rank is no help.

My laptop today has 100x more RAM than the drive of my laptop of 10 years ago and 6000x more disk. I'm lucky if I'm 1x as smart. Run the numbers forward 10 years to roughly 500 GB RAM and 30 TB of disk. Some smart grad student has a bright future if they can fix this one.

I know help is on the way, with improvements to Spotlight planned for Mac OS 10.5 this October. Yet I suspect I live a more search and data intensive life than most people today, so I'm seeing the problem sooner.

Comments welcome: what works for you? Vista users, feel free to give your 30 second take on how the search function works vs Spotlight.