As a long time tech journalist, I've also been in the "awards" business for a long time and as such, I often marvel at the process, the methodology, and the urge to do technology awards. Don't get me wrong. There are definitely some well-deserved awards that are based on rigorous testing. But then there are those "Best in Show" awards that you see at certain trade shows where the judges are expected to render their decisions on little more than booth walk-bys (or reading some application in advance of the show). Practically speaking, the judges can hardly be expected to actually give any of these Best in Show candidates a workout. In the past, when I've been a judge, I often tried. But the effort was futile because my vote was usually one of many, the rest of which were doing no testing whatsoever.
Then, there are the categories. When I was running PC Week's testing labs (now eWeek Labs), we were constantly changing category names to fit the products that were in the marketplace. This caused subtle havoc to the best intentioned testing. For example, when Lotus Notes hit the scene, it had all these features that redefined the category of collaboration. You couldn't readily ignore some of the ground breaking features when comparing a product like Notes to the status quo, so you'd expand the testing criteria beyond something simple like e-mail and group calendering to include some of those market-disruptive features like workflow, off-line capabilities, and replication and then, as a result, Lotus Notes would win in a shoot-out versus other products because those other products would score a zero on the newer criteria.
To manage the problem, we'd perform these unnatural acts where, in one comparison, the category would be "Groupware" and if other products at the time (like Microsoft's Exchage for example) had nothing in certain areas, then so be it. They lost. Then, we'd do another comparison under the category of "E-Mail" and all the cool features in Lotus Notes would have to be overlooked to keep the comparison down to just the e-mail functionality. As a side note, after doing such comparisons, the Lotus folks used to send us nasty grams complaining that Notes should not have been included in an e-mail comparison because it wasn't an e-mail system. To those of us at Ziff-Davis (at the time), where Lotus Notes was actually the internal standard e-mail system, we found this to be quite laughable.
As another side note, it's always worse news for a vendor when the product or service being tested is one of the products that the testers actually use in real life. This is why, when I test products like the Motorola Q, my preference is to hang on to them for a long time and actually try to live with them. It's only then that you become deeply familiar with a product's or service's pros and cons (an approach that stands in stark contrast to the methodology that goes into the "Best in Show" award).
Why do I bring this up now? In advance of the Consumer Electronics Show (CES) in Las Vegas during the second week of January 2007 (where I'll be heading with the TV cameras), I'm getting blasted with pitches from all sorts of people, organizations, and vendors looking for coverage (please accept my apologies if you haven't heard back from me yet.... it's a lot of mail). Yesterday though, one of those Best in Show pitches came through that reminded me of the challenges in honoring products, services, or technologies. The pitch came from the National Television Acadamy which, for the first time, will be handing out its Technology and Engineering Awards at CES and it includes a lot of great categories that, knowing how such category-picking goes, I'm sure took a lot of work (and healthy debate) among a hard-working very well-intentioned group of people. But the category that made me laugh out loud was the one for which that was won by Gennum Corporation:
Technology Advances in Serial Digital Interface Solutions, Enabling Over 20 Years of Seamless Studio and Broadcast Infrastructure Migration
If I didn't know better (maybe I don't), this looks remarkably like the tagline for a product or a company (I did a cursory drive-by on Gennum's Web site for similar text and didn't find any). Even so, I still marvel at categories like this. What was even funnier is that right after getting that e-mail, one of Bob Sutor's (IBM) posts squirted through my RSS reader: his 2006 "BOBBY Awards" and I have to thank Bob for an equally good laugh. Sure, Bob has a disclaimer on the top right hand corner of this blog that says "The postings on this site are my own and do not necessarily represent my employer’s positions, strategies or opinions." But you can't help spot the serendipity in some of his selections.
For example, in the category of Best Open Standard Being Adopted Globally, the BOBBY went to the OpenDocument Format. That's the document format that IBM along with other vendors have spent gobs of cash on in hopes of finally breaking the world's addiction to Microsoft Office. That goal of course, isn't lost in one of the other BOBBYs (another one of those categories that seems made to fit the honoree): Best Traditional Lock-In Strategy Using "Open" for Marketing. The winner? Microsoft Office and it's Office Open XML file format. Even funnier was Bob's next category: Best Supporting Actor for a Traditional Lock-in Strategy Using "Open" for Marketing. The winner? Ecma (the organization that, despite IBM's objections, just recently put its imprimatur on the Office Open XML specification. In this case, I sort of agree with Bob's assessment of the situation. Forgetting Office Open XML for a moment (my assessment of Ecma has nothing to do with the specification in question), Ecma is a joke and is undeserving of the special ability that it along with only one other organization (the ITU) has to put its specifications on the so-called "fast track" of the International Organization of Standardization (the ISO). For more on why, you can read my post on Ecma's process and my interview with its leader, an excerpt of which goes like this:
DB: The OpenDocument Format is already in front of the ISO for ratification. How can the ISO possibly consider another format as a stanard that does the same thing? Wouldn’t that be two standards for the same thing? Doesn’t that defeat the purpose of a standards?
JvdB: Microsoft’s formats and the OpenDocment Format are not exactly same thing. They’re just alternative approaches to the same problem. There may be a possibility that some is merged into the other. But they’re not 100 percent identical and this is OK. Ecma made all standards for DVD — five competing rewriteable/recordable formats. They all do the same thing. The reason there are five is that there this is a patent war. All of those standards have been fast tracked to the ISO and all been approved without any comments. The ISO cannot decide for one industry group. It must be neutral. it can pick one over the other. There’s no possibility for a standards body to decide it in favor of IBM, Sun, or Microsoft. It’s very possible for the ISO to set both standards. There may be overlap, but it doesn’t matter. For the ISO, it’s impossible to get in the way of patent wars.
DB: But don’ t you think that encourages patent wars?
JvdB: I have never thought so deeply about it (how the pemission of multiple standards encourages patent wars). In a way yes, of course, there are hardly any subjects in hi-tech where no patents are involved. That is one of the big worries about bodies being concerned with patents So, we stay out of it. If you have a patent, you get an unconditional right and what you do with your patent is your business. You can ask "is that good?" Well that’s an interesting question.
To me, Ecma is not a standards body. As evidenced by the DVD situation (which is ridiculous if you ask me), it's little more than a puppet with a pipeline through which vendors can pump their proprietary technologies into the ISO standardization process (avoiding the rigor that should normally be applied to anything up for consideratoin as an ISO standard). As such, the ISO is sort of a joke too.
Back to the topic at hand, there are of course some well-deserved honors and rankings out there. Doc Searls blogged about some of the rankings he appears in. Doc, to me is the ultimate technology statesman and reminds me very much of former PC Week editor-in-chief Sam Whitmore (one of my earlier mentors) who now runs MediaSurvey.com. Doc and Sam may often disagree with what you have to say about people, events, or other entities and will never shy away from telling you. But their tip-of-the-tongue articulations are always so incredibly respectful (and statesman-like) that no one, not even the target of their criticisms can or should feel as though they've been insulted. For example, in responding to a part of my post about how TinyURL could be the next YouTube, where I say "I'm not sure whether Doc would agree," he could have slammed me. But instead, he wrote "I'm not sure either, but that claim certainly gets my attention." Take that however you want. He's clearly not in violent agreement and some including me probably interpreted it as disagreement. But, the way he phrased his response sounds very much like "I heard you, processed what you had to say, and respect your opinion."
Sam and Doc are the best communicators I know and are every bit deserving of any honors they get. Doc gets my vote for All-Time Best Headline Writer. His brain is a bottomless pit of creative, succinct, word-twisting, almost-perfect-every-time headlines. He could probably make millions doing nothing but writing headlines for others. Someone should figure out how to connect a Headline Web Services API to his brain. Call it the SearlsPearls API.
Finally, for a really good laugh that has to do with rankings, read Nick Carr's nearly perfect analysis of the different audiences that go to Google vs. Yahoo vs. AOL, appropriately headlined (is he testing the SearlsPearls API?) Dweebs, Horndogs, and Geezers. It's real, factual, hilarious, and ridiculous and it gets my nomination as one of the greatest blog posts of all of 2006.
Next year, be sure to look out (or try to self-nominate yourself) for my Dead-Finger Tech awards. Read more here. Who knows? Maybe, by the end of 2007, I'll be giving you The Finger.
Ah, awards. Cant' live with 'em. Can't live without 'em! Have you spotted