1. Things have changed. According to Thurrott, “The real problem here is that the feature set of Windows 7 was frozen well before the Beta release. So the feedback [Sinofsky] discusses throughout this post is 99 percent bug testing, really (and 1 percent, we hear your concerns but have a million reasons why we can't change a thing).”
And this is a problem? I don’t think it’s any accident that the two most troublesome releases in the history of Windows also had the longest beta cycles. I was running pre-beta builds of Windows 95 in 1993, nearly two years before it was released. The first alpha releases of Longhorn, which eventually became Windows Vista, were handed out at the PDC in late 2003, nearly three years before Vista shipped.
By the time a beta is released, the feature set should be pretty much frozen. That’s how you concentrate on things like quality and performance.
As for the complaint that Microsoft hasn’t listened to feedback and ignored its most loyal customers when developing the feature set for Windows 7, I say, “Give me a break.” Since November 2006, Microsoft has gotten an earful about its Windows design decisions. The feedback loop includes:
- Every blog post, review, newsgroup posting, and rant about Vista ever published
- Support calls to Microsoft and its partners
- Requests from PC makers and software developers
- Telemetry data (all those crash reports really do go somewhere)
- Field research and usability testing
- Interviews with opinion leaders, including Paul and me, who have given feedback to Microsoft in person and on the phone many times in recent years. You think they weren't taking notes?
That’s a helluva lot of feedback to take into account. There comes a point where more doesn’t mean better.
2. Windows design is a series of compromises. A lot of the complaints I’ve heard boil down to “Well, that’s not how I would have designed that feature.” Right. When you’re building a product that is going to be used by hundreds of millions of people, you have to find some common denominators. And as I wrote last year, sometimes there is no right answer: you can bet that for any decision you make, some nontrivial number of people will think you’re a complete idiot, no matter which option you choose.
I also hear lots of feedback suggesting that Microsoft should never remove a feature and should always give its old-time users a way to preserve the procedures they learned five or 10 or even 20 years ago. Seriously, I’ve heard people argue that Windows 7 should include the old File Manager utility from Windows 3.1. Can you imagine how complicated, even bloated, Windows code would be if no feature was ever cut and you could choose from a dozen or so Classic interfaces going back to 1991? But that's the logical conclusion from that line of argument.
Things evolve. Old features disappear, and new ones are introduced. Deal with it. If you think there's a better way to implement a new feature than the one Microsoft chose, blog it. But it helps if you can make a rational case - remember, you're dealing with engineers. Simply saying "XYZ feature sucks" isn't likely to win hearts or minds.
3. Writing good bug reports is hard work. I sympathize with testers who complain that their bug report was closed as “Non-repro.” But that’s reality. If it was easy to reproduce, the bug would most likely have been caught in one of the many, many automated testing cycles that each Windows build goes through. The really tricky bugs are those that are triggered by unusual combinations of hardware and software under specific conditions.
In fact, if you talk to the developers who dig into those incoming bug reports from technical beta testers, as I’ve done, you’ll quickly learn that it’s a pretty low-yield process. Most are duplicates and the vast majority are just requests for new features or changes to existing ones. When they get closed as “won’t fix” or “by design,” it’s because someone already considered that request and decided for any of a thousand reasons (budget, compatibility, risk of regression, or conflicting data) that the feature is going to remain as it is.
4. One more build does not mean a better product. In fact, you could argue as I did last week that the work of building an “official” beta release slows down progress. Pytlovany, who has worked inside Microsoft, agrees:
Every new beta release is a distraction to developers. The time it takes to create a frozen version takes away from a developers imagination and productivity. […] The internal testing required before any public beta is a lot longer than you might think.
If you’ve identified a bug and it’s made it onto the must-fix list, it shouldn’t take multiple passes to fix it. Microsoft’s Charlie Owen, who works on the Media Center team, had some great advice for beta testers in a blog post last week:
When the Windows 7 Release Candidate becomes available immediately download, install, test deeply and quickly provide actionable feedback.
Seriously: As the release candidate is downloading and with tenderness, kiss your spouse on the cheek and tell him or her you'll be back in a week or so. Then lock yourself in the home office and be relentless and unforgiving in your testing of the Windows 7 Release Candidate and provide feedback.
5. Shipping is a feature. There is no such thing as perfect software. If the developers of any complex software product like an OS waited for every bug report to be “fixed.” the software would literally never ship.
The one thing Microsoft can do going forward that it has not done well in the past is to incorporate feedback from the current test cycle into the next version. The best way for that to happen is for Microsoft to develop a consistent, professional process for planning and shipping new releases on a predictable schedule. In theory, features that don’t make it into this release have a legitimate shot at making it into Windows 8. As Charlie Owen put it, "You should consider the Windows 7 Release Candidate as your first and best opportunity to influence the next version of Windows."