What do Windows 7 beta testers really want?

Windows beta testing ain't what it used to be.

Over the past couple of days, this reality seems to be setting in among some Windows 7 testers.

Windows 7's testing and release schedule is like that for Office. (Given that Windows Engineering chief Steven Sinofsky ran the Office development team for years before moving over to Windows, is anyone really surprised?) Sinofsky and his team are holding their cards a lot closer to the vest and are releasing very few public testing milestones. But beyond failing to inform public beta testers of what's going on, the Windows 7 team also is remiss in providing even the core group of "technical beta testers" with adequate feedback, some of these testers claim.

We've known for a few months that Microsoft planned only one public Beta and one Release Candidate for Windows 7. But it seems that it took the fact that internal Windows 7 builds have started sporting the "Release Candidate" moniker that testers have begun to understand that they are not going to have much input in -- or even feedback on -- what Windows 7 ultimately will look like.

I've argued that the Windows 7 feature set was basically set in stone long before the majority of testers ever saw the first Milestone test builds. That's how Microsoft traditionally has run the Office beta process: By the time the public Beta 1 hits (something that still has yet to happen with Office 14, by the way), the product is basically done. Beta 2 in the Office world is a formality and often goes out just a few months before RTM. The feature set for each subsequent release of Office is relatively confined and "big bang" releases in the Office world are few and far between.

This week's calls for one or more additional Windows 7 Beta releases -- by my blogging colleague Adrian Kingsley-Hughes, among others -- leads me to wonder what testers really want.

Do they want Microsoft to take three or more years to roll out a new version of Windows (which I'm sure the Windows brass would claim would be the result of the addition of another public beta or two)? Do they want more testers to be granted access to more of the interim builds that Microsoft is doling out to only a small group of preselected testers? Do they want the Redmondians to release a public (or even semi-private) list of bugs that are fixed on a weekly basis?

My sense is testers are less interested in another Beta or another Release Candidate being added to the Windows 7 schedule than they are in getting more, regular feedback from the Windows 7 team.

Chris Holmes, one of the first of the Windows 7 testers to push this week for changes in the Windows 7 beta feedback process, said Windows 7 testers are in the dark:

"There have been a lot of us (testers) that have submitted feedback relating to design changes, things such as the superbar and the ability to have quicklaunch (this doesn't bother me personally but some people want it), and changes around explorer as a whole, things relating to grouping settings not holding sometimes, etc.," Holmes said.

But there has been no word back from the Windows 7 team on any of these issues, he noted.

Another example: The ability to switch the x64 flavor of 7 to use the 64 bit media player, Holmes said. "The setting does not actually take effect in (the Windows 7) 7000 (builds), and it is also not working in 7022 x64, but I was told a month ago that it was fixed in newer builds."

"Bottom line, and the point I was trying to make, is that Microsoft can't take in feedback and then not allow the community to verify that the issues are indeed fixed. Sinofsky's 'Office Culture' doesn't seem to be working in the windows world. He has a group of techbeta testers (there are about 5,000 of us i believe) that he seems to not be using for anything other than giving us the ability to track our feedback, which does no good when our bugs are being closed with the comment 'Try in a newer build.' What newer build?"

I'm not saying Microsoft isn't providing any feedback to testers. But it sure seems to take an awful lot of public outcry to get the Windows team to do so. (And even after providing a bit of feedback, the willingness to share more related details seems to end, leaving still more unanswered questions.)

In responding to Holmes and the Geeksmack commentators, Sinofsky said Microsoft is poised to share information on changes that have been made in Windows 7 in response to user feedback. It will be interesting to see how, when, where and the extent to which the Windows team shares that information....

In the interim, do you think Microsoft really needs another Windows 7 beta (or two) before the product is released to manufacturing? Or would more public disclosure as to what's happening behind the scenes be sufficient?