What do Windows 7 beta testers really want?

What do Windows 7 beta testers really want?

Summary: Windows beta testing ain't what it used to be. Over the past couple of days, this reality seems to be setting in among some Windows 7 testers.


Windows beta testing ain't what it used to be.

Over the past couple of days, this reality seems to be setting in among some Windows 7 testers.

Windows 7's testing and release schedule is like that for Office. (Given that Windows Engineering chief Steven Sinofsky ran the Office development team for years before moving over to Windows, is anyone really surprised?) Sinofsky and his team are holding their cards a lot closer to the vest and are releasing very few public testing milestones. But beyond failing to inform public beta testers of what's going on, the Windows 7 team also is remiss in providing even the core group of "technical beta testers" with adequate feedback, some of these testers claim.

We've known for a few months that Microsoft planned only one public Beta and one Release Candidate for Windows 7. But it seems that it took the fact that internal Windows 7 builds have started sporting the "Release Candidate" moniker that testers have begun to understand that they are not going to have much input in -- or even feedback on -- what Windows 7 ultimately will look like.

I've argued that the Windows 7 feature set was basically set in stone long before the majority of testers ever saw the first Milestone test builds. That's how Microsoft traditionally has run the Office beta process: By the time the public Beta 1 hits (something that still has yet to happen with Office 14, by the way), the product is basically done. Beta 2 in the Office world is a formality and often goes out just a few months before RTM. The feature set for each subsequent release of Office is relatively confined and "big bang" releases in the Office world are few and far between.

This week's calls for one or more additional Windows 7 Beta releases -- by my blogging colleague Adrian Kingsley-Hughes, among others -- leads me to wonder what testers really want.

Do they want Microsoft to take three or more years to roll out a new version of Windows (which I'm sure the Windows brass would claim would be the result of the addition of another public beta or two)? Do they want more testers to be granted access to more of the interim builds that Microsoft is doling out to only a small group of preselected testers? Do they want the Redmondians to release a public (or even semi-private) list of bugs that are fixed on a weekly basis?

My sense is testers are less interested in another Beta or another Release Candidate being added to the Windows 7 schedule than they are in getting more, regular feedback from the Windows 7 team.

Chris Holmes, one of the first of the Windows 7 testers to push this week for changes in the Windows 7 beta feedback process, said Windows 7 testers are in the dark:

"There have been a lot of us (testers) that have submitted feedback relating to design changes, things such as the superbar and the ability to have quicklaunch (this doesn't bother me personally but some people want it), and changes around explorer as a whole, things relating to grouping settings not holding sometimes, etc.," Holmes said.

But there has been no word back from the Windows 7 team on any of these issues, he noted.

Another example: The ability to switch the x64 flavor of 7 to use the 64 bit media player, Holmes said. "The setting does not actually take effect in (the Windows 7) 7000 (builds), and it is also not working in 7022 x64, but I was told a month ago that it was fixed in newer builds."

"Bottom line, and the point I was trying to make, is that Microsoft can't take in feedback and then not allow the community to verify that the issues are indeed fixed. Sinofsky's 'Office Culture' doesn't seem to be working in the windows world. He has a group of techbeta testers (there are about 5,000 of us i believe) that he seems to not be using for anything other than giving us the ability to track our feedback, which does no good when our bugs are being closed with the comment 'Try in a newer build.' What newer build?"

I'm not saying Microsoft isn't providing any feedback to testers. But it sure seems to take an awful lot of public outcry to get the Windows team to do so. (And even after providing a bit of feedback, the willingness to share more related details seems to end, leaving still more unanswered questions.)

In responding to Holmes and the Geeksmack commentators, Sinofsky said Microsoft is poised to share information on changes that have been made in Windows 7 in response to user feedback. It will be interesting to see how, when, where and the extent to which the Windows team shares that information....

In the interim, do you think Microsoft really needs another Windows 7 beta (or two) before the product is released to manufacturing? Or would more public disclosure as to what's happening behind the scenes be sufficient?

Topics: Windows, CXO, Microsoft, Operating Systems, Software


Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • They want something that works that they can use for free

    At least that's what the majority want. The ones who actually take the beta seriously just want to be taken seriously. MS uses the beta primarily as a marketing tool, so at least the majority gets what they want.
    Michael Kelly
    • Well then....

      [i]They want something that works that they can use for free[/i]

      Well there's always Linux for them, isn't there?
      Hallowed are the Ori
      • He said something that works!

        For most people that leaves Linux out. It doesn't run their favorite apps!
        • What favorite apps?

          Oh, you mean the web browser, e-mail client and word processor?
          [b]ANY[/b] platform, Linux distros alike, have these.

          What users should want beyond these apps is a computer which
          actually works, something which is debatable about Windows with its bazillion viruses, bugs and other annoyances.

          Mary Jo writes about Microsoft and its execs and developers as the
          usual nice people, which they likely are, but it makes it easy to forget
          what a substandard platform Windows really is.

          I'd argue any day that Linux is much better for most people, and Macs
          being far ahead on the evolutionary ladder.
          • How quickly the Linux advocates forget inauguration day.

            They were all up in arms because Linux systems could view the inauguration. Now they're arguing Linux is much better for most people. LOL!
          • Huh???? [NT]

          • Here is a link:

          • Silverlight is a MS controlled type of media

            Silverlight is MS's answer to Flash because, how should I phrase it, [i]Windows is a thirty-two bit extension and GUI shell to a sixteen bit patch to an eight bit operating system originally coded for a four bit microprocessor and sold by a two-bit company that can't stand one bit of competition.[/i]

            Pay special attention to the last part about the abusive monopolist not being able to stand one bit of competition....

            If you don't understand the problem, and you obviously don't, then you have a problem seeing the reality.
          • Linux advocates forget inauguration day.

            No one really cared about inauguration day. Big WOOP..
          • heh..

            My thoughts exactly. but appearently ye did.
    • You may be right ...

      I mean, beta testers complaining about finding bugs :/, isn't that the point?

      I just did a phone interview soliciting my feedback on Media Center. MS is working hard to make this version of Windows great. We all know, and Microsoft knows Vista wasn't what it shouldv'e been and they can't afford for that to happen again.

      Also, with all the leaks, MS has a right to keep its intellectual property close to vest. When beta tester's input is no longer valuable using the current builds, the will refresh.

      Sure, it's the standard for contributing testers to at least get a license for the RTM product, but you have to do the beta testing first. Some testers don't want to wait. Is it greed, no. In my opinion its because they see what the rest of us see, Windows 7 is gonna be good.
      • The problem with MS surveys

        I don't know if yours was any different but I hate the one a scale of 1 -5 type questions. I'd love for MS to also do surveys with the following two questions:

        What problems did you encounter?

        What items would you like improved and HOW?
        • This was pretty good actually ...

          At the end there was about 4 very specific scale questions, 1 to 7. And they let you explain why you gave it the score.

          It's was over an hour and other than those questions more conversational. They did ask for and take all suggestions with the obligatory we can't promise any major changes will be made a this point but we want to hear what you like, dislike, and they requested wish list.

          There's something different this time around ...
    • They want the chkdsk file 9 error fixed

      "I stared, in horror, as chkdsk replaced all of my security descriptors on my 10, 000 files and replaced them with "default security id's"."

      Wait until it happens to you. Then you'll find out it's been over 10 years and MS has failed to fix this catastrophic bug.

      Microsoft! FIX THIS BUG!!!
  • RE: What do Windows 7 beta testers really want?

    No i don't think that Microsoft needs another beta release.
    The next release should be the R.C.
    Windows 7 though impressive, is not a revolution from Windows Vista, to require several betas.
  • No, they do not need another beta, MS needs to

    listen to their users and let them know what is being done.

    Having been involved with Office as a beta tester the last 4 versions I can tell you it just doesn't happen, even when you have direct access to the development team(s).

    No, I will not waste my time doing a beta test of Office 14 and could not care less about Windows 7.
    • I agree

      ... remember all those Office 2007 betas we saw ...oh wait ...

      I agree with you, fro a beta testing point of view, there's just no point beta testing Office any more.
      Adrian Kingsley-Hughes
      • Adrian - tell me you're kidding ...

        <i>"I agree with you, fro a beta testing point of view, there's just no point beta testing Office any more. "</i>

        Have you EVER worked in a large IT organization that relies on Office for its day to day operations?

        Wouldn't you think that Beta testing both new OS and Office suites would be somewhat interesting and useful for such organizations to ... y'know ... make sure that all those in-house hand-cranked apps, tools, forms, processes etc., all work on the next version of Office / Windows?

        I'm being facetious above because I generally respect your views, but your post above was just infantile.
        • That's not what beta testing is for

          beta testing is to make sure errors and problems in the software are found, not to provide a freebie version for people to play with. If the IT department wants to check integration, I'm sure what they really do is wait for the final release version, buy it, and test THAT in their environment, and not some piece of software that has possible flaws etc still needing to be ironed out.
  • more responsive to bug submissions

    We really just need a very easy way to follow up on bug submissions and suggestions. Not individual submissions, but at least general areas and how Microsoft has addressed them.

    We're the customers, and customers who beta test a product should be provided with feedback.