Thoughts on Mark Russinovich's response to Vista network slowdown

Thoughts on Mark Russinovich's response to Vista network slowdown

Summary: Several ZDNet readers have asked for my thoughts on Mark Russinovich's response to Vista network slowdown issue.

SHARE:

Several ZDNet readers have asked for my thoughts on Mark Russinovich's response to Vista network slowdown issue.

Overall, I feel that Mark's response is both thorough, clear and accurate.  However, his blog post on the matter raises some interesting questions about the Windows Vista development process and how Microsoft approached the project.  I'll quote a few interesting passages from the blog and add my own commentary:

The MMCSS service runs in the generic service hosting process Svchost.exe, where it automatically prioritizes the playback of video and audio in order to prevent other tasks from interfering with the CPU usage of the playback software: [emphasis added]

From this we can assume that Vista has been developed with media playback in mind, in other words, it's broadly aimed at the consumer since there's no sensible way to disable to override this feature.

When a multimedia application begins playback, the multimedia APIs it uses call the MMCSS service to boost the priority of the playback thread into the realtime range, which covers priorities 16-31, for up to 8ms of every 10ms interval of the time, depending on how much CPU the playback thread requires. Because other threads run at priorities in the dynamic priority range below 15, even very CPU intensive applications won’t interfere with the playback. [emphasis added]

Again, an indication that pretty much everything is put on the back burner for multimedia, and for quite an extended period of time.  Remember, every system sound that plays will be interfering with performance.  This doesn't sound right to me.

Tests of MMCSS during Vista development showed that, even with thread-priority boosting, heavy network traffic can cause enough long-running DPCs to prevent playback threads from keeping up with their media streaming requirements, resulting in glitching. MMCSS’ glitch-resistant mechanisms were therefore extended to include throttling of network activity. [emphasis added]

This is interesting because I find it hard to accept that the only way to solve the playback problem was to throttle network traffic.  If this is the case, Windows needs a significant and widespread overhaul to improve performance.  Microsoft is clearly willing to apply temporary patches to some pretty deep and serious issues.

Further, there’s an unfortunate bug in the NDIS throttling code that magnifies throttling if you have multiple NICs. [emphasis added]

OK, so there's a bug too.

The throttling rate Vista uses was derived from experiments that reliably achieved glitch-resistant playback on systems with one CPU on 100Mb networks with high packet receive rates. [emphasis added]

So, not only is this a patch, it's a patch intended for low-end PCs running Vista.  A slap in the face for anyone who has spent money on decent hardware.

The hard-coded limit was short-sighted with respect to today’s systems that have faster CPUs, multiple cores and Gigabit networks, and in addition to fixing the bug that affects throttling on multi-adapter systems, the networking team is actively working with the MMCSS team on a fix that allows for not so dramatically penalizing network traffic, while still delivering a glitch-resistant experience. [emphasis added

OK, I have more trouble accepting this "with respect to today’s systems" bit than any other.  Wasn't Vista designed with "today’s systems" in mind rather than yesterday's systems?

Temporary Solutions

So far, I've found two possible "temporary fixes" to this problem.

The first was proposed by my ZDNet blogging colleague George Ou, and that is to switch over to gigabit network gear that supports jumbo frames.  It means spending some money, but not a lot.

Second, you can remove MMCSS from the Vista audio equation.  To do this you'll have to do a bit of registry editing.

Friendly Warning:  If you trash your system messing with your registry, well, you'll probably be too busy rebuilding your system to complain at me!  Seriously, take care ...

Fire up Registry Editor and navigate your way down to:

HKEY_LOCAL_MACHINE\SYSTEM\ CurrentControlSet\Services\Audiosrv

Once there find STRING called DependOnService.  Right-click on this entry and choose Modify ... and then from the Value data remove MMCSS.  Click OK, close the Registry Editor and reboot your PC.  Next, go into Services (the quickest way to get to this is to type services into the Start Search box and click on the entry that appears) and then find the Multimedia Class Scheduler service.  Once found, right-click on it and Properties and change the Startup type from Automatic to Disabled.  Click OK and then reboot.

Topics: Processors, Microsoft, Networking, Windows

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

43 comments
Log in or register to join the discussion
  • What what happens to network performance...

    When playing AACS protected content.
    mrlinux
  • Is anyone buying these excuses?

    I am a Windows user by choice. I prefer the platform, but will admit that I have not tried Linux, so I can only say I prefer it to Mac. However, this is troubling to me for several reasons, and bittersweet for others. I like that they are working to add quality to media, because that was an area where Windows was falling behind other operating systems. However, I find it odd that a company that touts itself as the productivity masters, business-oriented, etc, that they would place more priority on media than they do networking.

    Basically, I think that this was totally intentional, but I think that it was less than thought through, and now they are going to have to fix it.

    I'm not sure if there's a good fix that still provides what they want. It could involve a re-write of the kernel, because it sounds so deeply involved.

    I don't use Vista - yet. I am still running XP, as I am waiting for them to get the bugs out of Vista before the new PC purchase. This is probably why I've never had a problem with any type of crashes, viruses, etc - because I never buy brand new tech. However, I have infinite respect for those of you that are willing to lay down the bucks and work out all the flaws for the rest of us!

    I'm glad that this is something that was discovered before we took the plunge (my fiance needs a new computer and wants to switch to PC from Mac) and hopefully they will get it straightened out soon.
    laura.b
    • Buying Excuses

      We vote with our dollars. When the most significant open market choice we can make
      is a platform choice, a choice for Microsoft endorses this problem, and all the others
      problems before it. It would seem to me that you are the one buying the excuses.
      Literally. Your fiance has the right idea. Platform coexistence and open competition is
      the answer, not righteous indignation and homemade fixes. When there is nothing
      left to do but complain, it's gone too far. Vote with your dollars, or at least let your
      fiance vote with his. Expand your experience and further open up the tech market in
      the bargain.
      Harry Bardal
      • Reread what she said.

        She said her boyfriend wants to switch to the PC from the Mac and not the other way arround.
        ShadeTree
    • Not me...

      Fact is, Windows XP can do it and so can almost every other OS out there.

      Audio playback only needs about 1% of CPU time even on the oldest PC that Vista would ever run on. Hard-wiring it to be 80% on all machines is just nonsense. Nobody could be that stupid.
      jinko
  • There is still something strange with the multi-threading

    In one instance they state

    [B]depending on how much CPU the playback thread requires[/B]
    followed by
    [B]The hard-coded limit was short-sighted with respect to today?s systems[/B]

    It would appear that they performed experimentation with the lowest common denominator to decide how much CPU media playback required instead of the actual media playback thread simply using the CPU it needs as a higher priority process, pre-empting when required and releasing when not required. (and thereby leaving all remaining CPU to any other process that needs it)

    In a nutshell, are we seeing the OS deciding that 80% of CPU time is reserved for media playback, regardless of how much CPU media playback requires?

    I must be wrong. I couldn't even see this being a viable option for Windows 3.1, it is just such a fundamentally flawed design for a multi-tasking OS.

    [B]the networking team is actively working with the MMCSS team on a fix that allows for not so dramatically penalizing network traffic, while still delivering a glitch-resistant experience.[/B]

    Can someone with more experience elaborate? I believe that NT had a much superior multi-tasking algorithm whereby CPU time was based on priority with the process using as much as it wanted, but not rule based reservation of CPU time?

    TripleII
    TripleII-21189418044173169409978279405827
    • Windows never did true multitasking.

      They "approximated" preemptive multitasking by creating an efficient cooperative multitasking algorithm.

      I was under the impression that one thing Vista would finally do is true preemptive multitasking but that was scraped --along with many other plans-- before Vista shipped.

      That is my understanding of the situation. I have been wrong before. There was this one time back in the autumn of 1977 when....
      Logics
      • NT is a premtive multitasking OS

        You should find whoever told you that Windows is still a cooperative multitasking OS and slap them for telling you lies.
        toadlife
        • That's what I thought

          It did a real (or much closer emulation) of true multi-tasking, pre-emptive, etc. Reading the response, it seems to me that Vista might not be as good (in this one respect) as NT was.

          In any case, it appears not to be DRM related, which is one positive out of all of this.

          TripleII
          TripleII-21189418044173169409978279405827
      • You must have meant you have been wrong ....

        ... since 1977. Windows has done true multi-tasking since NT.
        ShadeTree
    • Suspicious

      All we know is what bubbles to the surface. Microsoft seems to be shying away
      from parallelism. This may cause problems with Intel, which has embraced it
      wholeheartedly. They had to, they hit a thermal wall, just as IBM has. Fighting laws
      of diminishing returns is now considered a better option than fighting the laws of
      physics. But in this new paradigm, has any Windows box leveraged more than 4
      cores/CPU's to any positive effect? As XBox 360s melt all over the country, it may
      be possible that MS devotion to clock speed increases is catching up to them.

      Intel's new compiler contains functions that distribute load across multiple cores.
      Apple's new pro machines run up to 8 cores. Meanwhile, Vista can't walk and chew
      gum at the same time.
      Harry Bardal
      • Selective memory.

        Is it possible for you to be more wrong? In the past couple weeks there has been articles on Microsoft's focus on parallelism like this one;

        http://blogs.zdnet.com/microsoft/index.php?paged=2

        The only thing being shyed away from is you from the truth. Your the one that can't walk and chew gum at the same time.
        ShadeTree
        • No... You Are

          I'm so delighted when I hear my own words parroted back. It makes me feel as
          though I'm having some small effect. Besides, "I know you are but what am I",
          what have you got for ammo. That link? The one that proves beyond a shadow of a
          doubt that Microsoft has begun to think about parallelism? They have even held
          meetings and assigned code words. Well, I wait with bated breath for something
          to ship.

          It's like I say all we really have is what bubbles to the surface. In this case you have
          promises, code words and research projects, and I have thousands upon
          thousands of red flashing rings lying in smoldering beige puddles. I have the
          overclocked and overcooled brag boxes of each of the PC OEMs. I also have this
          new fiasco, and a bald faced admission of yet more bugs and errors in judgement.

          You folks forgive more than a confessional priest on the sunday after mardi gras.
          Harry Bardal
          • Funny!

            With all the thermal issues on the original Intel MacBook Pro the same could be said of you. Not to mention all the battery recalls and display issues. It would seem your hypocrisy knows no bounds.
            ShadeTree
  • Still misrepresentation going on

    Let me state right off the bat that I don't agree one bit with the way this was implemented. [b]However[/b], I believe you are misrepresenting the situation.

    [i][b] The MMCSS service runs in the generic service hosting process Svchost.exe, where it automatically prioritizes the playback of video and audio in order to prevent other tasks from interfering with the CPU usage of the playback software: [emphasis added][/b]

    From this we can assume that Vista has been developed with media playback in mind, in other words, it?s broadly aimed at the consumer since there?s no sensible way to disable to override this feature.[/i]

    Since when does the ability to increase the priority of a process mean that the entire OS was developed with that single process in mind? I can give Open Office a real time priority in Linux so that means Linux was developed with Open Office in mind? Of course not.

    Vista was designed to support dynamic real time scheduling. The team that wrote MMCSS took advantage of that ability. The very fact that the fix is as simple as disabling the MMCSS service suggests that in fact, Vista couldn't give 2 hoots about whether or not MMCSS is running and is therefore not dependent at all on real time scheduling of multimedia playback. I will agree with you that there should be a more intuitive way for a user to fix this.

    [i]Again, an indication that pretty much everything is put on the back burner for multimedia, and for quite an extended period of time. Remember, every system sound that plays will be interfering with performance. This doesn?t sound right to me.[/i]

    Nor should it sound right to you because it isn't accurate. Everything is put on the back burner for [b]processes with real time priorities[/b]. As in my statement above, it was the MMCSS team that took advantage of this. Unless someone can prove that MMCSS is the only process that can take advantage of real time scheduling, no one can say that Vista was designed to give multimedia playback priority over everything else.

    You are also wrong when you say that every sound will impact performance. First off, reading George's post from a couple days ago suggests that MMCSS is [b]not[/b] a requirement for playing audio. Second, applications can choose which [url=http://msdn2.microsoft.com/en-us/library/ms684247.aspx] MMCSS scheduling category [/url] to use. Choose the "Low" scheduling category and the thread won't be given 80% of the CPU's time.

    [i]Wasn?t Vista designed with ?today?s systems? in mind rather than yesterday?s systems?[/i]

    Again, there seems to be confusion between the words "designed" and "configured". XP was designed to support restricted rights accounts but it was (unfortunately) "configured" to default users with Administrator privileges. From everything I've read about this situation, they chose a poor "configuration" but the design of the system, as proven in the MSDN documentation I linked to, is flexible enough to accommodate different configurations.

    Finally, the fact that you can so easily remove the service from the audio pipeline suggests that MS actually did a fairly good job of creating a flexible multimedia system. It could probably use some UI work to make it easier for the end user but the design itself seems to be solid.
    NonZealot
    • You missed the phrase..

      "where it automatically prioritizes the playback of video and audio."

      This is not like you setting priorities for OpenOffice. You have no control over this,
      and that's Adrian's point: Media playback is given higher priority in Vista than
      network throughput. That is strange in an OS marketeed to business.
      msalzberg
      • And despite the quote you gave from George,...

        ...systems sounds have been shown to impact network performance. Have you not read the fist three blogs on this?
        Logics
      • And you misunderstood what "it" was

        [i]"where [b]it[/b] automatically prioritizes the playback of video and audio."[/i]

        "it" is the MMCSS service, not Vista itself. Vista doesn't automatically prioritize the playback of video and audio, it simply respects the wishes of a system service that asks for real time scheduling.

        [i]You have no control over this[/i]

        Hmm, funny then how Adrian shows exactly how to disable the MMCSS service after which playback of video and audio are [b]not[/b] given higher priority than network throughput. I agree (and stated) that the control you have has a lousy UI but you most certainly do have control. I'm also assuming you followed the MSDN link I so kindly provided but you just happened to miss the paragraphs that describe, in detail, all the control you have over the MMCSS service. Again, the UI isn't nice (it is more programmatic than graphical) but the control is there.

        My bigger contention with the blog is that it suggests Vista was [b]designed[/b] to sacrifice network performance for multimedia performance. It wasn't for the exact same reasons that Linux wasn't designed to sacrifice network performance for OpenOffice performance. Vista was configured, by default, to do this but that is a very different thing than saying it was designed to do this.
        NonZealot
        • "My bigger contention with the blog is that it suggests"

          Nope, it doesn't "suggest" anything. It
          merely points out the facts.
          Ole Man
        • A couple of questions...

          Can MMCSS be turned off? If not, I'd have to say it's a part of Vista.

          I will agree with you that the defaults are designed to sacrifice network
          performance, which is not the same as Vista being designed that way. On the
          other hand, it's taken a week, a lot of head-scratching, and a message from a
          Microsoft Technical Fellow to figure this out.

          It's not exactly obvious.
          msalzberg