Patches: Once a month is not enough

Patches: Once a month is not enough

Summary: Microsoft needs to rethink its policy of stockpiling security bulletins and patches and releasing them all on the second Tuesday of each month. It doesn’t solve a problem for any customer, and it exposes an unknown number of innocent Windows users to unnecessary risk.

TOPICS: Security

Microsoft needs to rethink its policy of stockpiling security bulletins and patches and releasing them all on the second Tuesday of each month. It doesn’t solve a problem for any customer, and it exposes an unknown number of innocent Windows users to unnecessary risk.

The whole concept of Patch Tuesday started in November 2004. Ostensibly, the reason was “to assist customers with resource planning for the scheduled monthly security bulletin.” In reality, it was a symbolic move to take control of the news cycle, after headlines had been appearing seemingly every week announcing Yet Another Horrible Security Flaw and Yet Another Critical Update for Windows. It’s true that corporate customers like an orderly update plan, but the real impact of the Patch Tuesday program is that Microsoft gets to dump all their bad security news in one big heap, once a month. They get positive press when they only have a few updates to release in a given month, and – with the exception of extraordinary events – the news cycle is contained to a few days between the preview announcement and the actual delivery of security bulletins.

That’s not necessarily a bad thing. Critical updates by definition are, well, critical. But that doesn’t mean the world is going to end if a Critical patch is delayed by 5, 10, or even 30 days. Reducing the level of hysteria around Internet security is a good thing. Unfortunately, in January, Microsoft decided for the first time to make an exception to its once-a-month patch policy. And in doing so, they open a Pandora’s box.

Once you start issuing out-of-band patches, two things happen: First, you effectively create a new security category, above and beyond Critical. For the sake of argument, let’s call these patches Super Critical. Second, you encourage people to begin agitating for each Critical update to be reclassified as Super Critical, and you encourage the irresponsible-disclosure community to release their proof-of-concept code the day after Patch Tuesday, so they can force this sort of arbitrary reexamination and drive Microsoft nuts. And in the process, you force your security team to start coordinating security bulletins with the PR department. Bad idea.

Why is this subject in the public debate right now? It came up last week in George Ou’s post about Secunia’s disclosure of an unpatched zero-day exploit for IE6. George wrote:

This IE6 vulnerability is serious enough that Microsoft should immediately create an out-of-cycle patch before the next monthly patch and spend less time lecturing about Apple's missteps. Microsoft was forced to release an emergency patch for the WMF vulnerability in January. Waiting for next months cycle for a zero-day critical flaw is unacceptable. [emphasis in original]

And he’s not alone. Here’s a sampling of the scary headlines that appeared at the top of Google News last week:

And you could read 70 more just like those if you kept scrolling.

The outraged press wants a patch. They want it now. They got Microsoft to cave in January, and this situation looks identical. Several commenters on George’s post connect the dots and suggest that the monthly patch cycle is part of the problem. Commenter Tic Swayback (heh) spoke for the majority when he asked this eminently logical question in the TalkBack section of George’s post:

This is something I've always found confusing. Why is it better to have to wait on a patch at all, whether it's critical or trivial? If a company wants to patch their own machines on a regular, monthly schedule, why can't they just do that? Why must the software provider hold back on patches?

George responds:

If there is a proof-of-concept out in the wild, release the patch ASAP. Other than that, keeping an orderly patch schedule is a good thing. …

IT departments barely can keep up with the monthly cycle. There is a lot of testing evolved before IT departments will implement any patch. If you drop the orderly cycle and you have patches every week or just random, it will result in chaos. What’s needed is a flexible approach that if a critical exploit is responsibly disclosed without enough details or PoC, then the best thing to do is leave the cycle intact. But if a PoC is already available then we need to break out of the cycle because it’s an emergency situation. This IE6 flaw is an emergency.

Sorry, I have to agree with Vic and disagree with George. Microsoft management made the same argument in favor of releasing an out-of-band patch for the WMF exploit in January. In that case, as in the present instance, proof-of-concept code had been (irresponsibly) released, and Microsoft’s Security Response Center had to scramble to fix it. According to Mike Nash, Corporate VP for Security, the patch was released five days earlier than Patch Tuesday because corporate customers demanded it:

I reminded them of their past feedback about out of band updates being an inconvenience and their preference for the monthly release schedule. Overall, they felt that we had made these out of band releases so infrequent, that doing it once when it matters was not a big deal.

That’s the trouble with “doing it once” – people expect you to do it again. In this case, less than three months later. And then next month. And the month after that, and so on, and so on, and so on…

My question is simple: Why should any Critical update be delayed? When it’s ready, release it with a preliminary Security Bulletin. Continue to publish the fully edited, fully vetted collection of Security Bulletins on the second Tuesday of the month, but give security-conscious Windows users and well-organized IT admins the chance to protect themselves ASAP. Overtaxed IT administrators can choose to wait till Patch Tuesday; you shouldn’t have to.

So, Microsoft, forget about agonizing over whether to wait for Patch Tuesday or release a Critical update out of band. When you hear about a new vulnerability (especially if exploit code is available), jump on it. Make the patch. As soon as it’s ready, release it.

We won’t mind, honest.

Topic: Security

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • I called for an out-of-band patch

    "Sorry, I have to agree with Vic and disagree with George. Microsoft management made the same argument in favor of releasing an out-of-band patch for the WMF exploit in January."

    What's the difference, I'm essentially calling for the same thing since an exploit is available. Microsoft should only go out of band when the PoC is out in the wild, they should hold on to a patch for the next cycle if an exploit is not out. Microsoft is actually in agreement with me, they're not proving your point because of that WMF exploit. I've spoken to Microsoft about this exploit and they've already told me that they're considering an out-of-band patch.

    You have to weigh the risk of expert hackers having an open exploit to the risk of reckless worm writers who get a kick out of doing maximum disruption. Patches are like vaccines, and that vaccine will cause collateral damage because people will reverse engineer the patch as soon as it's available. The patching debate is not a simple one, we have to weigh our options carefully.
    • Out-of-band must be exceptional

      "You have to weigh the risk of expert hackers having an open exploit to the risk of reckless worm writers who get a kick out of doing maximum disruption."

      Right. I could see script kiddies doing that just in order to jerk MS's chain.

      Also, by releasing details of any ol' vulnerability out of cycle, forces corporate IT managers to fix ASAP otherwise the now well-known vulnerability could result in an exploit before their next monthly corporate fix installment.
    • Reactionary vs. proactive

      I'd rather reward the proactive than let it be a crap shoot as to who gets exploited.

      The problem with your method is that many computers will have been exploited before the the second Tuesday of the month. What do you say to those people? And suppose a vulnerability becomes known on that second Tuesday after the patches are released. That gives virus writers a full month to create an exploit and get it out in the wild. Even if that takes three weeks you'll get a full week of mayhem.

      Whereas if MS is proactive and releases the patch as soon as it's tested and ready, at least the proactive administrators have a chance at protection. Personally, I'd rather have protection be in my hands rather than Microsoft's (and I don't say that as an ABMer, I'd say the same thing about any company). At least then if I got exploited the only place I could point any fingers is at myself.

      Now obviously you have to weigh the good and the bad. If there is an exploit found that can only work in some obscure and unusual configuration, then perhaps it would be best to wait for the cycle. But if a vulnerability exists for the default configuration, or a well used "safer-than-default" configuration (like the configuration you use for your family), then it needs to be released as soon as it's ready whether or not a zero day exploit is already known to exist. Because for all we know an actual exploit may be set loose in the next five minutes. And it should be given the highest of priorities within the company until it is ready (as I'm sure MS has given this vulnerability).
      Michael Kelly
    • The real reason for patch Tuesday

      Ever noticed that patches for critical flaws almost always require rebooting Windows? That's more of an annoyance with workstations but for servers it's a big problem. Rebooting servers 4 or 5 times a month makes administrators very angry and they start looking more fondly at their Linux boxes. And that is something Microsoft surely doesn't want.
      • Nope the real reason is

        that a couple of years ago Windows was hit by severe attacks almost every second day. It was 'This weeks severe vulnerability', hitting all versions of Windows.
        MS was attacked severely and rightfully by the press and everybody else, and the situation was really bad, with confidence in MS products plummeting. Then came the 'thrustworthy computing' initiative and the scheduled releases of fixes. I can't recall ANY OS manufacturer ever of having to market the phrase 'trustworthy computing' in order to try to reestablish faith in their product.
        Naturally the monthly schedule - and I think it was bi-weekly in the beginning- leaves a sizeable Windon (pardon the pun not intended) open for attacks. The WMF disaster caused by a 25 year old technique form the stand-alone time being used unaltered in a networking world, was fixed by someone else than MS. Someone still have to explain to me why a program for viewing pictures must be able to handle code other than it's own.
  • Make up your mind. I still get paid.

    When Microsoft was issuing patches as they became available most everyone in the field was screaming bloody murder that Microsoft was being inconsiderate for the Corporate and Filed techs that had to go around and patch systems all the time and were not able to get their regular work done.

    Now, you're screaming that Microsoft needs to re-think their strategy. You forget that the current strategy was your strategy in the first place; not Microsoft's. Microsoft was only responding to the demands of the field industry.

    Personally, as a field guy, either strategy works for me. I still have to visit my customers once a month to ensure the patches are applied. I still get paid as a result of the Microsoft Money Machine.

    Perhaps someday, Microsoft will put me out of business by making a zero patch OS. I think not.
  • This is one of the reasons

    This is one of the reasons when some one ask me what to use for a O/S on a Server I did not even slow down when I said LINUX ! And gave them a gob of reasons to do it . . And the shut down to finish a UPGRADE is one of them ! . . I some times wonder if MICROSOFT was in league with virus writers as they only came out with patches after a virus or a worm hit their system and did strange things to them . .
  • Apache OS

    This would be a good name for Vista. as it will have been bread from a patchy OS XP. I feel this is the end of times for bill gates as more and more turn to open source. IBM with Webspere Novell with Suse Linux and Sun with Solaris and they just work better with less maintance than windows. Easier to update. and will even network with windows if you must. virtualization is better and everything is comming to age for a dramatic end to the MS empire. Patch that!!!!
  • patches

    why dont we just throw windows out the window and go back to good ol DOS!!!
  • Patches

    I have a differant take on this.
    The way I see it is Automatic enabled machines should be updated as needed.
    No need to wait for the end of the month.
    Most home machines are set up this way and these machines have the less tech savvy users and as a consequence are the most dangerous to all.
    It must be worthwhile to patch these machines as soon as possible.
    Corperate machines should be updated on a schedule created by indevidual IT departments that can go to the download site and download all the patches in one go.
    This way they have the control they need over the process.
    Maybe a site set up specifically for companies which run group licenced machines.
    Now my logic tells me that in theory some patches could take over 5 weeks to be put out so for the less secure machines Iwould want immediate patching as and when one is released.
    How does that sound to you professional IT,s out there.
    • I Agree,But Most Likelly Won't Happen

      Your Logic Seem Ok To Myself,But As For
      Microsoft Inc. We Would Not Be At This
      Point If Real Security Was There Top
      Priority. As A End User I Can Go, Any Way
      Microsoft Inc. Wants To Run With The Way
      They Issue Patches. But I Would Like To
      Receive Any Patches As Soon as They Are
      Ready. That Way I Can Just Apply Them
      When I Am Ready, And Like You Have Said
      Everyone Can Do What Ever They Want To.
      But That Would Put The Final Control
      Out Of The Microsoft Inc. And That As
      I See It, Is Why Its Not Likly To Happen.
      But as I Have Said Your Logic Seem Very
      Good For All Involved. Any Other Logic
      Would Be To Just Put The Control Out Of
      Those Involved, Back Into The Hands Of
      Microsoft Inc. So They Would Be Able To
      Shift Any Fault, For Not Applying Any
      Available Patch's Back To Microsoft Inc. And
      It Seem To Be The Way Microsoft Inc. Wants
      To Do It.

      M. K. Duffy.
    • An even better way

      would be to dump the PC entirely for most business users. Maybe even switch to something else than MS products. Ah - the steep learning cycle of going from f.inst IE to Firefox, or from MS Office to StarOffice. How about the learningcurve from W/2000 to XP - or to Vista with a completely new GUI ?

      If your users are so lame that they only can use the MS Office package, then install a couple of Citrix servers, and run it from there and give the user a thin client.

      This will
      Reduce the number of machines needing upgrade dramatically.
      Reduce the number og badware entry ppoints by the same scale.
      Reduce maintenance cost markedly.
      Reduce support costs and time dramatically.
      Ensure that everybody runs same version and patch level of all SW.
      Cut noise and heat emission a lot in the workspace.
      Cut start-up times by a factor 5 - 10.
      Cut shutdown times even more.
      Make a change to other products very easy, as you only have to install on the servers.
      Make updates just as easy for the same reason.

      I did that, and I would never even dream of giving the usual office user a PC. The PC is obsolete years ago, it's the biggiest dinosaur in the IT world.
  • Not a stockpile

    This article is rather ridiculous and I?m surprise it even gets to be published.

    Microsoft is not stockpiling. If you know anything about software design and patches, you know that you can easily introduce more bugs than you fix if you don't do your homework testing them. Let me just say, I?m a fan of using rapid development approaches to speeding up fixes and reducing release overhead of software. I?m a fan of refining this process to keep best-practices best, because ?best? changes everyday.
    And let me admit that Microsoft had a long history of sucking at making their software secure or reliable. But times have changed.

    For starters, let?s note there is no mention of ?testing? or ?quality? in this article, except George?s mention of customer-end testing. That shows the lack of understanding that is at best elementary of any role in software design. Relying on customers to test your software is negligent or even dangerous. Shall I assume that testing software before it?s released is something Ed knows little or nothing about? Maybe he knows something, but doesn?t find it relevant. He mentions ?When it?s ready, release it??? but he doesn?t seem to understand that there isn?t a little elf in the back that brews up a fix, and then ?It?s ready?. That?s the whole question: When IS it ready? What defines that it IS ready? How ready? Again, this is all about quality and testing.

    When you?re managing a mammoth of over a hundred of million lines of code with dozens of production versions and, legacy versions and dozens of languages, process doesn?t make it better, it makes it possible.

    With hundreds patches in the pipeline all the time, it's better to test them before you release them and at the same time if you can afford to, if the released software can afford the time. Testing takes time and you can only speed this up by adding resources to a certain point. This is really a black art of knowing when to make an exception and bypass the pipeline, and how to bypass it. If the patch is so important that it's worth releasing early because a worm is rapidly spreading through a majority of online systems without mercy, then certainly tips the scales towards releasing early. With all good business processes, there are exceptions, up to and including leaking fixes, as we have seen.

    The above the fold teaser "It doesn?t solve a problem for any customer, and it exposes an unknown number of innocent Windows users to unnecessary risk" is really just a one-dimensional, naive perspective to building and supporting quality and secure software. It?s either an ignorant view on software design or it?s a weak attempt to grandstand, or maybe both. The purpose for patching software is to improve quality; do it wrong and your better of not patching at all.

    Really this is about prioritization, good quality control, good testing, and a continual review and refinement of best practices. Not just saying ?Hey, stop hoarding those fixes, Microsoft?. This paints a picture of scores of poor Windows users limping about, malnourished, diseased, with bloated bellies and bugs in their hair, holding their mice and keyboards, too weak to bat way the bugs in their hair all sadly slumped over and below a massive glass desk with a sign ?Microsoft monthly patch release desk?, behind which lie the antidotes to their ailments, sealed in shrink wrap, locked in a big metal cage, and a big sign that says ?Out for golf, will be back in 28 days?. Let?s get real.

    If you believe there should be more exceptions that bypass this cycle, then you need to look at the downsides to this. If you give everyone the opportunity to determine if they are their own exception, then you need to examine the scenarios and configurations that are being used to determine the risk of destabilization for every customer. Just the time to determine what these are would take more time than to test them.

    That being said, customers, especially large enterprise deployments, prefer to have a mechanism that allows them to checkpoint what patches are installed and decide if they want to test them and deploy them. Having an iterative (and exception driven) business process for release management of patches enables enterprises to align with Microsoft's patch release system and it works pretty well. Could be better, but it?s better than anything I?ve heard here.

    The only thing I can think of is finding ways to shorten the cycle, but this is not a policy driven thing, this is a resource thing and not one that you can simply squirt more cash and manpower at to solve. If you look at the amount of testing and release management that must go into something as massive as Windows, taking a 30 day hotfix release cycle down to 20 days or less, would not come without compromise to something and my bet is that it would cost in quality and would cost in confusing the customers on what patches are critical and when. If the customers get confused, the ultimate result is lower quality and that defeats the purpose of the entire problem.

    I think Microsoft is doing a pretty good job, and this is not about stockpiling. This article seems more like an ignorant whine rather than a constructive call to action for improvement.
    • Agreed -- article is idiotic

      I would argue it DOES solve a problem and the concept of assisting customers in resource planning works! At least it has in my case.

      I must be missing something -- you seem to shoot down your own argument that Microsoft needs to re-think this.
      "That?s not necessarily a bad thing. Critical updates by definition are, well, critical. But that doesn?t mean the world is going to end if a Critical patch is delayed by 5, 10, or even 30 days. Reducing the level of hysteria around Internet security is a good thing."

      It is not a bad thing and most critical patches can wait.

      It is fine to have workstations setup to automatically update. But when trying to patch several servers (schedule reboots, etc...) -- once a month is enough.

      Along with all the issues indoctrin8ed adequately brought up about testing, etc...
    • So you think...

      that if a patch needs "testing" and it's not ready to release until the second Wednesday of the month that they hold it for the next scheduled patch? Looks like they're sitting on it for a month to me.

      Personally I think that when they have a patch ready it should be released. I'm a big boy...I can make a decision for myself...All it takes is being willing to be responsible for your own actions.
    • At least your name fits....

      "And let me admit that Microsoft had a long history of sucking at making their software secure or reliable. But times have changed."

      Have they? Recent events don't give any indication of that.
  • Patch Installation Scheduler Proposal

    I suggest instead that MS update their Security Center - Automatic Updates "Behavior", and offer clients extended choices that would enable:
    1. "Instant Update" of all (Selectable?) Critical patches, or
    "Bundled Update" of all (Selectable?) Critical patches,
    2. "Install Date/Time" support for those choices.
    3. Clearly define the default, unmodified behavior in advance to ensure the "no surprises" optimum impact.
    This Win/Win proposal will enable/compel MS to deliver fixes ASAP, and yet allow clients the option to control (their IT) Implementation Schedules.
  • Its B$=M$ Markerting

    It is all about marketing. Obviously with windows automatic update under default configuration connecting everyday, not once a month, it makes not logistic sence. In a pure marketing sence, it means that updates and pacthes for their operating system stay out of the news most of the time, so instead of daily faults and patches, it only appears once a month. The second benefit, it gives the appearance of being regular and routine, like a bowel movement, nothing special, just normal computer operations. That's the lie of course, in terms of computer security it is another, Bill Gates security memo catasrophic failure, in terms of typical ballmer billy goat marketing, so what if it substantialy increasing the potential for harm to the customer's computer systems, it's better for sales.
  • Microsoft is bad

    Firstly let me say that, whatever the outcome of this debate, M$ is wrong, bad and evil. It is obvious that Bill $$$ Gates has personally released this PoC early so he can release the patch early to make the Earth hail his power.

    <you get the idea, i'm taking the p...>

    I have to agree with George on this one. There are corporate tools available to deploy updates and they can be configured to hold back. SUS allows for group policies to control when and who gets patches, and patches can be "approved" prior to release across the corporate network. Using SUS, out-of-band patches can be held for release with the rest of the monthly updates ***IF REQUIRED***.

    With this kind of control it really doesn't matter if MS decide to release OOB patches. Home users can immediately install the patch to protect from the PoC in the wild, and corporate users can decide when and where to deploy, and hold back if they need to.
  • My antivirus updates daily... M$, take notes, there's a quiz later.

    I use Avast! AV. One of the first things my PC does when I turn it on is update the virus database. In fact, that's the one thing I let my system do before going out on the net.

    If a company like ALWIL Software can afford to do daily updates, the deep pockets of M$ can do the same thing. Actually, the default setting of Automatic Updates makes XP look for updates [b][i]daily[/i][/b]. Why not take advantage of that? Once a month is just too long a period to go without updates, and can lead to PMS.

    (Permanently malignant systems) ;)
    Mr. Roboto