10 years since the Bill Gates security memo: A personal journey

10 years since the Bill Gates security memo: A personal journey

Summary: Ten years after the famous Trustworthy Computing memo, Microsoft principal cybersecurity architect Michael Howard shares memories from the Redmond security trenches.

SHARE:

Guest editorial by Michael Howard

I remember the security situation at Microsoft in 2001 and 2002 like it was yesterday. Perhaps no other couple of years will be so indelibly etched into my brain as those two. 2001 was not so good, but 2002 was a heck of a lot better! Given 2001, this was not a difficult achievement for 2002! So, let me start at the beginning...

In late 1999, a small band of us formed a small security team (as in “threats,” not as in “features”) to help raise software security awareness across the company. We had no name for a long time, until the vice president in Windows at the time, Dave Thompson, decided to call us the Secure Windows Initiative (SWI). Our charter was to start reviewing Windows code in depth looking

for security bugs, but having a small number of people reviewing something the size of Windows was clearly not going to work. So, we moved to a “Security Bug Bashes” model where we would deliver security education in the morning to a small development group within Windows (e.g., networking, terminal services, IIS, IE, etc.), and then for the rest of the day we would have the engineering team go look for security bugs. It was fun and we found bugs. But the most important point was raising awareness. It really didn’t matter how many bugs were found -- the key was to make people aware of the security issues and reduce the chance that mistakes would be made in the future.

follow Ryan Naraine on twitter The downside of the bug bashes was that even though they were more effective than the original SWI charter, they still didn’t scale very well and they were very labor-intensive. Still, the security bug bashes continued for about another eighteen months.

2001 was not a good year for Microsoft security because of CodeRed and Nimda, two worms that affected Internet Information Server 4.0 and 5.0. CodeRed was the result of a one-line error in some code running by default in IIS4 and 5. In hindsight, the code should not have been installed by default. Nimda was the more sophisticated of the two worms because it used more than one vulnerability to compromise systems.

While all this was happening, David LeBlanc and I were mid-way through creating the first edition of Writing Secure Code. We had written the book because the same security-related questions were being asked time and time again and we wanted a reference we could point people to. Little did we realize that Writing Secure Code would later become a runaway bestseller.

As 2001 wound down and Writing Secure Code was finally sent to the printers, I got an email from Loren Kohnfelder, who was one of the security leads in the .NET Framework. Loren is best-known for defining what is now commonly referred to as Public Key Infrastructure (PKI). You can read his 1978 thesis on the topic here. Loren was also one of the protagonists behind the STRIDE threat modeling mnemonic.

Loren told me that the .NET Common Language Runtime (CLR) team had uncovered a small number of security bugs during the final development phase of the project, and he was really concerned. We decided to do a bigger version of a bug bash; but rather than lasting only one day, it would be done when it was done. “Done” meant the rate of incoming security bugs approached zero. This became known as the “.NET Security Standdown,” and we even had T-Shirts made with the date of the start of the event. On the day the event was to start, the Pacific Northwest got a huge snow storm and the Microsoft Redmond campus was closed, so we started the standdown a few days later.

The standdown was a great success, thanks to Brian Harry and his team, who managed the process brilliantly. We reeducated the .NET engineering team, we found and fixed bugs, but most important, in my mind, we introduced the concept of reducing attack surface (i.e., limiting the amount of code exposed to untrusted users). That’s where the concept of AllowParticallyTrustedCallersAttribute (APTCA) came from and why we flipped ASP.NET to run in much lower privilege.

December 2001 saw the release of Writing Secure Code, and Doug Bayer and I had a lengthy meeting with Bill Gates to explain security vulnerabilities in detail. Clearly he was concerned by the worms of 2001 and wanted to learn more. At the end of the meeting I gave Bill a copy of Writing Secure Code.

At the end of December 2001, the .NET Standdown was over and we had learned a great deal about rallying the troops to a common security cause. But there was much more work to do!

In light of the success of the .NET work, we decided to aim our sights at Windows .NET Server (as it was called back then). Following the .NET model, we started in February and would be done when we were done. For the most part, that ended up being late March for most teams within Windows.

This became known as the “Windows Security Push.”

As everyone knows by now, Bill sent his famous Trustworthy Computing (TwC) memo to the company in January 2002, right as we were planning the security work for Windows. His memos are rare, and this one signaled the start of something big within the company.

During the push, we had three streams of education: I handled all the Windows developers, Jason Garms worked with all the program managers and architects, and Chris Walker trained all the testers. Steve Lipner and Glenn Pittaway led much of the day-to-day process management, keeping in constant communication with upper management.

One practice we borrowed from the security bug bashes was that we always had a senior person from management kick off the training. At one of my sessions, I had Rob Short, VP of Windows Base (Kernel down to the metal) open the day. Rob’s a tall, lean Irishman with a thick Irish accent, and there’s something he said that has stuck with me forever. He said, “There is nothing special about security; it’s just part of getting the job done.” Whenever I deliver a security talk to new engineers within Microsoft or am onsite with a customer, I always recite Rob’s words, because they are so incredibly true.

The Windows Security Push begat the SQL Server Security Push, the Exchange Security Push, and the Office Security Push. Slowly but surely things started to change across the company. Engineers and managers “got it.”

A key element of all the pushes was to reduce the default attack surface of the products. That’s why Windows Server 2003 (note the name change) had a reduced functionality browser, no Web server installed by default, and much more.

One thing that is not commonly known about the pushes is that a lot of documentation was written about the security implications of various technologies. Much of that learning ended up in the second edition of Writing Secure Code; the book ballooned from 500 pages to over 800 pages, and much of that was detail we learned and fine-tuned throughout 2002. A great example is the chapter concerning the security implications of internationalization and globalization. The text in the book is derived from a whitepaper written by the globalization team within Windows after they had gone through the push process and had looked at their important corner of Windows with a fresh security perspective.

The pushes were just the start, however. Real change came only when we implemented the Security Development Lifecycle (SDL). As I have said many times, you can’t build some software and then have a security push. It just doesn’t scale and, frankly, having a push at the end is too late. We needed something that was “part of the process,” and that is how the SDL was born.

There was a wrinkle along the way, however. In 2003 we saw Slammer affect SQL Server and Blaster affect Windows. Because one of the effects of Blaster was blue-screened computers, product support saw a huge increase in support calls. Many of us manned the phones to help out. Raymond Chen, a lead developer on the Windows shell team, and I were seated next to each other, and he wrote about it in his blog.

Blaster led to a lengthy and intense effort known as “Springboard,” led by Rebecca Norlander, Matt Thomlinson, and John Lambert. The end result of the process was Windows XP SP2, in which we not only found and fixed security bugs but also added numerous critical defenses to Internet Explorer, DCOM, and RPC. We also enhanced and enabled the Windows Firewall and added data execution prevention (DEP), and we made it easier for users to enable automatic updates by prompting them right after setup.

Microsoft has come a long way in the last ten years, and I am incredibly proud to have been a part of this watershed time. Much has changed. The SDL is now seen as industry-leading and is in use by many software developers outside of Microsoft. My role has changed too: I now work with our customers and partners as part of the Microsoft Americas Services Cybersecurity team to help them adopt SDL practices as they recognize the need for an increased focus on security.

It’s been an amazing ten years. We still have much to do, however. And no one knows that more than the incredibly talented people across Microsoft helping bake security into our products and our partners’ and customers’ products every day.

* Michael Howard is a Principal Cybersecurity Architect at Microsoft.

Topics: Software, Hardware, Microsoft, Operating Systems, Security, Windows

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

13 comments
Log in or register to join the discussion
  • This chapter was one of Microsoft's better moments

    I can only imagine the state of personal and business computing without these far reaching measures. Gate's memo couldn't have come at a more opportune time, considering the onslaught of malware that arose with the adoption of widespread broadband Internet access during that critical period. <br><br>Interesting to hear some of the finer details (and genesis) of how the process played out, one which culminated with the release of the security based XP SP2. A watershed time (and initiative) indeed.
    klumper
    • Takes me back

      Reducing the attack surface was a real insight (groan). <br><br>One day I'll get around to writing of my work at the time, coming from a Unix background, and the battle we had with the MCSEs and windows NT. The abuse we received for stating what had been common knowledge in real server environments for a long time. <br><br>Still MS changed and apart from the flood of windows PC originating spam and viruses the world is a better place because of the security teams work. We could only imagine what would have been without it. <br><br>Credit to the team, as bill gates famously said only 5 years earlier:<br><br>"There are no significant bugs in our released software that any significant number of users want fixed" ( justification: PC users would be prepared to pay for the higher standard )<br><br>Do we know if he he actually read the book given to him;-)
      Richard Flude
      • One day I'll sit down and write about all the annoying Unix admins

        @Richard Flude Sitting back and recommending with pay big dollars for Sun hardware, cause it was so great.

        Did some MCSE fire you or something? You really have an irrational hatred for these people.

        No technology is perfect. I have seen MCSE's who sucked, Linux loons who sucked, and smelly Unix admin neckbeards. Im too old for all of this nonsense. Nobody gets it right all the time, so lighten up.
        otaddy
      • RE: 10 years since the Bill Gates security memo: A personal journey

        @Richard Flude
        Interesting that it took Bill Gates 21 years after the first Microsoft OS to start prioritising security. Not a moment too soon.
        jorjitop
      • Adopting more secure code

        @Richard Flude
        [i]Reducing the attack surface was a real insight (groan).[/i]

        MS did come late to the party and that's a fact... but they did come. The guest author here played a big role in driving forth that welcome eventuality.

        @otaddy
        [i]No technology is perfect. I have seen MCSE's who sucked, Linux loons who sucked, and smelly Unix admin neckbeards.[/i]

        Spoken with eyes wide open. Amen brother [ditto: been there, seen that x2].

        @jorjitop
        [i]Interesting that it took Bill Gates 21 years after the first Microsoft OS to start prioritising security.[/i]

        You might want to revise your timeline to 19 years, in the name of historical accuracy. ;) Prioritization had changed by XP Vee-2, which they gave away for free by way of heightened expediency. It was then that MS *PROVED* they were serious -- to the twittering masses and sysadmins alike.

        It was also the time the groans and joke bait material largely began to subside. If you couldn't harden Server + XP's shell sufficiently with but a modicum of tweaks after that (to buttress safe computing practices 101), you really needed to head back to beginner's school.
        klumper
  • RE: 10 years since the Bill Gates security memo: A personal journey

    That was a really good read. Microsoft's products are now more secure because of the initiative of a few dedicated employees. They went from a reactive to a proactive approach.
    Loverock Davidson-
  • A somewhat contrarian view - default user privileges

    My first exposure to the Windows OS was my Gateway desktop running Windows NT 3.51 that I purchased in 1996. (I subsequently upgraded to NT 4.0 when it became available). Windows was installed on a FAT32 partition and I was dropped into the Administrator account. In order to create a non-Admin user and to run as that user, I first had to convert the partition from FAT32 to NTFS (this went flawlessly btw). But, I also had to *know* that I was running as the Administrator, how to convert to NTFS and how to create a non-Admin user account. [Note to the *Nix world: Windows NT had multi-user support from day 1.]<br><br>I replaced this desktop in late 2004 with an HP laptop running Windows XP SP2. When I first started the laptop I went through the setup routine and was subsequently dropped into the Administrator account. Since Windows was on a NTFS partition, all I had to do was create a non-Admin account. Again, I had to *know* that I was the Administrator and how to create a non-Admin user account. The default NTFS partition was a step forward. However, the default Administrator account on Windows XP (and 2000) caused many consumers (and more than a few enterprise users) to have significant problems with malware, even after the release of Windows XP SP2. Furthermore, many lazy 3rd party developers (and some Microsoft developers too) made running as a non-Admin user difficult to impossible without making modifications to files in C:\Program Files as well as making changes to the Windows registry. This may not be an insurmountable problem for the enterprise with their Windows sysadmins, but it *was* a big problem for ordinary users including consumers and small businesses.<br><br>Both Windows Vista and 7, after going through the initial setup, drop users into a standard user account. This is *not* the Administrator account or even an Administrator account. In the default account, the UAC prompt is responsible for mediating privilege escalation requests for system changes including updates and installing software to folder C:\Program Files. And the user simply clicks on a command button to allow privilege escalation requests. No further authentication is required as with the 'sudo', 'gksudo', etc. commands in the *Nix world. (And I'm not saying that sudo is without faults either.)<br><br>Windows 7 'turned down' the knob on UAC prompts relative to Vista. However, both security researchers and malware miscreants have found ways around the UAC prompt in the default account on both Windows 7 and Vista. Note: in spite of the questions and comments that follow, I do acknowledge that privilege management took a large step forward with the release of Windows Vista.<br><br>My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?<br><br>My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?<br><br>And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side? I think that it is. Running Windows Vista/7 as a "true" standard user, as opposed to the default user account, provides for enhanced security.<br><br>Michael, thank you for all that you (and others) have done for security at Microsoft *and* for the DropMyRights tool for Windows XP. I have it installed on all of my Windows XP systems. I never open documents (e.g., *.pdf, *.doc, *.rtf) as the Administrator in my Admin account.<br><br> <a href="http://blogs.msdn.com/b/michael_howard/archive/2007/08/13/update-on-dropmyrights.aspx" target="_blank" rel="nofollow">http://blogs.msdn.com/b/michael_howard/archive/2007/08/13/update-on-dropmyrights.aspx</a>
    Rabid Howler Monkey
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper
    • A few thoughts, should the author not respond

      @Rabid Howler Monkey
      [i]My question is why Microsoft chose *not* to require authentication for UAC in the default user account for Windows Vista/7 as *Nix does with 'sudo' et al?[/i]

      Because you are dealing with the pooter masses, who moaned rather loudly with the round one measures introduced in Vista alone. Access Control, it was learned (verified), can be a tricky thing (and thus the softeners deployed in W7). So is the Mother Hen annoyance factor you pass on to users, whether it be incidental or, uh hum, intentional. *doink*

      [i]My follow-on question is whether such authentication would cause more ordinary users to think twice about allowing privilege escalation requests than simply clicking a command button?[/i]

      Yes, but also complain more of what they might perceive as "overkill" -- namely user credentials and/or two-factor authentication via passphrase, password, captcha, and beyond. Maybe advances in thumbprint, voice or retinal scanning recognition will someday simplify such things (and whatever else might lie around yet to be determined corners).

      [i]And, finally, is UAC under the Windows Vista/7 default user account still too heavily weighted on the convenience side?[/i]

      Yes, but understandably so (see previous responses). UAC remains a compromise solution at best, done in the name of finding acceptable middle ground for their targeted (= greater, and not select, or more savvy) constituency. It's constituted to meet a critical threshold called 'market acceptance.'
      klumper