Microsoft fires back at critics of its HTML5 strategy

Apple and Microsoft are at it again. This time, though, the two archrivals find themselves on the same side (more or less) of a tremendously contentious issue: Which video format will be adopted as the standard for the Internet over the next five (or more) years?

The answer from both companies is H.264. Coincidentally, both Apple and Microsoft issued manifestos announcing that support last week. But how they continued that discussion with developers, partners, and customers is a very different story indeed.

Steve Jobs’ Thoughts on Flash was published on Thursday morning, April 29, although the signature beneath the post simply reads “April, 2010.” It’s mostly a double-barreled blast at Adobe in general and Flash in particular, but references to HTML5 and H.264 are sprinkled throughout the 1681-word post. It’s abundantly clear that Jobs and Apple have placed their bets on H.264: it’s a “more modern format,” and H.264 videos “play perfectly” in Apple’s browser and “look great” on Apple hardware.

On Thursday afternoon, almost lost in the media frenzy over Jobs’ remarks, Microsoft’s Dean Hachamovitch, General Manager of the Internet Explorer division, hit the publish button on a post titled HTML5 Video. At a mere 364 words, Hachamovitch’s remarks got straight to the point:

The future of the web is HTML5. … The HTML5 specification describes video support without specifying a particular video format. We think H.264 is an excellent format. In its HTML5 support, IE9 will support playback of H.264 video only.

Despite the similar content, there was one dramatic difference between the two posts. Jobs’ remarks did not include an option for feedback. Hachamovitch’s post did. And as of Sunday evening, roughly 72 hours after the original post was published, it had attracted nearly 200 comments, some of them downright scathing. In addition, tech news sites and blogs offered all sorts of reactions to the post, many of them wildly wrong. So Hachamovitch did something almost unheard of: he published a new post, Follow Up on HTML5 Video in IE9, addressing many of those comments in detail. (When I asked Hachamovitch last night why he took the time to prepare such a detailed response, he told me: “At the end of the day, we’re building a browser for the Windows customer. Listening to that customer, in whatever form that takes, is not just important, it defines what we’re here to do.”)

I’ve been researching this issue for several weeks now, so I was especially interested in what both companies have to say—and equally interested in the parts they leave out of the discussion. Here’s a summary of some of the key issues in this very controversial discussion. (And if you're wondering just who the mysterious MPEG LA organization is and why they control the "patent pool" for the H.264 standard, jump to page 3, where I explain.)

Why H.264?

Microsoft delivers software on a scale that is breathtaking. A billion PCs running Windows means a billion copies of one version or another of Internet Explorer. Making architectural decisions for a platform of that size isn’t something that’s done lightly. Apple’s installed base is considerably smaller, but it’s still large, especially when you factor in devices like iPhones and iPads, and its influence among the tech elite is much larger than its market share. For both companies, the decision to embrace H.264 is down to the same two reasons:

First, as Hachamovitch points out, it works—and works well:

[W]e think it is the best available video codec today for HTML5 for our customers. Relative to alternatives, H.264 maintains strong hardware support in PCs and mobile devices as well as a breadth of implementation in consumer electronics devices around the world, excellent video quality, scale of existing usage, availability of tools and content authoring systems

That performance edge isn’t just from software, either. Just about every modern graphic processing unit (GPU) now has H.264 decoding built into the silicon, and IE9 is going to take advantage of hardware acceleration for graphics and text. Jobs cited performance tests showing that hardware-accelerated H.264 video doubled battery life compared on an iPad compared to the same video not using hardware acceleration.

Second, and perhaps more importantly, the H.264 format has undeniable momentum. Hachamovitch pointed to one recent study from Encoding.com, which estimates that 66 percent of all videos on the web are now available in H.264 formats, up from 31 percent a year ago.

Next: What about those other codecs? -->

<-- Previous page

Does native support mean other codecs are locked out?

Short answer: No. Longer answer: Microsoft is huge, but its resources aren’t unlimited. In building IE9, Microsoft’s developers have to decide which codecs get native support and which are going to be supported via plugins. The same is true at Apple. That leads to some confusion, like this analysis from my ZDNet colleague Dana Blankenhorn, who concludes “Microsoft will only support H.264.”

That’s not completely accurate. At least for IE9, the decision to support H.264 natively in the browser doesn’t preclude third parties from adding their own codecs and plugins. And it doesn't prevent Microsoft from adding support for other codecs in the future. As Hachamovitch explains:

To be clear, users can install other codecs for use in Windows Media Player and Windows Media Center. For web browsers, developers can continue to offer plug-ins (using NPAPI or ActiveX; they are effectively equivalent in this scenario) so that webpages can play video using these codecs on Windows. For example, webpages will still be able to play VC-1 (Microsoft WMV) files in IE9.

And, of course, the Windows platform supports other browsers besides Internet Explorer. Firefox, Chrome, Safari, Opera, and a half-dozen smaller alternatives are available for Windows, each offering support for all sorts of alternative media formats. None of those will be blocked or disabled when IE9 is released.

So what about Ogg Theora?

Ogg Theora is an open-source video codec developed by the nonprofit Xiph.org foundation. Open-source advocates are lobbying aggresively for standards bodies to include Theora in the HTML5 standard, with only limited success. One insider told me, with an air of exasperation, “Ogg is a religion, not a codec.” The debate plays heavily on the confusion between open standards and open source. It also relies on fusion between “free” standards and open standards. As Apple’s most articulate outside advocate, John Gruber, points out: “Not all open standards are free.”

But even without official inclusion in the HTML5 standard, Theora isn’t being locked out, at least by Windows. As I explain in a companion post (see Ogg versus the world: don’t fall for open-source FUD), Mozilla has implemented Ogg Theora support natively in Firefox, and Google has done the same in current versions of Chrome. The Java VM supports Theora playback in Internet Explorer and Safari. The free VLC player, which was developed with help from Microsoft, supports the format, and a set of downloadable DirectShow filters add support for all Ogg formats to Windows Media Player.

The subject is obviously a minefield for Microsoft. Hachamovitch’s posts talk delicately around the controversy over Ogg Theora, never mentioning it by name but linking to articles that discuss the issue in great detail. So why won’t it be natively supported in IE9? Legal issues, mostly, which I’ll address on the next page, when I talk about the patents behind the AVC/H.264 standard.

Is Flash dead? And what about Silverlight?

I read one story last week from a reporter who skimmed over Hachamovitch’s original post and concluded, wrongly: “Microsoft has confirmed that its upcoming Internet Explorer 9 browser will not support Flash.” When I asked a Microsoft insider for an off-the-record reaction, It took a minute or two for the laughter to die down. No, I was told, Microsoft is not going to block Flash. “That would be insane, and we would never do that to our customers.”

In fact, it would be almost impossible for Microsoft to eliminate Flash or Silverlight, even if it wanted to do so. Corporate customers and third-party developers rely on third-party add-ins. It’s a key part of Internet Explorer and it’s not going away. There’s plenty of legacy content that uses Flash-based video players, and Silverlight will continue to have scenarios where it makes sense as a video playback engine—for subscription-based content for example, as in the recent Winter Olympics. And both Flash and Silverlight are application platforms as well, not just video players.

So no, neither one is going away anytime soon. Silverlight in particular has a bright future.

Next: Who is MPEG LA, and why do they control H.264? -->

<--Previous page

Who is MPEG LA, and why do they control H.264?

Let’s start with a truism: Software patents are complicated, even for lawyers. And despite the most fervent wishes of the anti-patent crowd, they're not going away any time soon. The consequences of infringing on someone else’s software patent can be catastrophic, as Microsoft found out recently with an expensive loss to a small company called i4i. If the judgment survives the appeal process, Microsoft will have to write a check to i4i for roughly $290 million. Ouch.

So who holds the patents for the AVC/H.264 standard? Those are administered by an independent group called the MPEG Licensing Administrator (MPEG LA). As it turns out, both Apple and Microsoft are part of the patent pool for the H.264 standard, which naturally leads to conspiracy theories. Given the inherent distrust of both companies, I’ve read speculation that MPEG LA is a clever conspiracy to freeze out small companies, including open-source developers, and ultimately allow only big players with big bankrolls to play in the online media game.

That’s an interesting line of speculation, but here’s what you haven’t heard about MPEG LA from any of those conspiracy theorists.

First, a little history: MPEG LA is no newcomer. It was founded in the 1990s as a consortium to administer a group of “essential patents” related to the MPEG-2 standard. The original members (as listed in a 1997 article not available online) were Columbia University, Fujitsu, General Instrument, Lucent Technologies, Matsushita, Mitsubishi, Philips, Scientific-Atlanta, and Sony. All of those groups own patents related to the MPEG-2 standard; if consumer electronics companies and software makers had to negotiate separate license deals with each one, the complexity would be overwhelming. So the individual organizations turned over their patents to the pool managed by MPEG LA, which gave potential licensees a single point of contact for negotiation and payment of royalties.

Over the years, MPEG LA has expanded its charter significantly and now manages multiple patent pools. A July 2003 press release revealed the lineup for the “[p]arties with patents or patent applications determined by MPEG LA’s patent experts to be essential to the H.264/AVC standard”: Columbia University, Electronics and Telecommunications Research Institute of Korea (ETRI), France Télécom, Fujitsu, Matsushita, Mitsubishi, Microsoft, Motorola, Nokia, Philips, Polycom, Robert Bosch GmbH, Samsung, Sharp, Sony, Thomson, Toshiba, and Victor Company of Japan (JVC).

The AVC/H.264 patent pool as it exists today has 26 members, including Apple and Microsoft and a slew of consumer electronics manufacturers. You can see the entire list here. But when it comes to the actual pool of patents, neither company is a big player. I downloaded the complete list of patents managed by the AVC/H.264 patent pool. The PDF document goes on for 47 pages. By my count, it encompasses 1,137 patents. Only one of those patents is from Apple, and another 65 patents are from Microsoft (less than 6% of the total).

With those facts in place, a different picture becomes apparent. If the MPEG LA patent pool didn’t exist, Microsoft and Apple would have to negotiate with 25 separate companies to license patents for modern, hardware-accelerated video playback. They’d also have to sit on the other side of the bargaining table from every consumer electronics company that wanted to build a device that used any of their patents. Pooling the patents eliminates the hassle of those negotiations and also adds some confidence that a single patent holder isn’t going to come out of the woodwork with a potentially devastating challenge.

That gigantic collection of patents is also, ultimately, a cudgel that can be used against the developers of Ogg Theora. As I noted in my other post, the CEO of MPEG LA, Larry Horn, has done his part to strike some fear, uncertainty, and doubt into the hearts of the open-source community with this statement:

[N]o one in the market should be under the misimpression that other codecs such as Theora are patent-free. Virtually all codecs are based on patented technology, and many of the essential patents may be the same as those that are essential to AVC/H.264. Therefore, users should be aware that a license and payment of applicable royalties is likely required to use these technologies developed by others, too.

The Free Software Foundation insists that the Ogg formats are “designed to be completely free of patents.” But as my UK-based colleague Mary Branscombe pointed out in a comment on that post, “If only confident claims that patents don't apply were enough to ensure that patents don't apply...”

And if you think this is a money-grab on Microsoft’s part, Hachamovitch argues that the opposite is true:

Several comments speculated about Microsoft’s financial interest in the codec. (Microsoft participates in MPEG-LA with many other companies.) Microsoft pays into MPEG-LA about twice as much as it receives back for rights to H.264. Much of what Microsoft pays in royalties is so that people who buy Windows (on a new PC from an OEM or as a packaged product) can just play H.264 video or DVD movies. Microsoft receives back from MPEG-LA less than half the amount for the patent rights that it contributes because there are many other companies that provide the licensed functionality in content and products that sell in high volume. Microsoft pledged its patent rights to this neutral organization in order to make its rights broadly available under clear terms, not because it thought this might be a good revenue stream. We do not foresee this patent pool ever producing a material revenue stream, and revenue plays no part in our decision here.

The most disquieting clause in the MPEG LA agreement is the time limit. The current agreement expires on December 31, 2010, at which time it will be renewed for another five years. The fear among those who don’t trust Apple, Microsoft, or MPEG LA is that after five years of low royalty payments, the consortium will spring a trap in 2016, jacking rates sky-high and forcing licensees to pay or stop playing. Could that happen? Sure, it could. Is it likely? Here’s what Hachamovitch had to say on the subject:

The majority of H.264 video content on the web today is royalty-free.   MPEG has said that individuals can create video files in the H.264 format and distribute them and play them over the internet for non-commercial purposes without further obligation on licensed platforms like Windows. We are aware that this commitment is set to expire in 2016, but fully expect to commit to supporting the extension of this license and associated terms beyond that date. In general, distributing encoders or decoders or offering sophisticated pay-for-video requires a license from MPEG-LA.  Third-party applications that simply make calls to the H.264 code in Windows (and which do not incorporate any H.264 code directly) are covered by Microsoft’s license of H.264. 

In other words, it’s in Microsoft’s interests to keep license fees at or near their current levels. And given the business model of MPEG LA, it’s not in their interest to suddenly raise licensing fees. But ultimately, that requires a level of trust that some people simply aren’t willing to give.

Update: The H.264 license terms have been updated to extend for another five years, through 2015, with substantially the same terms as the previous agreement. On August 26, 2010, MPEG LA announced that it would "continue not to charge royalties for Internet Video that is free to end users (known as “Internet Broadcast AVC Video”) during the entire life of this License." For details, see the press release (PDF).

Update 2: In January 2011, Google announced that it was removing support for the H.264 codec from future versions of the Chrome browser in favor of its WebM technologies (which I discuss on page 2 of this post). For a detailed discussion of the economics behind Google's decision, see By dropping H.264, is Google avoiding a trap or walking into one?