How Apple is sabotaging an open standard for digital books

Update: This post is part of a series. If you find this topic interesting, I recommend you read the two follow-ups as well:

• Some standards are more open than others
• Closing thoughts on Apple’s greedy, “crazy evil” iBooks license

Apple has built its iBooks platform on the back of an open standard. With last week’s introduction of iBooks 2.0 and the free iBooks Author software for Mac OS X, Apple is deliberately locking out that popular open standard.

Apple’s behavior is a modern, sophisticated version of the “embrace, extend, and extinguish” behavior that got Microsoft in so much trouble in the 1990s: Enter a product category supporting a widely used standard, extend that standard with proprietary capabilities, and then use those differences to disadvantage competitors. (The strategy is even more effective if you have a dominant market position in another, related category that you can use for leverage. Think Windows in the 1990s, iPad in 2012.)

If you read, write, or publish digital books, you should be concerned.

I’ve already made my feelings known about the iBooks Author license agreement (it’s “mind-bogglingly greedy”). The first reaction of Apple-watcher John Gruber, who knows the company better than anyone outside Cupertino, was that the new EULA represents “Apple at its worst.” (In a later post, he backtracked somewhat and wrote a complicated defense of Apple’s actions.)

It’s certainly Apple’s right to control access to its bookstore and curate its content, and if they had implemented this policy with some thought, I could even have supported their decision.

But I can’t. Here’s why.

The iBooks format is ePub (or, as it’s formally known by its caretakers, EPUB). Here’s a snippet from the iBooks FAQ, as last updated December 22, 2011:

In the original version of the FAQ published in April 2010, when iBooks was launched, Apple was even more definitive about the format: “iBooks only uses books published in the ePub format.” An Inside iTunes page written by Apple at the same time is still available online. It states in no undertain terms that ”the iBooks app uses ePub, the most popular open book format in the world.”

Apple is quite proud of this fact, even bragging in the current version of the iBooks FAQ about its support for “the industry-leading ePub digital book file type.”

So, for nearly two years, Apple has wooed digital book publishers and authors with its unconditional support of an open, industry-leading standard. (The EPUB standard is managed by the International Digital Publishing Forum [IDPF], of which Apple Inc. is a member.)

With last week’s changes, Apple is deliberately sabotaging this format. The new iBooks 2.0 format adds CSS extensions that are not documented as part of the W3C standard. It uses a closed, proprietary Apple XML namespace. The experts I’ve consulted think it deliberately breaks the open standard.

Related posts:

• Apple’s mind-bogglingly greedy and evil license agreement
• Amazon: “Primed” to disrupt Apple’s textbook plans?
• Apple announces iBooks 2, Textbooks, iBooks Author
• The poor get poorer and the rich get richer with Apple’s iPad-based textbooks
• Apple’s iBooks Textbooks initiative is a welcome and natural progression
• Apple’s textbook plan’s biggest flaw is that it’s tied to the iPad

First, there’s the issue of mimetype, which defines how an ebook reader parses a digital book file.

Baldur Bjarnason, an expert on digital publishing who earned a PhD for his work on ebooks and interactivity, says “Apple’s new format is mostly ePub3″. But that “mostly” is problematic:

Apple’s new format is mostly ePub3. It has valid NCX and OPF files. The XHTML files are all XHTML5. It uses SVG extensively.

The mimetype iBooks uses for these files is application/x-ibooks+zip. In and of itself not bad news, but it is a clear indicator that Apple doesn’t want this to be treated like ePub. That is, they do not want to have to worry about making sure that the output of iBooks Author is readable in ePub reading systems.

Daniel Glazman, co-chairman of the W3C CSS Working Group, see the same problem:

It looks like an EPUB3 format. It smells like an EPUB3 format. But it’s not at all an EPUB3 format and here’s why…

First the mimetype file. It’s correctly placed in first position in the package, but the EPUB3 format states that its content must be application/epub+zip. And it’s not. It’s application/x-ibooks+zip and that is enough to make conformant EPUB3 readers choke on a *.ibooks package.

So Apple, which claims to use the EPUB format exclusively, has now created an incompatible, proprietary version of that format. And with iBooks Author they’ve added licensing terms that restrict what an author can do with the generated content.

The designers of iBooks Author went to great lengths to make sure that the program will not work with “the industry-leading ePub digital book file type.”

I tried creating a book using a formatted Word document and iBooks Author. When I changed the .ibooks extension to .epub, the book opened properly in the free, open-source Calibre e-book management program. The cover and table of contents were a mess, but the content itself looked just fine. So there’s no question that EPUB is still at the core of iBooks Author.

But the differences are substantial. Here’s Glazman’s analysis:

IBA is not EPUB3. A wysiwyg EPUB3 editor will not be able to edit correctly an IBA document because of the different mimetype and the proprietary CSS extensions. iBooks Author is not able to reopen [an] iBook it exported in their pseudo-EPUB3 format because there is no Import mechanism! That means that on one hand EPUB3 readers cannot reuse a document created by iBooks Author because of its HTML/CSS/Namespaces extensions, and on the other iBooks Author cannot create an iBook from an existing EPUB3 document because it cannot import it. But wait, can we open an EPUB3 or a regular HTML document into another app and copy/paste the content inside IBA? I tried from an HTML instance in Safari and from an EPUB reader based on Safari. It does not work, all markup is lost, it pastes text. Ugly result. Oh, and changing file extensions from ibooks to epub or vice-versa does not help either.

Bjarnason reaches a similar conclusion:

The differences between the iBooks 2.0 format and ePub3 seem all but trivial. But when that format is built around non-standard extensions to the CSS rendering model and all of the XHTML and the CSS are built around that extended model, the file is likely to forever be useless and unreadable in other reading systems.

Cynically, Apple is positioning this authoring tool and the new format as the savior of K-12 education. All school districts have to do is buy one iPad for every student and buy textbooks through the iTunes Store, and their problems are solved. Wrapping themselves in the education flag is a transparent attempt to win praise and deflect criticism.

What’s most infuriating to me about all this is that Apple had an opportunity to play fairly and still win. If the interactive capabilities in the new, enhanced iBooks format are so compelling, the resulting books should be able to compete on their own in the marketplace.

Apple, which uses the EPUB standard as the core for iBooks, could easily have produced their free authoring tool so that it continues to support what they acknowledge is the “industry-leading standard.” The program could offer users a choice of output formats: a standard EPUB file or a fully interactive iBooks file.

They also could have included the option to import EPUB files. As a publisher and author myself, I would have welcomed that option. I could create a book using the industry-leading standard EPUB format, for sale in any outlet, then import it into iBooks Author, add interactive elements, and sell an enhanced version in the iTunes Store for the same price.

In that scenario, I as a publisher would have had multiple choices. My readers would have had multiple choices. Apple would have had the opportunity to prove in the marketplace that its new format was a superior option to the industry-leading standard. Why buy a boring old e-book when you can get an exciting interactive experience?

But none of those things are likely to happen. Instead, Apple has chosen to leverage its dominant position in the tablet market to try to hobble its competitors in the ebook-publishing business. As Bjarnason argues, products created using Apple’s new authoring software will “forever be useless and unreadable in other reading systems.”

This isn’t the first time Apple has appeared to embrace openness and then stepped back from it. In June 2010, at the WWDC Developers Conference keynote, Steve Jobs debuted FaceTime and made this promise:

FaceTime is based on a lot of open standards, and we’re going to take it all the way. We’re going to the standards bodies and we’re going to make FaceTime an open industry standard.

Some 19 months later, no such “open standard” has been proposed, and one observer has found evidence in iOS5 that Apple no longer intends to follow through with that public promise.

Antitrust regulators should be taking a long look at this pattern of anti-competitive behavior.