Whatever happened to Microsoft's Midori operating system project?
It's been more than a year since I last wrote about Microsoft's skunkworks "Midori" operating system project.
I first blogged about Midori in 2008. Microsoft was assembling a team of crack engineers to build a new operating system that wasn't based on the Windows kernel. The Midori team was charged with building not just the OS from scratch, but a full software stack, including a browser, related tools and more.
At its peak, there were as many as 100 developers at Microsoft working on Midori, according to team member Chris Brumme's LinkedIn profile. Brumme identified himself as the "Co-founder, Architect and eventually General Manager for a significant OS incubation effort," that dated back to 2005. (Brumme joined Google in 2015, his profile says.)
Joe Duffy, a development manager and language architect on Midori, recently kicked off what is planned to be a series of about a dozen blog posts about his work on Midori. The first post from November 3 is about the three safeties (type, memory and concurrency). Duffy is currently engineering director for the Compiler and Language Platform group at Microsoft.
With Midori, "although we started with C# and .NET, we were forced to radically depart in the name of security, reliability, and performance," Duffy blogged earlier this month. "Now, I am helping to bring many of those lessons learned back to the shipping products including, perhaps surprisingly, C++. Most of my blog entries will focus on the key lessons that we're now trying to apply back to the products, like asynchrony everywhere, zero-copy IO, dispelling the false dichotomy between safety and performance, capability-based security, safe concurrency, establishing a culture of technical debate, and more."
For a number of years, Microsoft officials tried to downplay, if not hide, Midori's existence. A few of the project's participants made cryptic references to a secret technical incubation with which they were involved at Microsoft. But these days, it's fairly easy to find rather detailed information about the components on which the Midori team worked.
One team member who worked on Midori from 2011 to April 2015, according to her LinkedIn profile, worked on "porting the distributed storage and compute engine to a managed language and bringing it up in the new OS environment." That engineer, Svitlana Tumanova, also worked on helping port the existing C++ codebase to a safer language and on designing the DOM Tree and CSS parts of the browser on the new OS.
Leif Kornstaedt, another engineering lead and architect involved on Midori from 2008 to June 2014, noted in his profile that he worked on "avant-garde approaches to a number of technologies, such as inter-process communication, promises, managed code runtime, ahead-of-time compilation of managed code, managed/native interop, concurrent garbage collection, asynchronous execution models, JavaScript execution and hosting, and Web browser architecture."
David Tarditi, whose LinkedIn profile says he led the Midori tools team from 2007 to Novemer 2013, provides some of the history around Midori:
"I led the tools team for the Midori project, taking the team from 4 people to lots of people. Midori was an operating system written entirely in C# that achieved performance comparable with production operating systems, minus the security and reliability problems found in OSes written in C or C++. We continued developing the Bartok compiler used in the Singularity project so that it could be used to write production-quality systems software in C#. We added support for generics, shared libraries, generic shared code across libraries, and other cool features for systems programming. We connected Bartok to the Phoenix compiler infrastructure and drastically improved the overall quality of code generated by Phoenix so that it was competitive with production C/C++ compilers for SPEC benchmarks and added extensive support for managed code optimization."
(M# emerged as the language that grew out of the Midori team's compiler work.)
As one of my favorite Microsoft sleuths, The Walking Cat (a k a h0x0d on Twitter) has been documenting for years, many of the Midori team members left Microsoft. Once the project was moved under the current Operating Systems Group, even more ended up departing the team, if not the company. Earlier this year, Eric Rudder, who sources said was the executive champion of Midori, also left the company.
The Microsoft party line is that the Operating Systems Group and other teams at the company are incorporating "learnings" from Midori into what Microsoft builds next.
It was never clear to us Microsoft watchers what Microsoft ultimately planned to do with Midori. From Duffy's blog, it sounds like the team members weren't so sure, either.
"I'll be the first to admit, none of us knew how Midori would turn out. That's often the case with research," Duffy said.
"My biggest regret is that we didn't OSS (open source) it from the start, where the meritocracy of the Internet could judge its pieces appropriately," Duffy added. "As with all big corporations, decisions around the destiny of Midori's core technology weren't entirely technology-driven, and sadly, not even entirely business-driven. But therein lies some important lessons too."