Two Quora threads about Facebook caught my eye this week: Is it true that Facebook has no testers? and Why does Facebook not need to produce particularly high-quality software? Both have excellent answers from Evan Priestley, an engineer who worked at Facebook between 2007 and 2011.
First let's look at the testing answer. Priestley says it's "mostly true" that Facebook does not have a separate group of testers or QA people, and that the social networking giant mainly relies on automated testing. Testing is the essentially the responsibility of the person who develops a given feature.
Priestley left Facebook in April 2011, so he can only say that as of then "Facebook had no employees who were dedicated to QA or otherwise performed QA as their primary job responsibility. There were some employees who do some vaguely QA-like things, but this was a small part of their job."
He noted that Facebook has a huge amount of automated testing if you measure it according to lines of test code or the time it takes to run the tests, but in terms of scope and coverage, he notes Facebook has a fairly limited amount of automated testing. The company reportedly uses the following signals to identify bugs and other problems that have made it past review:
- Everyone at the company uses the software all the time and has tools to report bugs they find in pre-release versions of the codebase.
- Aggregated error logs show when a problem that is identified by logging code or by language features (e.g., calling a method that doesn't exist) is occurring at scale in production
- Aggregated user reports and graphs of workflow success (e.g., number of messages sent in the last hour, versus in this hour last week) show when a feature is completely broken for some significant percentage of the user base
- Major clients of the Facebook platform are effectively forced to do QA for Facebook because their businesses depend on it, so they report Platform problems (they have access to new code 24 hours before Tuesday releases via beta.facebook.com)
- Ex-Facebook employees have some privileged channels they can use to report issues; I personally report around 13,000 bugs per month
Priestley ended his answer by saying that having no testers works for Facebook partly because Menlo Park "does not, by and large, need to produce particularly high-quality software." This is what prompted the second question, which caused Priestley to elaborate on his train of thought.
He says Facebook has lots of bugs, broken features, and UI glitches. His definition of "high-quality" software is code that has very few of these things, and says examples include software from companies like Apple, Amazon, and Google. "Facebook is the buggiest software I use regularly, by a wide margin," Priestley admits, but also notes that "empirically, Facebook produces software that has all these problems and is also hugely successful."
He thus believes Facebook doesn't need to produce higher-quality software and lists the following reasons as to why he thinks this is:
- My definition of "high-quality" sets a very high bar. For instance, back when MySpace was big and Facebook was small, Facebook was higher quality than MySpace. Facebook isn't low-quality, it's just lower-quality than other software I use regularly.
- By paying less attention to quality, Facebook has been able to focus on other things, like making the company a fun place to work at that can attract and retain talented engineers. Facebook would probably be less fun if it cared more about quality.
- Facebook's product is a website, so it can fix things quickly. It has a process which permits rapid deployment of new code, and rapid rollback of buggy changes. This reduces the cost of recovering from bugs.
- Facebook's product has a lot of momentum and lock-in. The barrier for users or businesses to move off Facebook is very high. This gives Facebook a wider margin of error to ship glitchy software. If Google was broken for a day, you'd probably go to Bing and might not come back. If your iPhone pissed you off all the time, you'd probably buy an Android when you're faced with the decision in a year or two. If you can't order something on Amazon, you can order it from somewhere else. If Facebook is broken, you keep coming back until it works again.
- In the case of businesses, this is even more true. Facebook Platform has gotten better in the last year, but it was pretty awful to develop against for a very long time. But there aren't options -- apparently, 94% of Zynga's revenue comes from Facebook. Platform developers don't really have any leverage to hold Facebook accountable.
- Social networking isn't really critical to people. It's important, but it's not banking or space shuttles or nuclear reactors. It's not bridges or cars. It's not even e-mail (at least, in most cases) or phone calls. This also gives Facebook more margin to work with.
I can agree with all of these reasons, but I don't think Facebook can keep this up for very long. Everyone gets tired of poor quality, even if the service is free. The company will have to step up its game, not just because of its upcoming IPO, but because both developers and users are becoming increasingly frustrated. The social networking alternatives are getting better, and Facebook can't just let them keep closing the gap.
- Facebook announces 2012 Hacker Cup
- Facebook: Cookie tracking issue is limited, fix coming today
- Facebook launches security bug bounty programh
- Is your Facebook News Feed broken too?
- Developer survey finds Facebook has the worst API