Update: Reaction from Google (Feb 7 ).
Update 2: Response from NSS Labs (Feb 7).
Every leading web browser incorporates technology designed to detect and block malicious downloads.
With Internet Explorer, Microsoft uses a feature called Application Reputation. Chrome, Firefox, and Safari use Google’s Safe Browsing API to check URLs against a constantly updated list of known phishing and malware sites. But a security research firm that specializes in measuring the effectiveness of these filters says its most recent data suggest Google is not playing fair with its partners.
NSS Labs, which has performed testing under contract for Microsoft in the past, says its most recent round of testing was “unsponsored” and “produced as part of NSS Labs’ independent testing information services [with] no vendor funding.”
The firm laid out its results in a 12-page paper, provocatively titled, Did Google pull a fast one on Firefox and Safari?
“Since the adoption of Safe Browsing API v2 and the elimination of proprietary solutions,” the report’s authors write, “both [Firefox and Safari] have demonstrated a reduction in effectiveness at blocking traditional malware downloads [compared to Chrome].”
One data point in particular jumped out at the researchers:
The latest round of testing occurred from November 21, 2011 to January 5, 2012, during which NSS researchers observed what appears to be a significant change when compared with historical results. Chrome’s protection rate steadily climbed to just over 50% before suddenly falling back to 20%. Over the same time period (Nov 21, 2011 – December 21, 2011), Firefox and Safari’s block rate remained at 2%, and then inexplicably jumped to 7% on the same day Chrome’s protection fell precipitously (December 22nd).
This is what the test data looked like in graphic form:
NSS Labs noted the apparent coincidence that Google and Mozilla announced that they had renewed a lucrative search agreement on December 20, 2011:
On December 21-22, 2011 NSS Labs observed a reorientation of protection whereby proprietary protection offered by Chrome dropped dramatically while shared Safe Browsing protection within Chrome, Firefox and Safari increased. While these events may not be related, the timing raises questions.
The more serious allegation is that Google is withholding key parts of the Safe Browsing API from its supposed partners:
NSS Labs determined that Safe Browsing API v2 includes undocumented functionality that has been integrated into Chrome, but not Firefox or Safari. This undocumented functionality provides reputation services for executable files, or as Google describes it “malicious downloads”.
Chrome uses an undocumented API call to block malware once download begins. This API is not utilized by Firefox or Safari, apparently due to lack of documentation and a proprietary format.
Despite claims to the contrary, Google has developed proprietary functionality via Safe Browsing to block malicious downloads. This functionality is not available to the other Safe Browsing API v2 browsers (Firefox and Safari).
The latest round of research confirms that Microsoft’s App Rep technology is significantly more effective than the alternatives. I’ve written here about the reasons (see IE9 versus Chrome: which one blocks malware better?). Based on NSS Labs’ findings, Google appears to be trying to build a reputation engine of its own.
What’s not so clear is why Google isn’t sharing that technology with its partners.
Update: Via e-mail, a Google spokesperson denied these charges: "As before, this analysis from NSS Labs is inaccurate. Google has not withheld malware protection from the Safe Browsing API." Google criticized NSS Labs for not making its full data and research methodoology available for others to reproduce, and they suggest that it's possible the researchers were picking up a client-side download protection feature that Google has been testing. The company also argues that other browser vendors could simply examine the Chrome source code if they want to see how the experimental features have been implemented.
We have offered the new Safe Browsing features to Mozilla in the past, so to say that we are holding back this functionality is inaccurate. From our conversations, our understanding is that Mozilla is still waiting for more data from Google about the effectiveness of our new technology, and is considering those benefits against the limited circumstances in which their users would send URLs to Google for scanning (this only happens if a page looks sufficiently suspicious or an executable download is not whitelisted).
This new protection, which is designed to detect new phishing pages as well as malicious downloads, was highlighted recently on our Chromium Blog in more detail: http://blog.chromium.org/2012/01/all-about-safe-browsing.html.
We believe this is a reasonable solution for Chrome users, and Microsoft takes a similar approach in Internet Explorer that involves sending URLs to Microsoft. The offer remains for Mozilla to have access to our new APIs for Firefox should they decide that it's in the best interests of their users.
Update 2: NSS Labs responds.
1. The methodology is the same as we have used over the past few years and is publicly available. The additional analysis goes in depth to show exactly which API calls are occurring with Chrome vs. other browsers.
2. The client-side download protection blog post that they reference is from January 31, 2012 - after our test was concluded. The download protection was announced by Google in April 2011.
3. That Google suggests other vendors have to scour Chrome source code to determine the functionality of the new features rather than provide documentation is unhelpful and "strange". It appears as though Google's new services are designed to transmit a user's browsing history data back to Google, and other browser vendors are unwilling to leap in without evaluating thoroughly the merits of that approach
4. NSS is making the tools used in this test available to allow other researchers to reproduce our findings.
Thanks for those additional remarks.