'

NSS Labs files lawsuit over alleged CrowdStrike, Symantec, ESET product test conspiracy

Updated: The antitrust case claims that the cybersecurity vendors have conspired to prevent independent, unbiased tests of their antivirus products.

NSS Labs has filed an antitrust lawsuit against CrowdStrike, Symantec, and ESET, alleging that the organizations have conspired to restrict independent product testing through AMTSO membership.

On Wednesday, the security product testing company's CEO, Vikram Phatak, said that the antitrust lawsuit relates to vendors which "are actively conspiring to prevent independent testing that uncovers product deficiencies to prevent consumers from finding out about them."

Among these vendors, Phatak claims, are CrowdStrike, Symantec, and ESET.

The cybersecurity companies are participants in the Anti-Malware Testing Standards Organization (AMTSO), which is a project designed to "introduce definitive standards for fair and useful testing," as well as "provide detailed advice on how to run tests."

Other members of the scheme -- but not related to the antitrust case -- are AV-Comparatives, Bitdefender, Carbon Black, FireEye, Microsoft, Kaspersky Lab, and Trend Micro.

NSS Labs is also a member of AMTSO.

The suit was filed with a US district court in Northern California on Tuesday. According to court documents (.PDF), NSS Labs alleges that the company is a "direct target" of the "conspirators" through AMTSO in efforts to "restrict competition in the testing of cybersecurity products that are critical to, but often fail at, the protection of computer systems operated by governments, businesses, and consumers."

"NSS Labs frequently uncovers product deficiencies during our independent tests," the executive says. "We tell customers about those deficiencies. As you can imagine, this can hurt a vendor's sales. So, what is a vendor to do? Some (the good ones) fix their products. Others try to avoid being tested."

A core reason for the lawsuit appears to be how AMTSO operates and which companies are members.

The testing of AV products is important. Not only can this uncover security vulnerabilities and weaknesses which vendors can patch before customers are put at risk of harm, but this can give consumers a marker for which cybersecurity solutions to adopt -- and which to avoid.

Phatak says that the project's idea of "fair and useful" testing is inherently flawed as they are "driven by the same security vendors whose products are being tested; not a neutral, independent third-party setting a higher bar for the security vendors and the industry."

In turn, this potential conflict of interest could be detrimental to independent, unbiased product tests.

The AMTSO Testing Standard, which the complaint deems "unlawful," raised the objections of members including NSS Labs, AV-Comparatives, AV-Test, and SKD LABS.

Despite these objections and a vote, the standard has been adopted. NSS Labs alleges that the cybersecurity firms named in the complaint allegedly agreed to boycott any testing company which did not adhere to the standard.

The testing firm says that such alleged behavior is illegal "or, at a minimum, unreasonably restrains competition" in the cybersecurity product testing market.

TechRepublic: Websites are attacked 58 times a day, even when patched properly

"Further, vendors are openly exerting control and collectively boycotting testing organizations that don't comply with their AMTSO standards -- even going so far as to block the independent purchase and testing of their products," Phatak alleges.

The complaint claims that through AMTSO, the cybersecurity vendors have come together to blindside NSS Labs, actions which have already caused "substantial injury."

The security testing firm also alleges that it "will suffer further injury, including irreparable injury such as permanent loss of market share," unless the apparent conspiracy is stopped.

CNET: Julian Assange may have sought sanctuary with Russia before turning to Ecuador

NSS Labs says that there is no competitive justification for the AMTSO Testing Standard and the project's goals will likely only result in restraining competition.

"AMTSO's efforts at determining how products are tested does not advance compatibility, interoperability, consumer safety or any other pro-competitive basis for standardization," the company added. "Rather, AMTSO and the AMTSO Testing Standard exist solely to enable product vendors to avoid competition on quality and price with no offsetting benefits to competition."

See also: Hackers hijack surveillance camera footage with 'Peekaboo' zero-day vulnerability

The NSS Labs executive specifically mentions CrowdStrike, pointing to clauses in end-user licensing agreements (EULA) which allegedly prevent tests occurring without the firm's permission.

"This unethical and deceptive behavior hampers transparency and hinders consumers in their ability to assess whether a product delivers on its promises," NSS Labs says. "If it is good enough to sell, it is good enough to test."

This is not the first time that NSS Labs and Crowdstrike have clashed. In 2017, Crowdstrike failed to prevent a report being made public which related to the Falcon Host antivirus product.

NSS Labs conducted the testing, which Crowdstrike claimed was performed poorly and was "deeply flawed." Crowdstrike also said at the time that NSS Labs' behavior was "unethical, illicit, and subversive."

Update 15.21 BST: A Crowdstrike spokesperson told ZDNet:

"NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing. We believe their lawsuit is baseless.

CrowdStrike supports independent and standards-based testing -- including public testing -- for our products and for the industry. We have undergone independent testing with AV-Comparatives, SE Labs, and MITRE and you can find information on that testing here. We applaud AMTSO's efforts to promote clear, consistent, and transparent testing standards."

See also: CrowdStrike customers that suffer data breach can claim up to $1 million in coverage

Symantec told ZDNet, "As this is a pending litigation, we have no comment."

Update 27.9: Symantec issued the following updated statement:

"Much of the security community have expressed concern and frustration with both the methodology and lack of transparency associated with the testing performed by NSS Labs. In our own experience we have felt concern regarding both their technical capability, as well as the practice of the NSS Labs "pay to play" model in relation to public tests.

We are aware of the lawsuit filed by NSS Labs and we believe that their claims against us are entirely baseless. While it's understandable that NSS Labs' desire for profits may be inherently at odds with a non-profit, standards-based organization such as AMTSO, the integrity of the testing process should be of utmost importance, starting with transparency and equity for all participants. We welcome the opportunity to bring the discussion of fair and open testing further into the public conversation, while also shining a light on certain business practices within the testing industry."

An ESET spokesperson said the cybersecurity firm is yet to receive any official, legal communication, and as such, "we are unable to say more at this time, beyond the statement that we categorically deny the allegations."

"Our customers should be reassured that ESET's products have been rigorously tested by many independent third-party reviewers around the world, received numerous awards for their level of protection of end users over many years, and are widely praised by industry-leading specialists," the spokesperson added.

When asked whether the companies named were the only entities involved, Phatak told ZDNet, "Those vendors (CrowdStrike, Symantec, ESET) and the vendor-driven AMTSO are named because they are the ones who are leading the conspiracy. There are other unnamed co-conspirators and their involvement will be determined during the course of this antitrust case."

"We are where we are because we refused to be pay-to-play and CrowdStrike knows it," the executive added. "Their smear tactics are par for the course. They should be ashamed of themselves."

Update 21.9.2018: AMTSO has released the below statement.

"AMTSO is disappointed by the antitrust lawsuit raised by a member organization (NSS), and we categorically deny all claims made against us.

We want to clarify who we are and what we stand for. AMTSO was founded in 2008 as an international non-profit association that focuses on addressing the global need for improvement in the objectivity, quality, and relevance of security testing methodologies. [...]

The testing standard is voluntary. It holds both testers and vendors accountable to ethical and fair practices, including ensuring that tests are fair to all participants.

It does not tolerate backroom deals, "fitted" results, or offering private, pay-to-play, undisclosed advantages to some vendors but not others.

NSS is a member of AMTSO, and one of their employees was an important member of the working group that developed the standard.

Rather than trying to use the legal system to tear down what we all built together, we encourage NSS to bring its concerns back to the table and engage with the rest of AMTSO membership to make our industry better."

Previous and related coverage