The US Government Accountability Office (GAO) has attacked the Department of Homeland Security's cyber-defense system for being ill-equipped to protect agencies against most web threats.
But a former National Security Agency computer scientist says Einstein is being misjudged.
According to the GAO, the DHS' National Cybersecurity Protection System (NCPS) is meant to prevent malicious traffic from crippling federal agencies' ability to operate.
By 2018, the system, also known as Einstein, will have cost a total of $5.7bn. But, according to the results of a GAO audit, the system's capabilities still fall well short of the promise to provide intrusion detection, intrusion prevention, analytics, and information-sharing capabilities.
On all fronts, the GAO judged its capabilities to be "limited". For example, the system currently can detect known attack traffic that has been given a signature, but it lacks the ability to spot potential attack traffic that deviates from normal network activity.
Those features, on the other hand, are probably available in commercial security network appliances that may already be used by agencies. In the report, DHS officials conceded some of these commercial products probably also contain more signatures than Einstein possesses.
So why support the multibillion-dollar system at all if it can't protect agencies from the latest or even older, known threats?
Officials from DHS' Network Security Deployment (NSD) and US-CERT countered that it is only tasked with providing a baseline set of protections and "government-wide situational awareness", which form part of the US government's overall defense-in-depth strategy.
Besides this point, DHS documentation shows Einstein was only ever intended to be a signature-based intrusion-detection system. It's also just one of many security tools available to federal agencies.
Nonetheless, GAO urged DHS to implement anomaly-based detection since it is more effective at combating so-called zero-day threats -- attacks for previously unknown flaws -- than backward-looking signature-based detection for known flaws.
As per the report, US-CERT officials said they don't buy zero-day flaws and as such "there is no way to identify [a zero-day bug] until they are announced" publicly. After that, US-CERT can develop a signature for them.
"While we acknowledge the challenge of developing signatures for zero-day exploits, enhancing NCPS's current intrusion-detection approach to include functionality that would support the development of a baseline of network behavioral analysis," GAO said.
Dave Aitel, a former National Security Agency computer scientist, thinks Einstein is being judged incorrectly.
"The developers of Einstein are not stupid enough to think they're going to build a big Palo Alto box. Nor do they want to be in the business of writing thousands of IPS signatures," Aitel wrote.
"Instead, Einstein allows the government to do analysis across individual intrusions, detecting where attackers go when they laterally move from, say, [from the Office of Personnel Management], to the State Department," he continued.
As for US-CERT's inability to preempt zero days, Aitel points out the US government is in a position to marry that data with information from the NSA, which does buy zero-day flaws.
With that combination, Aitel said, "You have something very very useful. Much more useful than an IPS would be."
"It is about situational awareness and response, not protection. It still needs testing, but of a very different sort," he added.
DHS secretary Jeh C. Johnson has also responded to the report's claim its system is only partially meeting stated objectives.
He points out that the new phase of Einstein, dubbed EINSTEIN 3A, has the ability to actively block -- not just detect -- potential cyber attacks.
"Unlike commercial products, EINSTEIN 3A can rely upon classified information, so the government is protected against our most sophisticated adversaries," he said.
He's also directed DHS' cyber security team to "build capabilities that will allow us to detect never-before seen attacks, leveraging the best of government and private sector technology and expertise."