The 55 Australian organisations that took part in Australia's cyberwar games, Cyber Storm II, suffered "death by a thousand cuts", according to the head of Australia's Cyber Storm II effort.
Speaking at day three of the AusCERT 2008 security conference, Steven Stroud, head of Australia's Cyber Storm effort and director of e-security exercises at the Attorney General's Department, told delegates that the incident response teams of participating organisations often became short-sighted under the simulated attacks, leading to chains of command crumbling, careless mistakes, and the loss of vital information.
"A lot of organisations wanted to exercise senior incident response (IR) boards, and to do that they had to create a crisis on the shop floor. What they found out was, that it was very hard to get people to escalate. The IR teams were putting out spot fires here and there and no one took a step back to see the whole house was on fire," he told delegates.
"They're only talking about what they know about. They're only talking about what they can deal with, or deal with shortly. They are not projecting out how bad can this be... That doesn't happen. Some of the feedback is that it's not because they're busy, but it was because of a mindset."
Incident response teams too often dealt with localised problems, but failed to see how to tackle the high-level issues.
"An example from the banking sector was a number of, let's say, theoretical customers [who] had their credentials compromised through Internet banking. So the response was to reset the credentials. However, no one dealt with the actual problem, which was that these people all had keyloggers, so resetting credentials was a waste of time," said Stroud.
Many organisations were also surprised by the level of pain caused by the attacks — despite being well aware of their impact.
"If you hit your hand with a hammer, it's going to hurt. In Cyber Storm, a lot of people hit their hand with a hammer and were surprised that it hurt," said Stroud.
Standard operating procedures, such as logging incidents and following a chain of command, crumbled as the intensity of attacks increased in certain exercises, resulting in the loss of vital information. Stroud said incident response teams need to take a "101" course in data forensics.
"They need to be able to handle information to sort out their troubles and know where [information] is."
An incident response manager from one Cyber Storm participant studied how these procedures stood up to different levels of intensity, said Stroud.
"These guys had quite robust, formal communications paths. The way they did things — they were generally pretty busy and it all worked really well. So they took the phone call, they logged it. They got an e-mail, they logged it, etc, etc. And as they got busier, the logs became, instead of on the keyboard, scraps of paper and then became shouts across the room. As they got less busy, they went back to formal, but as a result a lot of information got lost."