Outages cost large firms US$288K

update Over 90 percent of firms worldwide had to execute their disaster recovery plan last year, new report notes. Downtime in Asia costs US$150,000 on average.

update SINGAPORE--Over 90 percent of large enterprises globally have had to activate their disaster recovery (DR) measures within the last 12 months, a new report has highlighted.

Released Wednesday, Symantec's Disaster Recovery Research Report findings noted that 93 percent of organizations globally had fully or partially implemented their DR and business continuity plans over the past year. Within the Asia-Pacific region including Japan, the same trend occurred in 94 percent of enterprises.

Simulate, simulate, simulate

When Datacraft Asia tested its business continuity plans for its Singapore operations earlier this year, it found "missing links" that had to be rectified on the spot.
The company in February simulated an overnight bomb blast at its office premises in the island-state. Some 45 personnel were subsequently notified to report at an alternative site for the next working day. Its CEO Bill Padfield also participated in the exercise.
Datacraft has in place an umbrella business continuity planning (BCP) strategy for its 1,450 employees in 50 offices across 13 countries in the region. Globally, it has 11,000 employees.
Johan van Vuuren, human resources director at Datacraft Asia, told ZDNet Asia in a phone interview that practising a DR plan is important to organizations as it helps staff to internalize what actions were required in different scenarios. Variable components of the plans must also be updated frequently.
Catherine Tan, Datacraft Asia's administration and facilities manager and the business continuity coordinator, noted that through the exercise, the team learnt that what was on paper "may not necessarily work in real life". For example, those involved in the simulation realized during the exercise that one hotline would realistically not be enough to handle calls from families of employees, the media as well as customers. Instead, four hotlines were set up.
van Vuuren added that the team also had to consider getting adequate resources not only to man the hotlines, but be able to provide the right answers as quickly as possible.
Citing Symantec's study, systems engineering manager for Singapore and Indonesia Ronnie Ng noted that one in four DR tests in the Asia-Pacific region fails. Failure could be due to lack of testing, or inappropriate processes or lack of employee preparedness, he said.

Conducted by Applied Research, the global study included IT professionals and decision makers in 1,650 companies with headcount of at least 5,000. Of these, 450 companies are based in the region.

Globally, the top reasons organizations cited for having to activate DR plans were system failures, external IT security threats and natural disasters. Over in Asia, external threats edged out system failures as the most commonly cited causes, while power failures also play a prominent factor in some decisions.

In the Asia-Pacific region, the average cost of implementing DR plans per incident reported was around US$150,000. Worldwide, the execution of these plans cost an average of US$287,600.

Regional respondents reported that their operations would resume in about five hours after a site-wide outage--an hour more than the global average. Across the world, skeleton operations were said to be achieved in a median of three hours.

Gaps in DR planning
Ronnie Ng, Symantec's systems engineering manager for Singapore and Indonesia, noted in a media briefing Wednesday that compared to 2008, enterprises in the region have improved on their business recovery times and the level of executive involvement in DR committees has gone up. However, there still exists gaps in DR plans, he pointed out.

Some areas of technology, especially laptops and handheld devices, were not adequately addressed in DR plans, he said. For example, 78 percent of the region's respondents reported having laptops within their organization, but only 37 percent said their DR plans took into account the notebook environment. Similarly, 64 percent of those surveyed said mobile devices were utilized in their organizations but less than one-third reported such technology to be part of the business continuity planning equation.

Within the region, database servers, Web servers, e-mail and enterprise applications were the most likely technologies to be included in DR plans. Asia-Pacific findings largely mirrored those worldwide.

Source: Symantec

The study also found that 38 percent of businesses do not back up their virtual environments, and 28 percent do not test their virtual servers. This was despite the fact that virtualization had caused 61 percent of organizations to re-evaluate their DR plans.

Resources, said Ng, were a key challenge for respondents, but such constraints could be "easily addressed" through automating tasks and processes. Virtual machine backup can also be simplified using tools. "Whatever [an organization] has that works for the physical environment will work for [its] virtual environment," he explained.

According to Ng, a basic DR plan must be able to recover its data and operations within 24 hours, and have the ability to move storage tapes to a secondary site. Businesses should also engage hosted BCDR (business continuity and disaster recovery) services if they do not have their own DR facilities. Symantec recommends DR plans to be tested twice yearly.