Westpac datacentre crashes ATMs, online

Westpac datacentre crashes ATMs, online

Summary: Westpac is this morning scrambling to restore services after a datacentre glitch took down its ATM, EFTPOS, website and online banking facilities this morning.

SHARE:
TOPICS: Outage, Banking
18

Westpac is this morning scrambling to restore services after a datacentre glitch took down its ATM, EFTPOS, website and online banking facilities this morning.

Westpac ATM

Westpac ATM tells users it's "closed" (Credit: Kamilah Hopewell/ZDNet Australia)

Westpac has blamed the outage on an air-conditioning problem in Westpac's datacentre, according to Westpac spokesperson Jane Counsel.

"I can confirm and assure customers that this is something we're working on as an absolute priority," Counsel told ABC Radio this morning.

Counsel said that St George services had also been affected, but added that she wasn't aware of the full impact.

Hundreds of angry customers took to the social-networking service to protest.

"Yesterday #Westpac [announced] a $3.17bn profit. Today I can't even get $30 out of tier ATMs," said one user.

"If I cannot get my pay this week #Westpac will be on the wrong end of a #ClassActionSuit. Sorry for the inconvenience," remarked another, re-enforcing analyst views that customers will start voting with their feet when it comes to technology glitches.

The bank's outage has also affected CityRail payment services in NSW, with customers forced to pay with cash due to the crash.

Westpac's telephone banking is still operational.

Josh Taylor also contributed to this article.

Topics: Outage, Banking

Luke Hopewell

About Luke Hopewell

A fresh recruit onto the tech journalism battlefield, Luke Hopewell is eager to see some action. After a tour of duty in the belly of the Telstra beast, he is keen to report big stories on the enterprise beat. Drawing on past experience in radio, print and magazine, he plans to ask all the tough questions you want answered.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

18 comments
Log in or register to join the discussion
  • Gosh! isn't it amazing how these little "glitches" can take down an entire Bank's IT Services. Sounds like their datacentre didn't have backup airconditioning, probably because the bank's management didn't want to spend the money to make the datacentre more bullet-proof...

    Anyone IT manager that uses the hoary old "glitch" excuse deserves to be fired instantly.
    Doctor_Sardonicus
  • Less than one week after United Group Services has taken over the Facilities Management contract to run the data centres it’s down. It not easy to loose a high quality constructed data centre on just air conditioning and does this point to quality of new facilities staff training????
    They did not pick up any of the old Facilities Managers due to unacceptable employment conditions so the knowledge of over 20 years experience was lost.
    Well the higher cost of quality management for 10 years is insignificant compared to the cost of this down time.
    Great decision Westpac
    Gottobesaid
  • Westpac have a redundant data centre but non of the banks can swap over at the touch of a switch. Also Westpac are in the process or obtaining a new redundant data centre with Fujitsu.
    The upgrades that have occurred in the main data centre are high quality and the resilience is of a high level thus you still have to ask about the site management or change management id due to upgrades.
    Gottobesaid
  • It was not an upgrade that took it off line, it was the air con - are did you mean that they upgraded the high quality aircon?
    amckern-b0f83
  • No sorry to not make clear. the upgrading of the main data cantre is occuring in stages and the final stage is to use co-generation and obsorbion cooling systems. I'm not certain if this has started but was scheduled early this year. They dont normally do cut overs or chages during the week but you never know. The existing air con that cools the computer romms is of good quality and plenty of redundancy and they have installed quality high effienancy chillers so what i mean by the project is if they were continuing the upgrade. I dont know the letest details
    Gottobesaid
  • Air-conditioning is the forgotten thorn in most IT Disaster Recovery plans. We cater for most scenarios with backup power and redundant networks, servers and storage and SLAs but when it's the air-con our recovery plans grind to a halt as nothing can startup while we sit and wait for the air-con supplier's maintenance workers to arrive...
    des@...
  • Westpac have 24hour techniical staff on site and due cooling system right from the power supplys to the mechanical system, duel seperated redundant chillers & pumps and all 100% back up by generators. It is a very high redundant data centre. It has to come to the new staffing training, compentancy and experance if we are being told the truth as to the cause.
    Gottobesaid
  • I enjoy the comments here but could everyone please try to make their comments readable?

    Spell checks might also help...

    We all make spelling mistakes, but when mixed with poorly phrased sentenced it leaves us wondering just what was meant to be said.
    Scott W-ef9ad
    • He he he, even I goofed up. 'sentences'.
      Scott W-ef9ad
  • The real question is why didn't any know that the aircon failed and the hardware was left to shut it self down before any damaged can occure ! Makes a mockery of Gail's and Bob's statements that all is good with there IT !!!.
    jeff_syd
    • Exactly the Facilities staff should have had sufficient time and notification from the building automation system. As above it is difficult to loose a site with the amount of Westpac redundancy on cooling only. Big questions to be asked I’m certain United Group services are having trouble sitting down from the kicking.
      Gottobesaid
  • Sorry, old tradesmen cant spelll they are best at fixing things. Spell check what that one of those new fangdangle computer thingies. Hey Bill pass the spanner :-)
    Gottobesaid
  • Even if the aircon failure was quickly detected it wouldn't take long for the hardware to start heating up (which generally will trigger an alert to staff).

    It would be fair to assume some contingency actions were applicable for this eventuallity, but if there's no aircon there is no aircon...

    I've seen companies willingly spend millions to rectify an outage, but then happily cut back on having those same redundant systems when all is going well . Either way, you can't afford 100% Operational Availability (uptime) and sh*t happens.

    As to what caused the outage and why it took so long to rectify, that's another story which will probably come down to a chain of human errors rather than a lack of backups/redundant systems.
    Scott W-ef9ad
  • Agree with Scott. But I can't believe this day and age, they can't get portable airconditioning units in there when it was first detected. I mean if the room is getting warm, its not hard to detect as data centres are usually freezing. Whatever happened to the Operations people who should be escalating the issue and kicking off contingency plans? Or did someone say "PCs run in hot weather all the time, mainframes will be right."? LOL. Ooops...someone just got fired.
    binary0
  • Did anyone else have problems receiving their wages due to this? I want to know if it's the banks problem or my employers.
    ACartmill
    • This is the perfect time to use Westpac as the excuse for money not getting through. The only way you can detect whether its Westpac or not is to find out when the money was supposed to be deposited. As the Westpac issue happened this morning (probably sundown AM hours) and if your employers were suppose to be depositing monies into your bank account yesterday then accountability clearly rests with your employers.
      binary0
  • Aircon? Really, truly, cross your heart? See the Courier Mail "Public servants face web bans to minimise risk of password cracking" (28 March 2011) - RSA customers all being hit at once - NAB/Commonwealth Bank/centrelink/Airport/and now Westpac/St George's all within a few weeks of each other. Look up bt.dna!! And then think of all the tech. outsourcing.
    LLAMA-98509
  • I'm with St George, which was also hit by a similar problem Thursday. The ATMs were down, but that morning (about 9.30am) I was still able to do my Internet banking without trouble. I ended up withdrawing money at an NAB ATM, which also had no problem with my account.
    dmh_paul