For Peter Jackson's Weta Digital the Cloud does not compute

For Peter Jackson's Weta Digital the Cloud does not compute

Summary: Kathy Gruzas, CIO of digital effects production house Weta Digital, explains why the company is sticking with its own datacenter.

SHARE:

When Kevin Baillie, co-founder of California-based special effects house Atomic Fiction, took the stage at last year’s AWS re:Invent conference in Las Vegas he made a compelling case for the benefits of cloud computing.

Baillie said Atomic fiction was able to dial up “Pixar-sized” render farms in minutes and faced no physical limits on its computing power by using AWS.

That in turn allowed Atomic Fiction to put its resources into people, not into computing and datacentres.

In an industry with immense and sudden peaks and troughs in computing workloads, cloud seems a no-brainer.

But for Peter Jackson’s Weta Digital, maker of the Lord of the Rings and Hobbit trilogies and of King Kong among other projects, the sums don’t add up.

CIO Kathy Gruzas says while there are many cloud applications Weta could use and which are “theoretically very attractive”, when the company has studied cloud costs they do not stack up.

Weta Digital has invested over many years in its own supercomputing capacity. Its datacentre is powered by 5000 HP servers from the latest Ivy Bridge microarchitecture to older G5 blades.

The datacentre has to scale for peak capacity, not average use, and that’s not cheap. It also means the render farm is not busy all the time, so capacity is not being fully utilised.

Gruzas said Weta has looked at cloud, which could address that problem. The company even performed costings, but a number of factors mean it has yet to be embraced.

Bandwidth is an issue for the Wellington, New Zealand-based company. It’s expensive, but the real issue is the volume of data and latency.

Latency is managed using IBM Aspera’s FASP transfer technology because FTP won’t cut it, said Weta Digital’s systems manager, Matt Cunningham.

Weta can render 50TB of data overnight during production. That is then studied and changes made before it is rendered again.

Fast turnaround is essential to keep up velocity, Gruzas said.

In Sydney, where she used to work, production houses were linked by LAN-speed networks, allowing them to collaborate and share each other’s capacity.

If there was a suitable cloud datacentre in New Zealand accessible at LAN-speed, it would be a no-brainer.

Weta also looked at using Sydney-based Steam Engine, but that didn’t add up either.

But it isn’t just computing power that fails the test. As Weta finishes production, files are archived on tape. The company has at least 8PB of data stored this way.

Last year the company looked at renewing that capacity or finding an alternative. Weta took a close look at Amazon’s Glacier archiving service, which is available through the AWS Sydney datacentre.

“When we costed it up it didn’t make financial sense,” Gruzas said. “We find that with a few cloud services.”

She concedes if you were a “greenfields” site faced with capital investment it might be different.

And that leads to the issue of competition. Weta Digital’s investment in computing capacity has been a "moat' against challengers, but is it still?

Gruzas said cloud will certainly change industry investment patterns in the future but Weta’s direct investment has been a good one. It can also be written off against numerous projects over its five-year lifespan.

“Quite a bit if capability is with size. We have the infrastructure to get things done quickly.”

A similar-sized service to Weta’s in the cloud would cost a lot, she said.

“I would be surprised if you could cost it up for less.”

But there is another, perhaps more important, constraint: the clients who dictate how content is delivered and stored.

Clients mostly dictate delivery over the net but with full encryption and security. Sometimes this is done in partnership with a specialist ISP.

Sometimes encrypted hard drives are shipped by plane as backup.

Data ownership is a particularly sensitive area, Gruzas said. Some clients are not happy about having their expensive intellectual property in the cloud.

“We do what the client tells us and provide options,” Gruzas said.

A lot of the technology is also quite proprietary and sector specific, she said.

Storage, via a is via high performance NAS, is the biggest issue, said Gruzas.

Weta is constantly reevaluating  and testing storage technology.

“It almost makes the cloud a non-issue,” Cunningham said. “If you are having troubles [accessing] internally let alone across the net.”

Gruzas said more standard use cases for the cloud could be considered, especially where there is no client data involved.

Fro Atomic Fiction, though, it is the cloud that makes everything possible. Baillie said 1000 servers for one hour costs the same as 100 servers for 10 hours. that provides computing power and flexibility.

Once production is finished, that can be scaled back to zero servers so unused infrastructure is not being paid for.

Topics: Data Centers, Cloud, Storage, New Zealand

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

2 comments
Log in or register to join the discussion
  • Sometimes it makes sense, sometimes it doesn't.

    Sometimes it makes sense, sometimes it doesn't. I think this whole "it's a no-brainer" thing is a fiction invented by bloggers to hoist their religion.

    There are a lot of variables at work, and the requirements that ultimately lead to a decision may be complex. It's actually far from being a "no-brainer." A "compelling case" to a blogger at a conference may not be compelling to somebody else in a different industry.
    CobraA1
    • No Argument

      One of the complexities of a production facility in a field with rapid technological advancement is capital costs. This slammed the music recording studio industry in the 80s and 90s as digital gear became very capable and relatively cheap very quickly. The studios that had acquired top of the line analog mixing desks and tape decks found that the new studios could produce good recordings while charging less. The old studios had to a) borrow to get new digital equipment and b) had to pay off the analog equipment they had just bought to get the big accounts. Moore's law meant in two years this year's digital equipment would be obsolete. The new digital gear also meant home studios and recordings disrupted the studio business from below. Artists who composed in the studio could do so at home and bring their floppies and Zip disks into the studio for clean-up, vocals, and mixing. Billings dropped.

      The VFX industry here in Hollywood is suffering, some of it has to do with globalization and some if it has to do with the VFX budget rigidity, production and payment schedules of Hollywood. The point that cloud may solve some issues regarding variable needs is a good one. Weta is basically saying that because of bandwidth complications, they capitalized a data center, but they have times when they aren't using the capacity they still have to pay off.

      Bottom line: don't ask me to invest in a VFX company. Too much turbulence and risk.
      DannyO_0x98