How the Financial Times built its own industry cloud
But who knows the ins-and-outs of your company's requirements better than those who've spent years working for it?
What's more, it's never been easier for an in-house team to build cloud-based services that are tailored to the needs of your organisation.
That ease comes from the abundance of building blocks on offer, in the shape of cloud-based infrastructure and services offered by major providers such as Amazon and Google. Deploying and swapping data between these services is simplified by automation, thanks to APIs that allow them to be automatically configured and controlled by tools such as Chef and Puppet.
UK-based newspaper the Financial Times built the FT Platform to take care of its online needs. The platform automates the deployment and management of the virtual infrastructure and the software that runs on top, both on in-house servers and on the Amazon Web Services (AWS) EC2 platform.
"We still look at the FT Platform and go 'That's worth building because it's allowed us to do things were still can't get from products'," said FT CTO John O'Donovan.
"Media companies are quite fickle, so you may think you've built the perfect media solution and you turn up at a media company and they go, 'We quite like it but we'd like to do it like this now'. What Amazon do very well is put all the right pieces together for the toolkit."
Tabloid favourite Kim Kardashian proved how rapidly public cloud infrastructure can be thrown together and torn back down recently, when the now infamous photos of her bearing her bottom proved so popular they were said to have "broken the internet".
" Paper magazine did the Kim Kardashian photos and had six days to scale their website up to meet unprecedented demand," said O'Donovan.
"What they did was take a lot of bits from Amazon, put them all together and served massive audiences and threw it all away again afterwards. That's the sort of flexibility a media company wants because we have incredibly peaky [traffic]. It goes from zero to massive at certain times of day."
30-fold productivity boost
The FT Platform supports the core of the Financial Times online -- the Apache HTTP Server and Apache Tomcat software that serves its news and other content, its DNS servers, the MySQL databases and the jetNexus load-balancing tools that keep its sites fast-loading.
The FT's developers are able to build and test code, and deploy it into production within a single day, compared with about 30 days using the old platform.
That 30-fold boost to productivity stems from the automated pipeline between the developers and the live environment, with new code being pushed from Git into AntHill Pro for building and testing, and then into QA, test and finally production environments, all managed using Puppet.
"FT Platform automates everything that's automatable," said O'Donovan, adding: "We won't work with you unless you have an API now."
Removing much of the manual work involved in deploying code and infrastructure allows the FT to upgrade its online offerings far faster than before.
"We have massive improvements in times-to-market: it used to take us 99 days to deploy some infrastructure and we've now got that down to minutes," O'Donovan said.
The FT also recently moved its data warehouse to Amazon Redshift, which, with a small amount of customisation, is supporting the same business functions with costs that are 80 percent lower than the managed service it used before, as well as answering some queries more than 90 percent faster.
Managing legacy infrastructure
Having control over the building blocks of its cloud infrastructure, rather than relying on that of a third party, has also made it easier for the FT to move applications between its in-house virtual machines running on Cisco UCS servers and on AWS.
For companies that traditionally had large in-house IT estates, it's important they can easily manage that legacy infrastructure alongside the cloud, said O'Donovan.
"Finding a way to involve your legacy [is important]. That's what the hybrid panel does; it's just our thin interface so that Amazon and our own infrastructure looks the same," he said, adding that they had integrated Active Directory with VMs running Windows Server on EC2.
"Because they look the same, that makes it quite easy and simple for people to engage with and they stop caring about where things are."
That control over its cloud infrastructure allows the FT to test its resiliency by trying to break things -- an idea made famous by Netflix's Chaos Monkey software.
"We have Chaos Snail. It's based on Chaos Monkey but it's more annoying -- it slows things down rather than kills things. The guys who wrote this had great fun writing it in order to terrorise the development teams."
According to O'Donovan, generic infrastructure services are increasingly available on the major cloud platforms, such as AWS, that can take over roles currently met by industry-specific providers.
"It changes all sorts of cost models for us. We're not even sure if we need a CDN [content delivery network] anymore," he said.
"You can take full advantage of Amazon's distribution network so you can move video around across their platform. Previously it would have been a horrible job and slow. But they have a great backbone and you can edit stuff in Hong Kong when it was shot in New York, which is an important thing for us."
Containerisation
Looking to the future, O'Donovan wants to make greater use of containers -- a lighter-weight form of virtualisation that makes it easier to move applications between machines, as well as Docker, software that automates the creation and deployment of apps in containers.
"When it comes to being cloud-agnostic and being able to move through development environments and deploy things really quickly, containers are just fantastic. It's software-defined and gives developers a lot of control, and this is where we're going next."
With Amazon offering a single control panel for managing containers hosted in-house and on AWS, the shift to containers will open up the prospect of an even tighter link between the FT's in-house and public cloud infrastructure.
"That's getting close to having Amazon in your datacentre, which is a very interesting thing -- being able to say 'I can move it into scaled Amazon environment or my local Amazon environment'."