New Zealand utilities management company ESP turns to the cloud for data analytics boost

When New Zealand's largest utilities management company, Energy Solution Providers (ESP), wanted to find a way to combine its big data techniques and advanced artificial intelligence (AI) analytics in May 2018, it knew it had a problem.

Like many companies in the sector, it was not a cloud-native player. However, it had on-hand data from 70 clients and 500 sites that needed to be optimised when managing their utilities usage.

ESP's clients came from different industries, including the manufacturing and industrial sectors, where energy costs are especially vital to bottom lines.

The retail and tourism sectors also used ESP's intelligence to better manage their usage, as did government departments and city councils that have to answer to taxpayers.

The challenge for ESP was bringing all of this information collected by its infrastructure of meters and sensors throughout the country and forming it into a coherent, actionable strategy for customers. ESP knew the fix was in using the readings it collected on energy and water usage, as well as indicators such as temperature and weather.

But it had to find a better way to prepare, analyse, and manage that data to extract useful insights. The old way of using Excel files and a web client to extract data from an energy ICT source had restricted ESP from coming up with an end-to-end view of data for analysis.

To resolve this during its move to the cloud, ESP sought to reduce its dependency on the EICT database by creating its own data lake. This would allow it to collect data from any source under any format, meaning the data could be analysed across various dimensions. Having such a capability would create a big boost in its capabilities for the future.

With this, external sources providing weather data, retailers' rate information, or even production data could be used for delivering a more complete picture for customers.

Harnessing the data fully

To get there, ESP adopted an agile development model, eschewing the traditional waterfall approach where each stage of the final app or service would need to be tested before launch. Going agile meant ESP could deliver a service quickly and correct any issues on the fly. This was possible with the flexibility afforded by the cloud.

ESP sought out Blazeclan for its ability to deliver an AWS solution, which enabled it to better harness the data it collected. The new system, among other factors, allowed ESP to become more agile when delivering data metrics to customers, who were increasingly seeking ways to optimise their usage.

ESP worked with Blazeclan closely to deliver this solution. The tech stack included Amazon EC2 to compute capacity management for developing its apps. This helped reduce the time needed to spin up new server instances to minutes, allowing ESP to scale up or down according to its requirements.

Separately, Amazon Redshift was used to manage the work of setting up, operating, and scaling a data warehouse. To store and retrieve the data, Amazon S3 provided the necessary capabilities.

Finally, Amazon EMR enabled ESP to manage the cluster platform, simplifying the way it used big data systems like Apache Hadoop. This allowed ESP to process and analyse large amounts of data more easily than before.

Delivering data as a service

The AWS solution developed by Blazeclan empowered ESP to deliver on its data-as-as-service business model. This transition to the cloud was smooth, and it had to be, so ESP could concentrate on delivering value to customers.

ESP is an example of yet another large enterprise successfully moving large amounts of mission-critical data and workloads to the cloud, in a bid to scale more flexibly and be ready for the demands of the digital economy.

With the added headroom offered by AWS's cloud platform, ESP now has the freedom to focus on clients' needs for more insightful, actionable intelligence. It is pioneering new ways to improve sustainability and is safe in its knowledge that its cloud platform is well managed.

There are also fewer staff hours spent to process the data, thanks to AWS' ready-to-use AI and machine learning tools. Instead of preparing large amounts of data for analysis, for example, staff are now able to jump into the more interesting part of delivering insights more quickly.

The data verification process provided by AWS' cloud tools also helps take the manual work away while improving data accuracy -- a key ingredient of ESP's future success.

For example, automated data exception checks have helped ensure that data is more accurately used. With this process becoming automated, it is now much easier for ESP to perform for each customer.

At the same time, predictive analytics is something that ESP can now deliver with more certainty than before. When there are changes in usage patterns, alerts are automatically shared with ESP and its clients so both parties can reduce energy costs or detect early signs of mechanical faults to perform preventive maintenance.

"Blazeclan has developed and deployed our Big Data Machine. They have connected a multitude of data sources into a single EDL, allowing us to generate actionable insights for our customer base like never before. Blazeclan has been so accommodating we simply could not have asked for a better partner," said Jeremy Allen, managing director of ESP.

"By connecting our data to AWS EDL and enabling Sagemaker, Blazeclan has provided a complete end-to-end solution that has dramatically improved our ability to engage our customers," he added.

"Through this improved engagement, we will deliver significant reductions in Green House Gas Emissions through the optimisation of our client's plant and machinery," he noted.

Amit Bassi, managing partner at Blazeclan Technologies, an AWS Premier consulting partner, says, "We are proud to be supporting ESP's vision of bringing world-class data analytics to this critical social issue."

"ESP is yet another example of the power of AWS-enabled business transformation – bringing new insights to established industries. ESP's in-house expertise is perfectly complemented by Blazeclan's experience in building this data-driven decision solution," says Bassi.

Discover how organisations in various sectors are making the leap onto the cloud in their digital transformation journeys with AWS.