Cross-checking payment details with particular Twitter hashtags to work out the economic impact of an FC Barcelona match in a given area. Analyzing data traffic to help find out why accidents occur. Combining physiological signs with clinical and environmental data to improve diagnose a disease and find the best therapy. These aren't theoretical uses of big data - they're examples of how it's already being put to use in enterprises, helping them develop new business models and improve their services.
'Data is the new crude oil' is a refrain common among companies that sell big data products. The allegory goes that data is now a vital underpinning for almost all industries and, used well, it can lead to significant profit. But for many companies, especially SMEs, big data projects are seen as unapproachable, too complex to implement because of the difficulties inherent in making sense of huge datasets - particularly when they're hugely diverse too.
How can this perception be changed? How can businesses even begin to get to grips with planning a big data strategy? What parts of their IT infrastructure do they need to change or strengthen to get the most of from data that is large, diverse, and unstructured? Where can they find data scientists to do the job? And how much will it all cost? Those are some questions that a new Big Data CoE (centre of excellence), announced recently for the Spanish city of Barcelona, plans to address.
The Big Data CoE Barcelona is an initiative led by Barcelona Digital Technology Centre (BDigital), a member of the Catalan government's technology hub known as Eurecat. The CoE was created with the support of Oracle and has the city council and the Catalan government as partners. For the latter, the CoE is the realization of a fundamental part of its SmartCAT strategy, a scheme intended by Catalan government to promote 'smart city' initiatives across the country.
Accelerating market adoption
According to director Marc Torrent, the Big Data CoE Barcelona has four main objectives: generating knowledge through case studies and use of existing big data products; offering advice about data recovery services; providing training; and fostering data culture in business, especially with SMEs, and social sectors.
Torrent said that BDigital is already working on setting up a conference to bring together the big data community, in addition to organizing industry meetings to different companies networking together.
"We want to accelerate market adoption to create a powerful ecosystem and help companies grow to become international leaders in big data," he said. "Catalonia doesn't want to be just a buyer of technology. It wants to have companies that make great contributions on a global scale."
The CoE has its headquarters in the Barcelona Growth Center, created by the city council to promote economic development. It's not a separate legal entity, but rather a BDigital project.
"Each partner invests in a success story and provides money, time or data to carry it out," said Torrent, but didn't detail how much funding is being used for the center.
The center will be sharing case studies from cooperating companies and institutions, including Oracle, Caixabank, Bism@rt, the Barcelona Supercomputing Center, and the University of Barcelona.
Oracle and Spanish financial services group CaixaBank, for example, are using big data to help the bank get a better view of its clients activities. By cutting the time it takes to get useful data from the bank's systems from days to hours, the company can now quickly create more personalized offers, enabling it to better cross-sell and up-sell. The project will also make it possible for the bank to strengthen risk management and fight fraud.
Bism@rt, a Barcelona-based business intelligence company, has developed a product to detect fraud in public transport correlating the information given by the machines to punch tickets. Using Hadoop tools, the system can identify where fraudulent tickets are in use - for example, if a ticket is used at two stations at the same time.
The Barcelona Supercomputing Center (BSC), meanwhile, is aiming to aid the development of Big Data applications through the ALOJA initiative, which makes the open source platform Hadoop more affordable for developers. The BSC, along with Microsoft Research Center in Barcelona, has published a study on how to get the most of Hadoop under different deployment scenarios.
The University of Barcelona has set up a system to decode the information provided by bees through open-source sensors placed in urban beehives. The data collected includes the hive's temperature (internal and external), ambient noise (inside), weight, and when the bees enter and exit.
Doubtless many more interesting case studies will be published in the future as, according to the CoE's director, there are lots of processes that can be improved through big data. Yet some obstacles still prevent many companies from turning data into knowledge, and turning knowledge into profit.
Professionals on board
Manuel Bruscas, the head of analytics and insights at Desigual, the Barcelona based casual clothing brand, and co-founder of BCNAnalytics, a non-profit project launched last November to foster an analytics hub in Barcelona, says that "data-driven companies become smarter and more efficient, so that they can achieve sustainable growth".
"Those companies using big data effectively are able to better understand the dynamics affecting their business and turn data into actionable insights," he added. What's more, "companies leveraging big data will be better prepared to face upcoming challenges in an increasingly dynamic environment," he said.
Yet Bruscas also emphasized that "technology is not enough". Companies using big data need professionals - mainly data scientists - to fully unleash big data's potential.
"We will have a tremendous opportunity to generate a virtuous circle: successful companies will grow and need more data-driven professionals which will help those companies to keep growing," he said.
The challenge is to invest enough, not only in terms of technology but also human capital. "A big mistake many companies do is to acquire big data capabilities without having the right professionals on board."
As a result, CEOs see big data as generating a lot of cost without necessarily the corresponding amount of ROI.
Another potential risk, pointed out Bruscas, is to disconnect big data from the business itself. "It is critical not to turn big data in just another set of reports," he said. "Currently, algorithms and techniques bring a lot of value, but you still need brain to get the most of them."
In this sense, Bruscas is convinced results need to be shared. Of course, some topics can be confidential or sensitive, but most of key learning can be disseminated, he said. This is one of the reasons he co-founded BCNAnalytics, which aims to stimulate debate on issues around big data and make clear they are not only for techies.
Together with the Big Data CoE Barcelona, these are initiatives that open a new competitive window for companies which want to harness the power of the data they gather day by day.