Until quite recently, many people in Brazil didn't know what Big Data was. Until 2013, the year Big Data went mainstream, there was very little understanding about the concept of data analytics and its applications, the technologies behind it and the companies that operated in this space.
With growing interest in the topic and expanding media coverage, understanding grew and because of this, 2014 has been a year of widespread local experimentation, pilot projects and proofs of concept. Many of the largest companies experimented with the concepts and technologies, either internally or by hiring external help, and the demand has only been growing.
However, just as with every other emerging technology, there is still a lot of uncertainty and confusion related both with the underlying concepts of Big Data and the potential applications and benefits that it can bring. Actually, many people still have a hard time agreeing on what exactly Big Data is.
In this environment of high demand coupled with uncertainty, some noteworthy trends have arisen and will be shaping the evolution of the market in 2015:
Non-techies will help fix the skills gap
To realize all the potential of Big Data projects, it is not enough to have professionals that are skilled on the technical tools. To achieve results, they must also have a good understanding of maths and statistics, as well as deep business knowledge. This combination of skills is what characterizes a data scientist, or a data science team - and the demand for this kind of professionals has never been greater.
The lack of qualified professionals to work on Big Data projects in Brazil is one of the main problems companies face when starting analytics projects, and also the reason why consultants active within that field have no shortage of work these days.
Many of the top universities in Brazil have been trying to address the Big Data skills gap by launching new executive or part-time courses focused on teaching the basics of data analysis, and updating their CVs accordingly. Many companies have also been investing in training in a more focused manner to better respond to these issues.
One very interesting trend here is that much of the Big Data-related training is being offered to professionals who do not have an IT background, but rather a statistics or math background.
This indicates that the market appears to have decided that it is more productive to teach those who already understand data analysis how to work with the tools they need to handle Big Data, than to teach data analysis to those who already understand the technical side.
So it is fair to say that there's a huge opportunity for companies focused on training and education to provide courses for the market. There is also a lot of space for companies to provide tools for handling Big Data that anyone, regardless of their tech proficiency, can use.
"Big Data Washers" shall proliferate
Just as the advent of cloud computing brought about the "cloud washers" - companies that simply slapped the cloud computing label on whatever software or solutions they already had - today we are witnessing the rise of "big data washers" in Brazil: companies that are labelling their solutions "Big Data" regardless of whether they deliver on that promise or not.
While this is a common trend for any emerging technology that hits the mainstream, it seems to be happening even more in the Big Data space in this market. Part of the problem is the confusion and uncertainty surrounding it.
The lack of consensus and definitions then enables companies to rebrand their products as so-called Big Data offerings without giving customers a clear way to differentiate between the serious companies and those that are less so.
A lot of this relabeling also happens internally. Many companies, especially large corporations in the telecom and financial services verticals, have been claiming to have always worked with Big Data, that this is just a new name for the work they already did.
It is true that these companies might have been handling massive amounts of data for quite some time, but volume alone is not enough to label a dataset as Big Data.
Unfortunately though, not everyone understands this difference.
This trend will definitely continue to be seen in Brazil over the next year, as the large interest of the market on Big Data creates an incentive for solution providers and IT shops to jump on the bandwagon. Growing understanding and wider dissemination of information may help reverse it, but here in Brazil there is still a long way to go.
Focused vendors will succeed
Big Data is already a huge market, accounting for several hundred million dollars of IT spending in Brazil alone. Today, most of this money is going to technology companies from outside Brazil (infrastructure or basic application providers, such as Amazon and Cloudera), or to large consulting firms who are implementing projects.
The local companies having the most success and experiencing faster growth in the Big Data market are those who sought a particular Big Data problem and developed a product that solves that problem in its entirety. Such companies handle data collection, analysis and processing themselves, delivering the final result of the process to clients, instead of a series of technologies that can be applied to that problem.
Companies might have been handling massive amounts of data for quite some time, but volume alone is not enough to label a dataset as Big Data— Thoran Rodrigues
Why is that approach unique? Because it presents more valuable learnings to companies wanting to dabble with Big Data: finding a problem and focusing on solving it from start to finish means you are much more likely to succeed, rather than simply experimenting with tools and technologies applied to regular data.
Big Data is still very much a nascent market in Brazil, and it is expected to grow tremendously over the next few years. 2015 will be a year where the first movers of the market will start to shift from proofs of concept to production-oriented projects. This will present an excellent opportunity for local and international companies, meaning that the data analytics market in Brazil is definitely one to watch this year.