Q&A: Automating the data warehouse
The quality of your data is just as important as the quantity, says WhereScape chief.
WhereScape's tools allow organizations to prototype, build, deploy, and manage enterprise data warehouses and data marts, and big data infrastructure.
ZDNet recently spoke to the company's CEO, Michael Whitehead, about how to help companies do everything fast, his bootstrap beginnings, agile and DevOps, and key areas going forward.
Tell us about your company and its place in the data warehouse market.
It goes back to university where I came through the tech side. I discovered that some people were hellishly good at programming and I was not one of them. I could do it but there were plenty of other people who were orders of magnitude better than I was. I decided that I was not going to be a developer but I still enjoyed the tech side and wanted to work in that area.
Then I discovered that I was not a very satisfactory employee because I always thought that the company I worked for could do better and should do better. I didn't like the way decisions were made so I decided with a good friend who was better at technology to set up a company and to do stuff in a way that we thought was right.
We were in consulting then, and we decided really early on to get into data warehouse consulting. We were basically the first company in New Zealand to do this.
We would get called in because the big guys didn't know how to build and run data warehouses they way they should be.
WhereScape CEO Michael Whitehead: "You don't have to save the world, just do it easier and do it quicker."
It became obvious that, in order to build data warehouses, you had to do all the same things to get to zero before you could start to do anything of value. You had to set up an environment and make sure that you could manage code and promote things and that sort of stuff.
It was primitive stuff in those days.
The idea of our main product was: why can't we automate this? Why can't we just give you a GUI that does it? That was the beginning of the product and it just carried on from there.
It started off slowly as these things do, but as we sold to companies we started to get more and more interest in it. Then, four of five years ago, it just took off. It went crazy. More and more people jumped onboard. That was our automation journey.
We were trying to figure out how people would not have to do the thing themselves and still achieve what they wanted to achieve.
You saw it as showing businesses how they could simplify things?
Yes, you don't have to save the world, just do it easier and do it quicker. Do things so that people did not have to do what they didn't like doing. Feature 1.2 of the product was automating the documentation because nobody likes doing the documentation. It was all those sorts of things and we were building the automation story.
We now have 600 or so customers around the world. We operate in Europe, the US, and Asia. We have a distributor in Australia and resellers in Europe, South Africa, and pretty much all over the place.
You started in New Zealand and spread from there?
Yes. Development still takes place in New Zealand, and we've got a highly-integrated workforce. There is a good university system [in New Zealand] which is English-speaking and you are not competing with Google, Microsoft, or Oracle for the smartest people coming out of the system.
Are you a private company?
We are. We funded ourselves from day one and we only took on external capital about two years ago. To start with, everything was basically guaranteed by my house because banks don't like lending money to tech companies -- they don't have any assets. We just built it up from that.
We have a consulting business that we started and still goes really well, but that only works in New Zealand. We have a robust, multimillion-dollar services business that helps fund the international software business.
Are you looking at expansion?
Very much so. Our software business is bigger than our consulting business and the money we took on was to help build the development team and build the international operations out -- to put more people there. Now we have to get marketing people and PR people and so on.
Those are things we were thinking about from the start. Because my job was to build the business up, I made sure that we were thinking about marketing. We were thinking about more than just the functional aspect of the product. We were asking questions like: why would people want to use it? What's the emotional connection?
If you look at our product, it is actually a metadata engine but we talk about automation because it's a far more emotional thing to discuss. We tell them that we can automate functions for them and then they can build things much more quickly and enable change much more easily.
It's just a better story than to say, 'we can use metadata to generate code'.
You know, people like to work really fast -- to do something and then get on to the next. When you've got a lot of data swimming around, it really slows things down and there is this complete shortage of people who can understand the problems.
Isn't DevOps part of the answer to that?
One of the things that is really interesting to us at the moment is extending the automation from the development side to the DevOps side -- and automation is absolutely key.
If you imagine, in a standard environment, you have code and are making changes to it and maybe promoting it and so on. Basically, you have an algorithm -- code -- that exists with data. And with big data, these days there could be ten terabytes of that stuff and the code is so set with that. Well, in DevOps you just have to look at the data world in a different way.
The majority of the people that use our software use it in an agile fashion, which then brings DevOps into play very quickly. I don't think we are there yet. I think we've still got a lot to do in that space but it is certainly something that really interests me.
What do you think are the key areas in data warehousing and data mining?
What's really interesting to me at the moment is how organisations are doing so much work on data without having any real idea of how good that data is. It's like a whole part of the supply chain is missing. From a food standpoint, you can have something on a plate that is looking beautiful but you will have no idea what the orchard it came from was like.
With big data and the Internet of Things, the amount of data coming through adds to the importance of this.
But all of this is automatable and I think that stepping behind the graph and looking at the supply chain coming though, and looking at the quality of it, is the real issue that we must address. I am getting very excited about this because I think it is the next place for us to go to.