Big data in action: Using graph databases to drive new customer insights

Marketing company Quander is using graph database technology from Neo4j to help its customers make better decisions.

How big data gives Mercedes a performance boost in F1 Flash storage and advanced analytics allows Mercedes-AMG Petronas Motorsport to hone performance in simulations and out on the track.

Startup Quander aims to help brands "deliver a memorable, measurable, multi-sensory brand experience". It uses interactive technology to enhance the customer experience, while capturing the data that brands can use to gain insight into their customers. ZDNet spoke to Quander's chief executive Gavin Williams about how the company is using graph database technology from Neo4j.

ZDNet: What was the first moment when things started to take off for Quander?

quander-gavin-06r-jpg.jpg

Willliams: "The irony was that it was one of the experiences that was being taken out."

Photo:Quander

Williams: We started to realise that this innovative advertising that we had been creating was really a platform. The advertisers wanted to know not just how many people had come to their events and had shared information and that kind of stuff, they also wanted to get a deeper understanding of how consumers interacted with the brand.

We had a lot of events that we were helping with and at every single event we would have the Quander platform working behind the scenes. 

SEE: Launching and building a startup: A founder's guide (free PDF)  

Using experiential marketing at events, companies would have our marketing systems working behind the scenes so that all the data they picked up was captured, so that the customer would understand all the people who had been taking part in all of the activities. And they would have not just the current information but the information gained over, say, the previous two or three years. Then they could understand what they could take from the experiences they had taken part in, how long they had been engaged for and that was helping us understand how people were interacting in a physical space.

Talk me through how you are using the Neo4j software.

At the moment we are using it to try and understand the data. That's always the first step. We are taking that data from our existing DBMS - which is Postgres – and we've imported it all into Neo4j. And we're talking about 1.5 million to 2 million rows of data.

The thing about Neo4j when we started was that it kind of felt like it didn't scale. And that was a big thing for us because we were so used to having a managed database – from AWS, for example – which upgrades everything and you don't have to worry about it.

Whereas now we were getting into the realms of, "We don't know if it will do what we want it to, but we do want to use this product". But the starter version allowed us to play with the enterprise features of Neo4j without having to pay for a big overhead upfront.

It's got to the point now where we are thinking of using Neo4j as our only database platform because it's just phenomenal. It's crazy.

What turned you onto it?

Funny story. So, you could see the difference between a regular database and a graph database – one was rows of data, the other was a graph database which is all about understanding a world of things.

So, we moved over to GraphQL and with our APIs we started to ingest that data and we couldn't find a good database to actually restore and retrieve data and write and read queries and reach data out the back of it.

PostGres is really good at storing transactional data; it has the SQL SMB that allows you to store NoSQL data inside a database. We get the performance of PostGres with functionality that we might get from an SQL database.

We looked at MongoDB quite closely but the trouble with it was that it didn't do relationships very well. If you were storing documents and stuff – embedded documents – it's great for that, but for relationships it's not very good.

Big data: An overview

Big data: An overview

Data is being generated about the activities of people and inanimate objects on a massive and increasing scale. We examine how much data is involved, how much might be useful, what tools and techniques are available to analyse it, and whether businesses are actually getting to grips with big data.

Read More

And then we just looked through Google for a good database for the front end and then a lot of graph databases popped up and then I did some research of graph databases. Neo4j popped up and other graph databases from Amazon, and another graph database from Microsoft Azure, and they were really good and really performant but the problem was that we couldn't visualize our data the way we could with Neo4j.

So I got Neo4j. I downloaded it from our desktop app, forwarded all of our data to it and it was just, there! So I got down to query the data and the data [in Neo4j] was so similar to SQL that it's not a big learning curve for anyone.

We were very aware that graph databases are very new, so the talent [pool] out there is very small and if we could find something where the query language is very similar to what's available on the market we could concentrate on the people.

So, we could take some people who were confident on PostGres, and then train them on Neo4j. They wouldn't need to know how to scale and things like that, they just needed to know how to get the data. In fact, it was easier for us to get the information out of Neo4j than it was out of PostGres.

So, we had to think about joins, we had to think about performance and so on, all these things we needed to understand about for a DBMS. And when it came to storing data, we had to teach them all about normalising data and de-normalising data, when to do those things and what the performance will be, all that kind of stuff.    

Where there any real surprises that Neo4j turned up?

[Laughs] Yes, absolutely. The first surprise was that we took all of our data – a full product set from one of our clients, and they said, "We want to do what we want to do". So, after we moved everything out and changed the experience and we ran everything through Neo4j.

I had it running on my really rubbish laptop, and it took, like, two seconds to run this query. And you could see all the data points and so I could see all these different experiences inside the space, and I could see all the content that had been captured, and it showed all the consumers that were related to the content and I could see the one experience with the most clusters that had been dealt. The irony was that it was one of the experiences that was being taken out!

So, the experience would be taken out, all the numbers would go down, someone would be blamed for it. Now, because of Neo4j, we could do something about it.

I took that out and mailed the results to the client and within minutes they came back to us and said, yes, you're right we should put that experience back in.

And out of that we got more work and our database – which had one of the lowest priorities in the company – was suddenly making money. And I thought: "That's insane! That has never happened before!"