Context is everything when it comes to Big Data

The software we use to navigate our lives is becoming more context-aware — but as a result we risk spending our existence inside a huge filter bubble.

Context is finally coming.

Apple's Siri voice interface has switched us on to the advantages of a device that knows you and uses the context of where you are and what you care about.

Bing doesn't just resort the order of search results. It tries to take into account what your friends think of results and services to get you more useful information. Similarly Google is experimenting with adding results from Gmail to your web searches.

Instead of looking at the behaviour of everyone online, context is about mining your own information for your own benefit rather than just for putting in the right ads. Think of it as personal analytics.

Over the years I've used several tools, from SNARF — a social network and relationship finder from Microsoft Research that sorts mail based on how often I interact with the sender — to Xobni, which mines Outlook data to show me who I talk with most and brings together information about them.

I get the basics from the Outlook Social Connecter. If I need to find the email you sent me with the details of my next flight, I don't need to find the right message. I can look at any message from you and see a list of files we've exchanged and meetings we've been at.

The mail software in PlayBook 2.0 does a lovely job of supplementing that with information from LinkedIn, such as who we know in common, so it's easier for me to strike up a conversation.

I'm trying out Cloze, which mines email, LinkedIn and Facebook to prioritise messages and information about the companies I'm most interested in and tells me if I've missed messages from them. The irony of the fact that it warns me by email isn't lost on me and so far I haven't actually read any of the emails Cloze sends me.

The SNARF social network and relationship finder at work. Image: Microsoft Research

None of these tools comes close to what I really want. Something that combines traffic information with my calendar full of flights with the appointment I put in the diary for a friend's wedding on the west coast of America with available web feeds to give me a link for the cheapest upgradeable flight that gets me to Los Angeles in time.

The true intelligent assistant is still a long way but it's easy to be better informed.

I'm also been experimenting with the Bing News bar in Word 2013, part of the upcoming Office release, which shows me a list of relevant news stories whenever I select a phrase in my document — they could be inspiration, fact-checking or references as necessary, but sometimes I end up reading more than writing.

Getting the right balance between distraction and up-to-date information might take a while.

We're also going to need tools and habits to make the most of the data stream without our selection becoming a distortion. Think of the misleading filter bubble that you could get isolated in if your algorithms are wrong about what's valuable to you.

And then think what happens if that filter bubble is extended deliberately to a whole group of people.

Will Western individuals with access to uncensored information have an advantage over users in nations where the information is tightly controlled? What's the potential impact of that on education and on the resources and future of different companies? Authoritarianism can impose a way of looking at the world, but it can't make that worldview accurate.

And if you think you don't have any filter bubbles yourself and that the internet makes it easy to discern truth from unintentionally biased viewpoints from outright lies and manufactured rumours about Apple screws, check out an excellent book called The Information Diet and think about whether your online diet is more healthy, fresh facts or junk food.