Google releases transparency data for mashups

Google says the data it holds on takedown and personal information requests is now so complex that researchers and developers need to get involved in helping to visualise it

Google has released raw data for the law enforcement and takedown requests it deals with, in the hope that developers will use this information in research and mashups.

Egypt transparency report

Google has encouraged developers to use the raw data from its Transparency Reports on takedown requests and web access to conduct new research. Image credit: Google

The company launched its Transparency Report around a year ago, in an effort to be explicit about the levels of content removal and user data requests it must process. During that time, the dashboard has been useful in showing, for example, how many scam ads Google has taken down, or which services are blocked or allowed in certain countries.

However, on Thursday Google said it was releasing the raw data behind its Government Requests tool in the machine-readable CSV format, so that "interested developers and researchers can take this data and revisualise it in different ways, or mash it up with information from other organisations to test and draw up new hypotheses about government behaviours online".

"We'll keep these files up-to-date with each biannual data release," transparency engineering tech lead Matt Braithwaite wrote in a blog post. "We've already seen some pretty cool visualisations of this data, despite the lack of a machine-readable version, but we figure that easier access can only help others to find new trends and make new inferences."

User interface needed

Braithwaite explained that Google's Transparency Report data has grown so complex that the company can no longer build a user interface that covers all the questions the data might answer.

"For example, the Transparency Report doesn't allow you to ask the question, 'Which Google products receive the greatest number of removal requests across all countries?' Using Google Fusion Tables you can answer that question easily," Braithwaite wrote, adding that the top four products relating to that question would be Web Search, YouTube, Orkut and Blogger.

Google will continue to add more raw data and APIs to the Transparency Report, Braithwaite promised. "So much can be done when engineers and policy wonks come together to talk about the future of the internet, and we're psyched to see the graphs, mashups, apps, and other great designs people come up with," he said.

The Transparency Report raw data will form some of the material feeding into a hackathon at the European Parliament in Brussels on 8-9 November, Braithwaite added. The all-expenses-paid hackathon will be the first to take place at that institution, and will also delve into data from the Open Net Initiative and Herdict.

Get the latest technology news and analysis, blogs and reviews delivered directly to your inbox with ZDNet UK's newsletters.