US Commerce Department turns to Tableau, Enigma to help share public data

The US Commerce Department, which has a budget surpassing $9 billion and nearly 47,000 employees, is tasked with promoting job creation and economic growth. In doing so, it collects troves of data, on topics like US home sales, construction trends, and imports and exports.

Yet as anyone who's sought information from the government knows, that data is not always easily accessible.

"It's one thing to make the data available, but a small percentage of people would know what to do with, say, a CSV file of a million records," Ben Jones, director of outreach programs at Tableau, told ZDNet.

The Commerce Department is aiming to change that through a collaboration with Enigma, an operational data management and intelligence provider, and the business analytics software company Tableau. After the agency makes the public data more readily available via the Enigma platform, it can be visualized through the free Tableau Public data analysis tool. Anyone who signs into Tableau Public (more than 300,000 people currently have accounts) can make their own visualizations.

The toolkit could be used, for instance, to find out how many businesses are in a specific US metropolitan area. Or it could be used to track median home prices across the US from 1963 to 2016 -- the kind of dataset that could potentially help the US learn how to avoid housing market bubbles, Jones said.

"The more we as a country can understand why and how these trends occur, in something as important as the real estate market, it makes us less likely to fall into the pitfalls we've fallen into in the past," he said. "There's a promise there of more informed decision making... the ability to make our own analysis and from our own conclusions about what's happening in the coutnry."

Laura McGorman of the Economics and Statistics Administration, a division within the Commerce Department, said in a statement she is "thrilled" to see private companies stepping up to work with the government in this way.

"It's my belief that while government has the power and the responsibility to collect and share critical data for decision-making -- whether its about our economy, our weather, or our people -- the comparative advantage of making this data actionable lies with the private sector," she said.

Tableau has government customers at both the federal and state and local level, but this is the first time a federal agency is sending its datasets directly to Tableau Public. Visualizing government data can present some unique challenges, Jones said.

For instance, data covering categories like unemployment is often hierarchical -- organized into multiple groups and subgroups, such as unemployment by industry and region. That lends itself to treemaps but is otherwise challenging to work with.

Meanwhile, government agencies tend to load up datasets with headers, footers and merged cells, requiring an extensive clean up process before the data can be visualized. Tableau recently created a feature called Data Interpreter which streamlines the process.

Ultimately, Jones said, Tableau Public "is about driving a dialogue around data... We believe the stories of our time are stories that should be told with data."

Government data breaches: What's going wrong?