Google and public documents

Google is working with officials in fours states to make public record (gasp!) public. Utah, the state where I was CIO in 2001 and 2002 is one of these. While state governments have done a lot to put records online in the last seven or eight years, they've usually designed their systems in ways that aren't friendly to search engines. For example Utah's Dept. of Workforce Services has an extensive database of jobs, but that information wasn't searchable using online search engines. From a Salt Lake Tribune article:

  The process doesn't put new information online; rather it "optimizes the way you do the searches," said Steve Fletcher, Utah's chief information officer.
  Most government sites are not organized in a way that allows commercial search engines, like Google, to access the databases, so many people would have to root around on a state page to find what they were looking for.
  Fletcher and his staff have spent the past six months going department by department allowing Google's search software to have easier access to state information.

The irony here is that they don't really need Google to help them do this. Many of the principles are well known and have been enumerated for some time. The problem is that states--who have an obligation to make public data available--have never seen fit to create standards that make that mission a reality. Maybe the Google project will help change that.

There are, of course, privacy concerns. But the privacy problem doesn't lie with Google or state agencies working to put data online. Rather the problem exists at the point where our public policy comes into contact with the Internet. For years public documents have been private in practice because of the effort needed to access them.

As an example of the conflict this creates, there are public documents that contain social security numbers. Obviously we shouldn't be putting SSNs online. California's CIO, Clark Kelso directed state agencies to redact SSNs and other private information from records before they were put online. I'd guess that he doesn't have the legal authority to do that even though it's the right thing to do.

Even with the privacy concerns, I'm a huge supporter of getting public data online. Transparency is an important principle that is common to both the Internet and good government. While the Google story is interesting, the more important question is what should be public? There will undoubtedly be lots of good arguments why government information, especially information about individuals, shouldn't be put online in the interest of privacy. Be careful.