Comment 50128

By Ryan (registered) - website | Posted October 22, 2010 at 08:54:23

A bit more about the potential value of open data:

In my article last year I pointed out an example of the difference between the city's current approach to sharing information and the way an open source data model would work: monthly building activity.

Today, the city publishes this data in monthly reports in a collection of PDF files. If you want numbers, you need to open up each PDF, visually scan through it to find the numbers you're looking for, and then manually copy the numbers over into your own spreadsheet.

It's slow, tedious, hard to navigate, and cumbersome to analyze for trends or patterns.

In researching an article a few years ago, I bit the bullet and manually copied all of the monthly numbers by building type into a spreadsheet.

Some time later, I moved the numbers into a database on the RTH server. Since then, each month when the next monthly report comes in, I manually add those numbers.

As a result, I now have a modestly useful data set that anyone can access programmatically. Because it's structured data and not a bunch of read-only PDF files, I can also easily write a script that renders the data in a pretty, human-readable format like HTML.

It's both more accessible to programmers and more accessible to non-programmers.

Unfortunately, because the city does not publish its data in an accessible format, I had to do a lot of manual work to reconstruct the data from the read-only reports the city does publish. The time I had to spend on just this report can't possibly scale to the vast number of other reports that would benefit from being available in an accessible, structured format.

Worse, city staff themselves have to do extra manual work to transform the data from a format in which they can analyze it (my understanding from talking to staffers is that it's mostly Excel templates) into the summary format in which they publish it.

It's a double inefficiency that wastes citizens' time and ties city staff up in busywork when they could be adding real value through their expertise.

Permalink | Context

Events Calendar

Recent Articles

Article Archives

Blog Archives

Site Tools

Feeds