Open Data in D.C.: From Potholes to Apps for Democracy
David Strigel is the Program Manager of the Citywide Data Warehouse at the District of Columbia Government. Citywide Data Warehouse is the first initiative in the US that makes virtually all current district government operational data available to the public in its raw form rather than in static, edited reports. David shared the challenges and successes with the project with Systemscope’s Denise Eisner following his presentation at GTEC.
David, at GTEC you described the effort involved in getting government data sets online. What was the biggest lesson from the early part of the project and what might you have done differently?
Start small and start now. Washington DC did not wait to plan out a 10-year program. We started with one dataset in one format and put it online as is. We started with a single dataset in a single format: it was service request data, including things like pot holes and trash pickup.
Looking back, what might you have done differently?
The District, as one of the first to open their data in such a large volume, had to create custom code to build their data catalogue and data connections. With open data initiatives becoming more of a priority in many jurisdictions, many companies have appeared on the market offering solutions that can help a government launch a program similar to the Citywide Data Warehouse.
We now state in many RFPs that projects must share the data so the vendor and project budget take this effort into account at the planning stages. Adding this language to RFPs from the start would have helped us move along faster.
Can you describe some of the interesting ways in which people are using the city’s data?
A resident has created an intricate website that uses the District’s data feeds to provide economic development information, including permit status and crime statistics. There’s also crime reports as I mentioned and another called everyblock.
The District’s Apps for Democracy contest challenged residents to utilize information available from DC’s Data Feeds to develop consumer-based applications. Local developers produced 47 applications that were conceived, developed, and delivered in 30 days. Winning applications include a DC biking guide, government purchases over $2,500, parking meter locations, community garden sites, and more.
Our clients here in Ottawa always face change management issues with major projects. What are the change management issues you encountered for the warehouse project and how did you mitigate them?
The most prominent organizational obstacle is that some individuals within government do not always welcome the idea of creating opportunities for the public to criticize government operations. There are select employees that worry that the public will misinterpret raw data or that the data owner will be unable to affect how the data is used after it resides in the centralized data warehouse. These employees often cite budget cuts, limited production capacity, insufficient technology resources, or other top priority projects as reasons not to participate in data democratization.
How did we convince people? It depends on the objection:
- We have no money: If our program has the time, we can do the database work for the agency and create a read-only table in the customer’s database for CityDW’s use.
- This will create opportunities for the public to criticize government operations: we are in a way opening ourselves to criticism but in some cases the public can help the government QA the data, find issues and help us fix those issues.
- Limited production capacity: CityDW can connect and get updates in the middle of the night.
- We don’t have a way to get you the data: CityDW has become very flexible as to how agencies can send and update data. We prefer an Oracle to Oracle connection but as with any government many different technologies are used across the city. In many cases agencies update an Excel report and send a CSV file on a scheduled basis; CityDW automatically ‘watches’ for the emails and updates the data in the data warehouse.
What are the financial realities for executing an open data strategy in government? How do you make a business case for embarking on open data initiatives in these economic times?
You don’t need a massive budget to get started. You can start an open data program with a part-time: project manager, developer and DBA and grow from there once you have users requesting more data. Governments and resident will both benefit with having a centralized location for the data… there will be second set of eyes on the data. Issues may be found in the data that will enable the government to save money or identify revenue sources.
What was the desired outcome for your open data strategy and program, and how have you been able to measure its success?
We wanted the residents to be able to take our data and visualize it or create applications that would benefit other residents and create new ways of looking at the data that we never considered. We wanted another set of eyes on our data, looking for issues that need to be corrected.
All our data is available via other applications, dashboards, tools such as text search, and reports. So in addition to tracking downloads we have to track usage and log-ins of the other applications and dashboards… for downloads in fiscal year 2010 we had almost 2 million downloads. We can also measure our success by the number of incoming requests for more data, dashboards, and reporting environments.
Do you have any suggestions as to how an open data initiative at the municipal level might be leveraged across all levels of government (city, state/provincial, federal)?
Washington DC is a unique situation in that we are a government that has to function as a state, city and county. Building a program that contains the central location for enterprise wide data creates many beneficiaries and not just the residents and press. Once of the most common uses of data are Districts employees. They may not know who in the government controls access to a particular dataset but they do know that the data is published at data.dc.gov. There are no forms to fill out; employees just like the residents can download agency data so they can do their job faster and more efficiently.
A centralized data warehouse allows you to expand beyond open government and data sharing. Once you have the data, reporting environments can be created and used inside the government. Business intelligence applications and dashboard can be built to help executives view the overall ‘health’ of their agency without having to drill down in to all of the data line by line.
Denise Eisner is a Senior Consultant in the Government Service Excellence practice.