Open Data Implementation in Six Steps

A Phased Approach

We highly recommend you approach your implementation in phases. In this section, we outline the different phases needed and the tangible benefits at each step. Some of these phases can be completed in a few days while others take longer. You may find it helpful to combine or rearrange them based on your specific situation and goals.

General recommendations:

  • Encourage community feedback from end-users, developers, and advocacy groups throughout the implementation. Start getting feedback early and continue the engagement as you grow. It will be an invaluable asset to you.

  • Reach out to developers. You can start a developer program or run a hackathon at any time during your project. The only prerequisite is to have enough valuable data available to warrant creating apps. Opening up the channel of communications will take the guesswork out of the process. They will tell you what they need. See our “Four Essentials of Developer Evangelism” in Chapter 8 for inspiration.

  • Get your co-workers involved early. Empower data stewards and program managers in agencies and departments to publish and curate their own data, especially when you get to phase four on expanding your program with agency participation.

  • Build the program with sustainability in mind. Whenever possible, use automated publishing to eliminate manual labor. This is especially important for frequently changing datasets like permits, licenses, 311 request fulfillment, and 911 calls, to name a few.

Read Socrata’s interview with Brett Goldstein, the former CDO and CIO of Chicago, to learn more about Chicago’s phased approach to open data.

Think About a Pilot to Start

Pilots are a great way to get your feet wet with open data. A great number of Socrata customers started out with a limited-scope, limited-time pilot, usually for three to four months, but it can last up to a year. This approach has several advantages, such as:

  • Gives you an opportunity to test the technology, try out several different approaches, discover what’s possible using real-world data, and socialize your new platform with your organization’s leadership, your co-workers, and citizens.

  • Three to four months is enough time to run a developer event from start to finish. Having a platform on which you can support a successful hackathon, or a longer-running developer challenge with real data, user-friendly visualizations, social collaboration, and APIs, is a powerful way to put your initiative on the fast track.

  • When you launch a pilot site in beta status, you can be more nimble, set up the right expectations internally and in the community, and give your collaborators the freedom to experiment with new ideas. This can be liberating experience since a quest for perfection can impede rapid progress.

Phase 1 – Start Small

What’s the first dataset you should publish? Where do you start?

What if we told you that several open data initiatives started with one dataset? That’s right, just one. Kansas City, Missouri, and the United Nations Development Program started this way. They focused on a high-value dataset that they wanted to make available to their customers through multiple interfaces: downloads, APIs, visualizations, filters, embeds, and sometimes an app, all on day one.

Our point here is it does not matter where you start, as long as you start. Think like a lean startup and adopt a “Ship it!” mentality. Your community will support any meaningful first step towards transparency and information they can use.

Steps and Recommendations:

  • Go for the easiest tasks first. What data is already in the public domain, on your .gov site(s), as downloadable spreadsheets, PDFs, and ZIP files? Get all that data on your new open data site and bring it to life with easy-to-create maps, charts, guided browsing, and other interactive experiences.

  • Make maps interactive. Grab all the maps that are currently available on your site(s) as shapefiles and KML files and convert those quickly to online maps that are interactive.

  • Get your main team of data publishers involved from the outset. Your initiative will succeed and scale a lot more quickly if you don’t have publishing bottlenecks. You can empower the data stewards from the various agencies to take a leadership role within their department.

  • Open up the feedback channels on day one. Get your community to help identify what’s important to them. Start an organic process that will, hopefully, take on a life of its own.

  • Never worry about converting files. Your open data platform will take care of converting files into multiple formats or creating APIs for these datasets.

By the end of phase one, you will hopefully have made some strides towards making public data more discoverable and usable for your constituents, and with little effort. Phase one, check!

Phase 2 – Get Transparency Done

Reaching full transparency might take a while but is essential for true open data success. Here in phase two you’ll learn how to get all of the data we discussed in Chapter 6 on “The Data Plan” online, in a usable format.

Steps and Recommendations:

  • The easiest data to publish first is usually financial transparency and personnel data. This is because it is typically well-organized and can be easily extracted from your back-end financial systems as clean spreadsheets.

  • Once you load it on the platform, do a usability pass to make sure that the data makes sense to your constituents who don’t know your internal codes and acronyms.

  • Build helpful visualizations, especially charts, and guided browsing filters to help people interact with and understand the data with little effort. For example, if it is budget data, a pie chart of budget by department might be helpful. If it’s expenditure data, think about building facets to browse the data by department, by month, and by type.

  • You can then move on to ethics data, like campaign donations and lobbyists’ salaries, which may require more clean up and data collection for your team. It’s a worthwhile effort.

See how Deschutes County, Oregon shows its budget in a graphical format using Socrata’s Financial Transparency Suite
More Examples
  • Explore the State of Connecticut’s OpenBudget portal.

  • Los Angeles County’s comprehensive Open Data Portal, including budget and performance resources

  • With, President Obama made good on a campaign promise to publish campaign funding and other data.

  • Chicago Lobbyists” is an app developed by Open City, a group of volunteer civic hackers in Chicago.

Phase 3 – Bring Developers Onboard

One of the primary goals of open data programs is to connect with your local developer community and get them what they need to build apps and other features with your data.

It’s best if you take a backseat role. Find open data enthusiasts in your tech community and let them lead the effort to build apps. Tom Schenk, director of analytics and performance management for the city of Chicago recommends “humility” in order to create an engaged developer community. He suggests letting these intelligent, creative people tell you what data would interest them most or how the data could be served up in a more useful manner.

“Chicago has the best developer community. I know many of them by their first name. They know a number of people here, from Brett Goldstein the CDO, to myself, to other developers. We talk with them on a very regular basis. Every Tuesday, we’re at their meet-ups. We don’t drive it, we’re just there in attendance. We work on projects like everybody else.”

Tom Schenk, Director of Analytics and Performance Management, City of Chicago

Steps and Recommendations:

  • Familiarize yourself with successful developer and apps programs out there, such as the NYC Big Apps competition, Chicago Digital, Evergreen Apps, and ongoing developer challenges in the federal government.

    Read how the City of Seattle created a transportation app in a Hack the Commute weekend including how to set up, promote, and run the event.

  • Look for local leaders in the tech community who are interested in open data. Meet with them personally. You may find them at your local colleges, or even high schools. Or, you could meet them at tech community meetings for any number of topics of interest to developers.

  • If there isn’t one already, start a regular meet-up for developers to discuss and work on open data projects. Choose an appealing spot with room for them to interact or do some work alone. And, try to pick a location with plenty of bandwidth so that they can all be online at the same time.

  • Ask them how you can help. Maybe your 911 call data comes in a format that is difficult to work with for certain projects. Or, perhaps you’re updating some data too infrequently to warrant a successful app for citizens, such as snowplow locations.

  • Create opportunities for recognition, be it an annual or bi-annual award, or prizes in a hackathon. Regularly include their achievements in your press outreach about the program.

Once a year, the National Day of Civic Hacking brings together hackers, designers, government employees, and more to solve real problems and build revolutionary new tools.
More Examples
  • The state of Washington, King County, and the city of Seattle host the Evergreen Apps Challenge to “help people create useful experiences out of the data that government creates on a day-to-day basis.”

  • The city of Chicago highlights all of the outstanding apps created by local civic hackers on its Digital Chicago site.

  • Check out the “Winners Gallery” for the latest New York City BigApps event.

Phase 4 – Increase Agency Participation

Your open data program will truly flourish when you can get broad participation from your agencies and departments. Getting their active participation often rests on your ability to articulate how the open data platform can help them do their jobs more efficiently and create better outcomes. Customize your message to them.

Remind them also that they are the primary beneficiaries of the Web and mobile apps that will become part of the service and information-delivery infrastructure for their programs. In addition, the program helps to break down internal silos and facilitate information sharing across agencies.

“Our goal was to improve the transparency. But, from an operational standpoint there is the added benefit of staff being able to have better access to the data. It is an efficiency gain where we can share data and build tools that find connections in the data.”

Chad Janicek, Director of the Office of Management and Budget for Weatherford, Texas

Steps and Recommendations:

  • Engage with your internal stakeholders in other agencies, operating divisions, and departments.

  • Focus on agency CIOs, department heads, public information officers, communications teams, and Web managers. Ask them about the information bottlenecks in their own processes. Ask questions like, “How do they currently create and update maps, reports, apps, and other data-driven content? Do you have to rely on IT staff for every update? Are constituents satisfied with their experience and the level of access to information?”

  • Show your colleagues how they go from raw data to an interactive experience on the Web in minutes. Show them how they can create visualizations and maps and embed them on their agency sites without any technical help.

  • Ask them to sponsor an app for your app contest based on their needs. Maybe this is the opportunity for them to get that mobile app they’ve been waiting for.

  • Showcase studies from other government organizations where their peers have transformed their business processes.

  • Identify easy opportunities to help them, even with publishing one high-value data set. Get them to register for online courses.

Remember, transparency is served every time they publish a dataset, even if that was not their primary motivation.

Phase 5: Optimize for Efficiencies and Cost Savings

Phase five is a defining phase in your open data program. Open data is not just an opportunity to increase transparency and improve the citizen experience. It is also an opportunity to save time and money by adopting new technologies that make the flow of data and information more efficient. This phase deals with practical opportunities to realize those cost savings, in IT and in every participating department.

Steps and Recommendations:

  • Move requests for information to self-service channels. Start by convening a meeting with internal stakeholders who deal with public information requests, such as city and county clerks. Create a quick inventory of common, repetitive requests for information, and identify the most logical candidates for proactive disclosure in self-service delivery mode.

    These can range from FOIA-related requests, to everyday requests for maps, business listings, licenses, and permits. How much would it save your organization if 20 percent or even 10 percent of high-touch interactions could be deflected to self-service channels?

  • Boost everyone’s productivity. Saving information workers’ time is an area where you have almost limitless potential to innovate. The productivity gains you can start implementing right away with your open data platform fall in two areas:

    1. Data and information discovery: Your employees will have a central resource where they can easily find data from their colleagues in other departments.
    2. Data collection and publishing: Your colleagues who are creating and updating interactive information on the Web — maps, charts, reports — can now do in minutes what used to take them days and weeks to do. In many cases, when you take advantage of automated publishing from your systems of record to interfaces for citizens, time-consuming manual updates can be eliminated altogether.

Read the Three Tips to IT Savings with Open Data

The Secretary of State for the state of Oregon avoided spending $500,000 on a custom application to run monthly updates of their trademarks data by posting the information on instead.
More Examples
  • The Recreation and Conservation Office at the state of Washington published its “State of Salmon in Watersheds 2014 Report” using and creating 350 interactive charts.

  • The city of Seattle’s near real-time “911 calls” map lets citizens follow crime activity and emergencies in the city.

  • Cities across the U.S. are federating their data on

  • The city of Chicago, Cook County, and the state of Illinois have federated their data on to a single, citizen-friendly site,

Phase 6 – Federate Data with Neighboring Cities, Counties, and States

The implementation plan so far has focused on data within your own walls and the experience of your employees and constituents. This next phase deals with two-way exchanges of data with neighboring cities, counties, states, or federal agencies, as well as multi-stakeholder collaborations to create converged data portals at a regional or national level. In the world of open data, this is called data federation, or sometimes “data catalog federation.”

Benefits of Federation

  • Enables the comparison of quality of life and other performance indicators across states, to provide new insights and inform decision-making in our communities. A likely initial focus will be comparisons of health data.

  • Creates derivative information products based on intelligent aggregation of state-level data, e.g. a national directory of community health facilities.

  • Provides citizens with greater access by enabling interactive data discovery across state catalogs, using a unified search interface and normalized topical categories, such as health, economy, public safety, environment, etc.

  • Lowers the bar for developers by providing consistent APIs from one jurisdiction to another and normalizing the schemas for high-value, state-level datasets that are semantically identical, such as unified crime reports.

  • Increases the reach of each participant’s open data efforts by developing a consumer brand that can aggregate audiences and build awareness on social networks.

  • Contributes to the development of common open data standards.