Why Does My Organization Need Open Data?

What Is Open Data?

Our definition of open data for this guide focuses on its use in government or the public sector, leaving aside philosophical aspects:

“Making data that belongs to the public broadly accessible and usable by humans and machines, free of any constraints.”

We like this definition because it is as important to eliminate technological and usability barriers as it is to remove legal barriers, such as distribution and copyright restrictions.

Why? Because the goal of open data is to take this valuable resource out of government database silos where it sits idle, or at best underutilized, and put it into the hands of people who can unlock its value.

The rest of this guide will focus on just that: How to put this valuable data to work and make it as useful as possible.

See the roadmap that moves raw data from disconnected databases, through data catalogs and dashboards, to platforms and APIs that allow both private and public development of applications.

Defining Open Data Terms

Open data and open government are related, but they are not the same. While an open government strategy must include open data as a component of enabling transparency and engaging citizens, it is also about a commitment to open public meetings; releasing public information in all its forms, if not proactively at least in a timely fashion; and engaging the public in decision-making. Open government is also a general mindset, backed up by clear policy, that citizens need to be empowered with information and a voice so they can hold their government accountable.

A good open data strategy should support open government goals by making structured data that relates to accountability and ethics — like spending data, contracts, staff salaries, elections, political contributions, and government performance — available in machine- and human-readable formats. Being financially transparent about spending and budgeting decisions helps to increase citizens’ engagement with government, as well as their trust in government. Read about the evolution of financial transparency, from paper to PDFs, as well as what’s next for financial transparency.

3 Examples of Data-Driven Government in Action

Making data available, and ensuring complete financial transparency is just the initial step: after the data is available, the next step is to put it to work, both identifying problems (e.g., neighborhood X has sharply higher crime rates than the city’s average), setting goals, and measuring the results of steps taken. Read about three open data success stories where implementing performance measurement standards helped solve problems, encourage collaboration, and enforce accountability.

Brief History and Key Open Data Milestones

In addition to serving as an essential instrument for a better democracy, open data also places valuable information in the public domain to fuel innovation and a stronger economy. We can see two big examples of this in the release of weather data from the National Oceanic and Atmospheric Administration in the 1970s, which spawned the weather industry, and in the 1990s release of GPS data and the transformative location-based services that developed as a result.

Here’s a brief timeline of major milestones in open data:

  • Early pioneers: When open data catalogs — sites where people can download datasets — started to appear in 2007 and 2008, modern-day open data pioneers saw the economic potential for new industries, just as entrepreneurs of the ‘70s had taken note of the value of weather-related data. Washington, D.C., San Francisco, and Seattle were amongst the first cities to release public data online. Medicare was also an early pioneer.

  • The Sebastopol meeting: Thirty Internet and open government activists met in the California town in 2007, developing eight guiding principles for defining open data.

  • Data.gov launches: The site went live in May 2009, and is a home for all the data collected by the government.

  • Open Government Partnership: Internationally, the U.K. open data initiative was the most prominent, which helped create a global movement culminating in the formation of the Open Government Partnership (OGP) in September 2011.

  • Digital Accountability and Transparency Act (DATA Act) signed: President Obama signed the DATA Act into law in 2014, after its unanimous passage through the House and Senate. Fundamentally, the DATA Act exists to improve citizens’ ability to track and understand how the federal government spends tax dollars.

  • Financial Transparency Act of 2015 introduced: This Act requires financial regulators to collect machine-readable data from banks and publicly traded companies, and other entities they already regulate.

  • Integrating relevant data: Socrata establishes partnerships with Yelp and Zillow so that public information about restaurant health inspection grades, neighborhood safety, and more important data can appear where consumers are already looking. (Socrata CEO, Kevin Merritt, discusses these partnerships.)


Why Open Data? Why Now?

Open data represents a unique opportunity for public sector organizations to apply innovative thinking to a resource they already have in abundance — their data. They can use it to find new solutions to age-old problems.

The challenges government organizations typically face are well documented. They range from bureaucratic inefficiencies to citizen satisfaction and fiscal constraints. And, they all lack satisfactory solutions.

So how would open data help? Consider the following basic framework for rethinking the strategic role of data in the context of today’s government and all its stakeholders.

1 – A Changing 21st Century Constituency

Transparency is the new default

According to Socrata’s “Open Government Data Benchmark Study,” 67.5 percent of people surveyed believe that, “In the 21st century, if government data is supposed to be public, it should be available online.” Further, 67.9 percent believe that, “Government data is the property of taxpayers and should be free to all citizens.” By a margin of three to one, respondents in this survey said they are more likely to vote for politicians who champion data transparency.

In an interview with Socrata, Mike Powell, chief innovation officer in the state of Maryland, put it best when he said, “The question is no longer about whether data is public or not. The real question is, is it accessible and usable?”

Quality of life improved through data access

We’re accustomed to getting flight arrival information, restaurant ratings, and real-time news from across the globe instantly online. But, we can’t easily find out if crime in our neighborhoods is going up or down, what’s in the next budget, or whether government programs are working as expected. Access to data and information permeates our daily lives, yet a great deal of quality of life information is idling in government database silos.

The citizen experience matters

Past generations might have been content to pick up a phone and wait for a government operator to get the information they need, but today’s constituents expect their experience with government to be every bit as easy as using Facebook, Yelp, or Google Maps. And that experience needs to be available on the Web and mobile devices.

Maybe it’s an unfair comparison, but if a government website looks like a relic of the ’90s, users may simply assume that their government is doing a bad job, or simply abandon the experience. It is time to bring the citizen information experience up to modern consumer standards.

2 – The Changing Nature of Government Work

Government employees are also changing. The demands on them for performance and productivity have grown (while resources have shrunk), along with the availability of modern technology to help them meet those demands. Government organizations –— like all other big organizations — are undergoing a profound change from process-centricity to information-centricity. In this mode, the key to higher performance depends on employee empowerment and the availability of data and information to make fact-based decisions in a transparent and collaborative fashion.

Employees need to collaborate and share data across silos

Gone are the days when government organizations could function without sharing information across agencies, departments and traditional organizational silos. Modern work requires the free flow of information between employees in different agencies. Almost every big problem that government is trying to solve involves a multidisciplinary approach.

Consider efforts to curb childhood obesity, for example. They require a collaborative effort between government and community professionals in health, education, and social services, at a minimum. How can these groups be effective if they can’t share data and converge crucial information about their programs, their strategies, and their impact in the communities they serve? No wonder some of the most avid users of open data platforms are government employees.

Services delivered quickly and efficiently

In our digital era, efficiency means people can do in minutes or hours what used to take them days and weeks to do. While not at work, almost every program manager, e-government professional, or public information officer in government is able to make a personal video of their kids’ vacation highlights, post it on YouTube, then share it on Facebook. But, at work, when it comes to making a simple map of boating facilities or ballot box locations, they have to rely on multiple layers of bureaucracy and tasking overloaded IT and GIS professionals to do that for them.

With modern technology, there is no reason why anyone should accept such inefficient processes. It does not serve them well in their mission, nor does it serve their constituents. Modern information workers in government need to be empowered to support their programs and mission as effectively as possible.

Oregon Marine Board public information officer, Ashley Massey, revolutionized how Oregon boaters received information about boat launch locations, permit purchasing locations, and more. She also did it without help from her already overworked IT department. She was able to take advantage of Oregon’s open data portal to create useful, real-time visualizations.

A high performance culture with data-driven decision making

Most government organizations want to develop a practical performance management program that helps people make facts-based decisions and improve performance.

“Leadership for open data initiatives should move from people dealing with communication, civic engagement, and the likes, to people who are responsible for government performances and budgets.”

– Andrea Di Maio, VP Distinguished Analyst, Gartner

Open data has increased the flow of information. By doing so, it has created an opportunity for government leaders and their teams to analyze it in order to improve outcomes, look for inefficiencies, and communicate better with their constituents. The city of Chicago, for instance, is using its open data platform to collect, measure, visualize, and communicate performance data to its residents. Similarly, the city of Edmonton, has developed “The Citizen Dashboard” where transportation data can not only be visualized but citizens have ways to connect, give feedback, and offer suggestions for better performance.

Performance Metrics – Transportation – Pothole Repair
Powered by Socrata

3 – Leveraging the Community for Innovation

A few years ago, it would have been hard to imagine the White House launching a National Day of Civic Hacking or that one of the most influential forces shaping the government of the future would be an organization called Code for America. A bunch of coders with computer science degrees shaping government?

That’s innovation and it’s part of a bold vision first introduced to the world in 2010 by Tim O’Reilly, open source advocate and founder of O’Reilly Media, Inc. He called the vision “Government as Platform.”

This innovation ecosystem is also powered by community activists and advocacy groups influencing policy and, increasingly, government organizations. They are pooling their data to create converged data portals that combine information resources from neighboring jurisdictions, and even at a national level.

Hear O’Reilly describe “Government as a Platform.”

Enabling civic developers

Would you like to see restaurant health inspection data on Yelp? How about receiving an alert when a product you bought for your child gets recalled? Would you be willing to adopt a fire hydrant and commit to keeping it visible and available when heavy snow falls in your town?

Civic developers are already creating applications with government data so that these possibilities can become a reality. As long as the data is accessible and available in formats they can use, civic developers can make all sorts of useful services possible. (More on that in Chapter 8.)

This tremendous potential for innovation in this developer community should compel every government organization to join the movement and contribute their data to the next generation of citizen services.

Check out these leaders in the civic developer community:

A civic hacking group uses public data to monitor building temperatures and protect New Yorkers from the the cold.

Support open government advocates

Much of our public discourse about the issues of our time — the U.S. federal deficit, gun control, and the influence of money in politics — happens in a “fact vacuum.” Too often we rely on ideology and preconceived ideas to come up with solutions rather than facts.

Why not shine some data on these issues? That’s exactly what organizations like the Sunlight Foundation and Socrata customer Taxpayers for Common Sense (TPCS) are doing. The Sunlight Foundation works to make “public” data mean “online” data. Taxpayers for Common Sense researches and reports on how federal dollars are actually spent, among many things. More community organizers, often paired up with developers, are building even more grassroots efforts in support of open government initiatives in their city, county, state, or country.

Old methods of feeding information to concerned citizens, like fulfilling Freedom of Information Act (FOIA) request one-by-one, are being sped up dramatically by creating sources of machine-readable data. Not only are citizens empowered with the tools they need to create positive change, but government agencies can hopefully reduce the workload caused by one-off public information requests.

An open data portal reduces FOIA requests

See how the Bureau of Alcohol, Tobacco and Firearms is streamlining agency operations, and eliminating time spent fulfilling FOIA requests by making data readily available.

Problem solving starts with the city next door

There is a tremendous new opportunity made possible by open data: sharing of data between neighboring cities, counties, and states, as well as pooling data to create national resources like data.gov/cities. We call this pooling of data “federation.”

A real-life example of data federation has occurred in the Chicago Metropolitan area. Not too long ago, if you lived in the city and you were looking for healthcare information, you had to go websites for Chicago, Cook County, and the state of Illinois, separately. Fortunately, the three governments decided to take the bold step of creating a converged open data site that combines their data and gives residents one interface to all their data called MetroChicagoData.

We think this is an exciting improvement to the citizen experience. It is especially exciting that federation of data can be accomplished so easily with current technologies.

We believe that further data federation will lead to breakthrough insights for cities, counties, and states in specific domains like health, public safety, and more.