“Set It and Forget It” Saves San Francisco Staff Time

Customer: City and County of San Francisco

San Francisco’s more than 800,000 residents come up with a lot of requests. From sidewalk repairs to campaign finance information, the City and County of San Francisco (CCSF) processes hundreds of requests a day.

Sound time consuming? It is.

That is why San Francisco moved to a Socrata open data portal in 2012 with the goal of digitizing requests for data, updating the data more easily, and publicly tracking progress on requests.

It’s also why San Francisco began using Safe Software’s FME platform to help staff move their daily dose of data automatically onto the Socrata platform.

Open, Transparent, and Automatic

In 2011, San Francisco’s leadership, including Chief Innovation Officer Jay Nath and Chief Information Officer Jon Walton, pushed for an open data portal as a “one-stop destination for all approved City data.” They had visions of greater transparency and convenience for citizens but soon discovered, as many organizations do, that standardizing City data onto one platform improves internal efficiency.

“It’s nice to be able to point to one spot and say to a colleague, ‘You want this information? It’s going to be on this site. You can download it, manipulate it, create views. And, then, also, you can embed it into your own website,’” says Jeff Johnson, Manager of CCSF’s Enterprise GIS Program and Open Data Services at the Department of Technology.

The Ethics Commission’s High Profile Job

One of Johnson’s favorite examples of greater efficiency through open data is the San Francisco Ethics Commission. It turns out that reporters, lobbyists, candidates, and others involved in San Francisco politics closely watch the data the Ethics Commission reports.

“We’re under a lot of pressure to be accurate,” says Steven Massey, Information Technology Officer for the CCSF Ethics Commission.

Before the November 2012 elections, the Ethics Commission decided to use the open data portal to optimize for accuracy and streamline their efforts. Massey worked with Johnson to move all campaign finance and lobbyist data on to the portal and have it automatically publish to embedded charts on sfethics.org.

“We replaced many of the tables throughout our website with embedded Socrata tables so that users can export the data. We've gotten really good feedback from the community about these charts, since they are much easier to read than our PDF financial reports,” says Massey.

In addition, Johnson helped Massey make uploading data to Socrata require, truly, no time at all. Johnson suggested Safe Software’s FME product to bring the Ethics Commission data into Socrata automatically. “I was familiar with Safe Software. I use it in my GIS work, but it can be used for more file types than just geographical information,” says Johnson.

Johnson asked Safe Software to develop a writer specifically for loading a broad range of data files to Socrata, eliminating the need for a unique connector to be written by San Francisco’s busy IT team. Having this writer paid off in significant time savings.

Better Reporting of Ethics Information

Before Socrata and FME, news sources would collect the data on candidate funding by downloading PDFs posted by the Ethics Commission and tallying up numbers by hand. The process often came with many questions for Massey and, sometimes, incorrect results were reported.

In the 2012 election season, reporters from the SF Weekly, SF Appeal, SF Bay Guardian, SF Chronicle, and SF Examiner were all able to report on the data the Ethics Commission publishes without going through that manual process.

“Now, when people call in, you direct them to the site and you’re done,” says Massey.

Johnson is excited about improving both transparency and efficiency for CCSF staff, without asking for much extra effort from them. “Our Open Data policy directs City agencies to load data onto the data.sfgov.org platform, so we need to make it as easy as possible,” says Johnson. “They have a job to do - get graffiti covered, fill potholes, share financial data. That’s their business.”

A Smooth Transition

Massey says that the transition from PDFs to automatic uploads on the open data portal was quick. “Within a couple of days I had what I wanted up and done. Jeff finished the scheduled data updates in August and I had the summaries up right after that with all the graphs,” says Massey. Plus, Massey enjoyed a surprising benefit from the new automated and visualized data that Socrata and FME made possible.

Campaign finance data has received wider attention now that it is more easily available. “Prior to Socrata, campaign finance data was most valuable to reporters. It made it into the newspaper quite a bit. Now it’s in regular demand from a broader audience,” says Massey. Massey considers this a win for public data transparency.

Getting Citizens What They Need, Easily

Overall, time savings, improved transparency, and greater accuracy have meant the most to the CCSF team. “I now have uploads that run every night from various databases around the city. We call it ‘set it and forget it,’” says Johnson.

Johnson is even canvassing city and county agencies to instruct his colleagues about the use of FME along with Socrata. “FME is a very simple tool. That’s the beauty of it.” Johnson wants to keep San Francisco’s staff working hard on what matters, serving more than 800,000 citizens with the data and support they need everyday, as easily as possible.