The Future of Open Data

Photo: iStockphoto
Photo: iStockphoto

Picture this: You’re a tourist in New York City and after a long day of sightseeing you’re looking for a bite to eat. But before you simply head into the nearest sandwich shop, you look it up on “NYC Restaurant Scrutinizer.” This mobile app isn’t your average restaurant guide: Along with food reviews, you can also see the results from restaurant inspections conducted by the New York City Department of Health and Mental Hygiene. After finding out that the sandwich shop doesn’t properly refrigerate its deli meats, you decide to go to the Chinese restaurant on the corner with an “A” grade. And delicious spring rolls to boot.

“NYC Restaurant Scrutinizer” was developed by a private app developer named Michael Boski in 2009. He beat the government agency to the punch by about three years: “ABCEats,” the health department’s own mobile app for checking restaurant grades, first appeared on iTunes in 2012.

Civic hacking and open data

Boski is a so-called civic hacker, a private citizen who taps into publicly available data to create useful apps for the general population. Since most governments endorse open data policies that disseminate such information, civic hacking is not illegal. The New York City health department, for example, has been posting its inspection results online for anyone to see since the mid-1990s. It’s worth noting that without this policy, “NYC Restaurant Scrutinizer” would not be possible. On the other hand, it was Boski, not the government, who took the first step in making the information easier to consume, which increased its value to citizens.

That is often the case: In its current form, open government data is large and complex and usually lacks the context for everyday use. But not everyone thinks civic hacking is the answer to this problem. For one thing, apps from private developers, like “NYC Restaurant Scrutinizer,” can’t guarantee the accuracy of information they provide. And they usually don’t take into account any downstream consequences that might negatively affect businesses and government agencies.

“The Dynamics of Opening Government Data”

SAP’s Public Services Industry Business Solutions team wanted to find out more about the impact of these open data initiatives on the public and private sector, so it commissioned the Center for Technology and Government (CTG), an independent research organization at the State University at Albany (SUNY) to do further research in the area. The resulting whitepaper, “The Dynamics of Opening Government Data,” examines two case studies of open government initiatives and presents four recommendations based on CTG’s findings.

Next page: Determining the impact of open data on stakeholders

One of those case studies was the previously mentioned example with the New York City health department. The other case study involved the Department of Transportation for the city of Edmonton, Canada, and its release of data on planned road construction projects.

“In both cases, it was clear that simply opening government data – making it available to the public – is not the whole story,” says Anthony Cresswell, a senior fellow at CTG. “It has longer-term consequences that are difficult to predict. In our research, we sought to come up with strategies and policies that help government agencies understand ahead of time how opening data will affect all the stakeholders involved.”

The information polity

Identifying and understanding this collection of stakeholders, what Cresswell and his colleagues call an ‘information polity,’ is one of the key aspects of the whitepaper. “An information polity includes the government agencies that create the data, employees who use the data, citizens who want access to the data, app developers who modify the data, and other stakeholders,” explains CTG senior program associate Brian Burke.

In other words, Michael Boski is part of an information polity, as are the people using his app, as are the restaurants being inspected, as is – most importantly – the government agency conducting those inspections and providing the data. “The timeliness, format, and quality of data that the government provides all affect the usability of that data for developers,” says Burke. “And it is the relationship between these information sources that ultimately influences the value of that data for the public.”

Four recommendations from CTG

How should government agencies go about implementing open data initiatives that maximize value and minimize risk? Cresswell and Burke explain the four recommendations outlined in the whitepaper.

1. Release government data that are relevant to both agency performance and the public interest.

As part of the Open Government Initiative launched by the Obama administration in 2009, U.S. federal agencies published high-value datasets online at Any person can access the Web site and explore a vast amount of information, ranging from the location of every farmer’s market in the country to the average energy consumption by household to the U.S. trade volume in tomatoes. But how many citizens really want to know what the current yield of the country’s tomato crop is? “As government agencies try to balance resources, time, and effort, they should choose to focus on those datasets that hold the most public value,” says Cresswell.

Next page: Recommendations 2-3

2. Invest in strategies to estimate how different stakeholders will use the data.

“Some datasets, like government budgets, don’t lend themselves to use on a smartphone. Others, like restaurant inspection results, make a lot more sense when you connect them to geospatial data so they can be used on the go. If you model how users are likely to interact with the data, you can choose the technology solution that will maximize value,” says Burke. In addition, different stakeholders may want access to different kinds of data. In the road construction case study, for example, a commuter might want to know what the estimated delay on a particular route is, while a construction site foreman digging near a road block might want access to the location of the newly-laid sewer line. Government agencies will have to decide whether they can – or should – invest in collecting new data to suit these different needs.

3. Devise data management practices that improve context in order to “future-proof” data resources.

“Good meta data will help you ‘future-proof’ data resources. It’s a very simple thing, but very powerful,” says Burke. The developer who created the road construction app for the citizens of Edmonton reported that building the app was very straightforward, thanks to the high-quality meta data already provided by the city. For example, the dataset already included geospatial data that was compliant with GIS standards, which made mapping the information easier and more accurate. Good data management practices are essential for government agencies looking to make their data more accessible and useful.

Next page: Recommendation 4

4. Think about sustainability.

Planning for the long-term sustainability of a given dataset is strongly linked to understanding the values and risks associated with releasing the data in the first place. Before restaurant grades were posted online by the New York City health department, restaurants had to display their rating on the premises, but they could easily hide a bad report in a less visible place. Once the agency began posting inspection reports online in the 1990s, restaurants felt the consequences of a bad rating much more severely. As a result, they began demanding more frequent health inspections, so they would have the chance to improve their score. The health department responded to this demand by hiring more inspectors. “If you identify the value and risk of releasing information, you can better predict what additional resources you’ll need down the line,” recommends Burke.

What does this mean for SAP?

While government agencies reevaluate their open data policies in light of these four recommendations, SAP is also examining what this means for the software industry: “CTG’s research gives SAP a better understanding of the challenges that the public sector is facing today to support open data and open government mandates,” says Elizabeth McGowan, director of technology & innovation for business solutions in the public services industry at SAP. “To generate public value going forward, it will be crucial for governments – at all levels – to invest in strategies that open data for exploration by citizens, stakeholders, and other government agencies. We believe that solutions for big data management and analysis will be a critical part of these strategies.”

Follow @SAPOpenGov on Twitter to stay up-to-date on the latest open government news, events, insights.