San Francisco, the lovely City by the Bay, is generally a safe place to live, especially when compared to other cities of comparable size. Homicides are about half of what they were a few years ago, and property crimes have continued to decline as well. With a booming tourist economy and resurgence in tech start-ups, it is no wonder that more and more people are leaving their hearts in San Francisco.
However, what appears to be a completely positive picture at the aggregate level turns out to be slightly less so when put under the lens of big data analytics. And that is exactly what a bunch of high school kids from the KIPP King High School in San Lorenzo, CA had the opportunity to do. They were working with SAP as part of a groundbreaking program called HANA for Humanity which enables students to work with data from non-profits and/or government agencies and help solve some of humanity’s most pressing problems using the SAP HANA in-memory database platform.
They analyzed around 1.5 Million crime records from 2003-2012 that the City has made publicly available (https://data.sfgov.org/), and overlaid additional data sets (e.g. location of liquor stores, presence of graffiti, presence of a police precinct in the neighborhood etc) to get a more comprehensive picture of the crime situation in San Francisco. When geo-location was taken into account, several hundred million data points had been collectively crunched to reveal the true face of crime & public safety in one of the most iconic cities on the planet.
What the kids discovered was at the same time instructive, fascinating, mind-boggling and downright scary. For example:
- The 800 block of Bryant Street has had over 30,000 documented instances of crime in the last 10 years, which translates to about 1 every 3 hours. From a pocket picked to a business burglarized to a hapless tourist assaulted, the list goes on and on. And this is from the list of crimes that have actually been reported to the police department. (Just in case you are wondering, the next spot down on this list belongs to the 800 Block of Market Street which comes in with about 7,000 crimes, or a crime every 12 hours)
- There are 10 liquor stores that have a vast majority of crime that happens within a mere 0.2 mile radius of those stores. And those stores are all within a 2 mile radius of each other. (Makes you wonder if this is a mere correlation or causality can be established as well. To my mind, it is hard to pin this level of crime down to mere coincidence).
- Crime against property continues to be an ongoing problem; and though violent crime appears to be under control, larceny/theft and vehicular theft account for 27% of the City’s reported crime incidents. (No surprises why cost of vehicular ownership and insurance is so high in the City).
- San Francisco is a sports loving city but every time there is a home game of the 49ers or the Giants, there is a 20% spike in crime. What was fascinating was that criminals too appear to have a notion of a workweek, with weekend games resulting in much larger crime spikes than when the game was held on a Monday. Not only that, crime levels varied depending upon the team we were playing against, and a win for the home team typically generated more crime than a loss or a tie.
In the end, we found ourselves dealing with public data that had a limited depth to it e.g. we didn’t know the identity of the perpetrator, his/her demographic information, prior arrest and conviction records etc, all of which would have contributed greatly to developing deeper insights that the City can use to make the right public policy changes to positively impact the daily life of its residents.
What the kids did discover, however, was that Big Data technologies like SAP HANA can have a real and meaningful impact on a very pragmatic basis, and that analytics can reveal layers that may otherwise be hidden at an aggregate level. For e.g. knowing which neighborhoods to police at what time of day, determining the optimal route for the police patrol car, ensuring that broken street lamps are promptly fixed and graffiti cleaned up before it begins to fester….these are all small actions that City Hall can take to ensure that San Francisco remains not just an iconic landmark but a safe place to call home for its 800,000 residents.
As the great Tony Bennett song goes:
“The loveliness of Paris seems somehow sadly gay
The glory that was Rome is of another day
I’ve been terribly alone and forgotten in Manhattan
I’m going home to my city by the Bay”