A few days ago I happened to be watching a re-run of Minority Report, the 2002 neo-noir science fiction starring Tom cruise as “Precrime” Captain John Anderton. As many of you might remember, the movie was set in the year 2054, pretty far out in the future. What is fascinating is that in a short ten years since the movie’s release, we have reached the point where all the strange precrime stuff that Cruise and his psychic “Precogs” dealt with is fast becoming a reality. The NSA spying scandal is merely the tip of this gigantic iceberg that may have roots in the US but impacts citizens and organizations round the world.
In the last couple of years, Big Data has gone from being a mere buzzword du jour to being a reality that is beginning to make a significant impact in all facets of our public and private lives. Whether it is social media sites serving finely tuned ads on the fly, or cities delivering better services to their residents or governments spying on their people, the impact and the potential of Big Data is only now beginning to be realized.
A lot of the effort appears to be focused on the analytical side of things i.e. using the right kind of Big Data tools to draw insights that have hitherto been buried in layers of data. Which is what the NSA is saying it is using the data trove for i.e. protecting the homeland from terrorists who would do it harm.
What is generally lesser known is that if you couple the advances in software technology with core mathematics, you can get pretty accurate predictions based on the high velocity data crunching (“predictive analytics”). Suddenly, it is not just what the NSA and most providers with a free or ad-based service model (Facebook, Google et al) know today but what they can predict about our individual or collective behavior tomorrow – that is what makes things very interesting, or scary, depending on your perspective!
After the London riots in Aug 2011, a Capgemini developer called Peter Chapman created the prototype Detective HANA application at one of SAP’s InnoJam competitions. He used cell tower records to identify which mobile phones (and by extension their owners) were present in the vicinity of the riots, and then narrowed down the list by filtering on the last known numbers of repeat offenders and parole violators. Each mobile phone generates over 7,000 cell tower records every year, so for a city the size of London, with 8 Million residents, over 56 Billion records are generated. Parsing those in real-time can be a challenge by itself, and which is where Big Data technologies like SAP HANA and others come in.
A company called Secure Alert based in Utah does something similar. They analyze the movements of several thousand ex-cons using data from ankle monitoring devices to identify patterns of suspicious behavior that can be passed on to law enforcement. You can see where this is going, tracking and surveillance is no longer the domain of professional spooks.
But the drive to utilize the power of Big Data is not just limited to big brotherly behavior only. The City of Boston is using its acclaimed Citizens Connect app to enable the general public to report city-wide issues related to things like potholes, graffiti, downed trees etc, and the request is then instantly directed to the appropriate person whose job it is to take care of those issues. Not only that, but a status of resolution is also available so the reporting citizen can see what exactly happened to that request, and the City itself publishes its own performance in a live dashboard called Boston About Results (BAR), which raises the performance management bar, if you will, to a whole different level.
In the Minority Report, the movie, the Precrime program was shut down and the prisoners were all released. With the political turmoil surrounding the unmasking of the NSA spying program, and the fact that Big Data is still in its infancy, it is going to take a lot of policy work if we are to adequately safeguard both the privacy and security needs of citizens.
Until then, Pandora’s Box is wide open and sales of George Orwell’s 1984 are skyrocketing.