Ever wonder how banks determine what limit to set on your credit card, or what rate to give you on a mortgage loan? They don’t rely on guesswork, of course; banks use sophisticated algorithms and models to predict how likely you are to make future payments based on your past behavior as well as external factors like the security of your job. Ultimately, the credit limit or loan rate that you’re able to obtain reflects the risk associated with your specific customer profile.
This process relies in large part on predictive analytics, a technology that looks for patterns in data and then applies algorithms to predict the future outcome of those patterns. This kind of analytics has been used in the financial services sector for over 20 years. Lately, more and more industries beyond the world of finance are becoming interested in the possibilities of predictive analytics. Why the sudden popularity? Two words: Big Data.
How companies profit from Big Data
Big Data presents some serious challenges for most businesses today. Beyond the problem of storing and maintaining multiple terabytes, even petabytes, of information, many companies across different industries struggle with the question of what to do with their data. How can they use it to generate revenue, save money, or improve processes? Predictive analytics provides a way.
On the following pages, we explore the question, what IS predictive analytics, from the origins of the technology, to concerns that companies should address before adopting it, to future developments in the field.
Where does predictive analytics come from?
Over time, businesses have always sought new and better ways to make informed decisions. In the past, they mostly relied on employees’ experience in the field and their “gut instinct” to guide business decisions.
Then, as record-keeping systems in the enterprise improved, companies were able to analyze historical business data more easily to search for interesting patterns and anomalies. These analyses, however, were backwards-looking, and employees usually had to interpret the insights themselves to predict what would happen next.
Saying goodbye to gut instinct
This is where predictive analytics diverges from the old methods of analysis. After identifying patterns in historical data, predictive analytics software uses advanced algorithms and models – not employees’ interpretations – to determine how the pattern will continue in the future.
The rapid growth of data within companies and on the Web contributed, more than anything else, to the emergence of predictive analytics. It’s no wonder that the financial services sector, a branch that has long dealt with massive amounts of data, was an early adopter of the technology more than 20 years ago.
Today, having good data for analysis is still the most critical aspect of predictive analytics. That’s why companies interested in using the technology should first have solutions for enterprise information management in place, such as master data management and data governance.
How are companies currently using it?
Adoption of predictive analytics varies widely from industry to industry. The airline industry has long used this technology to optimize prices based on the number of seats available on a flight. Retailers have used it for many years to analyze what products customers are more likely to buy based on their past behavior. And as previously mentioned, the financial services sector has been using this technology for over 20 years.
In comparison, the healthcare industry is just getting started with predictive analytics. Due to a changing market, many healthcare providers in the United States are looking to make their processes more efficient, for example in forecasting patient outcomes. And the manufacturing industry, which in the past mostly used predictive analytics to optimize its supply chain, is now applying the technology to different problem areas, such as machine maintenance. Companies can forecast when a machine is likely to break down and which spare parts will be needed to repair it. Thus, they are able to shorten the downtime of broken machinery and limit the amount of lost revenue.
Introducing the Chief Science Officer
The fact that more industries are thinking about how they can use predictive analytics in new ways is also reflected in the increasing number of Chief Science Officers and Chief Data Officers being appointed in the business. These new positions indicate a strategic shift. Companies are looking to gain a competitive edge through analytics and data, and the Chief Science and Data Officers will be leading that drive in the coming years.
Another major change has occurred on the user side. Until recently, predictive analytics was strictly the domain of statisticians and advanced programmers. This sometimes resulted in a bottleneck when too many requests for analyses came in from the business. Forecasts could be three or five days old by the time employees got them back. Now, more and more business users want to have direct access to the tools, so they can conduct analyses themselves.
Next page: What concerns do companies have?
What concerns do companies have?
Some call it a “democratization” of technology, but for many companies, it’s their biggest concern: What happens when powerful predictive analytics tools land in the hands of inexperienced business users? Could they end up making wrong decisions because they don’t understand the methodology?
The answer is two-fold. First, the tools should be constructed so that it’s impossible for business users to carry out an incorrect analysis. Developers should make it impossible for users to choose the wrong parameters, for example. Making the tools user-friendly and understandable for the non-programmer crowd is essential. Second, as analytics becomes a more strategic factor and competitive differentiator in the business, companies would be wise to give all employees some training in the field.
Labor shortage strikes again
Another main concern businesses need to address is that they often don’t have enough employees with advanced analytical skills on their payroll. And as more industries turn to analytics to make sense of their data and to profit from it, the market for data scientists will only grow more competitive.
Besides the people aspect, companies also need to make sure they have the right technology in place before implementing predictive software. First and foremost, this entails enterprise information management, and going one step further, knowing what internal and external information is relevant for each analysis. In addition, in-memory databases are crucial for companies to be able to process and deliver predictive analytics in real time. And finally, end users need a visualization tool to make sense of the results.
What does the future look like for predictive analytics?
One of the main advantages that predictive analytics currently offers is a competitive edge in the market. But will that still be a differentiating factor if more and more companies adopt predictive analytics? In fact, it is likely that there will be some leveling of the playing field in the future. But even then, not every business will be using predictive analytics. Some may not have the technology in place, and others may not have the analytical skills in-house to identify use cases and develop the algorithms.
Those businesses that do apply predictive analytics will probably be using more sophisticated algorithms and on a more granular level. The need for increased scrutiny of predictive models was made clear by the widespread defaults on home mortgage loans in 2008. Since then, the financial sector has seen an improvement in the quality of algorithms and an increased focus on the validation and maintenance of models. In the future, this trend will likely spread to other industries.
Furthermore, with more predictive analytics tools destined for business users, not just statisticians, vendors will have to work on making analytical insights accessible to a broader audience. User-friendly, understandable visualization tools for the end user are on the horizon.