Benefits of Models and Metrics Go Beyond a Single Project

Chris Schlueter Langdon
Chris Schlueter Langdon

Prof. Schlueter Langdon, how difficult is it to assess the profitability of a software solution upfront?

Schlueter Langdon: With paper, pencil, some data and a Ph.D. in economics, it is as simple as assessing the ROI of a manufacturing system or marketing campaign: What has to be done is to assess the impact of an investment on profit. I am an engineer and got trained to analyzing a part design before making it. So I find it curious when it is considered too difficult to estimate and anticipate how to turn a profit upfront, if, in the end, profit has to be generated anyway. This is where M&Ms or models and metrics come in. One needs a causal model for this, which reveals how pushing one button “X” improves another variable or metric “Y”. Without a model, it is like taking medicine without diagnosis. If the CEO wants you to update IT systems to cut the time between product model changes from two weeks down to two hours, you should know how to best achieve this flexibility.

Which IT variables would have to be measurable?

Schlueter Langdon: Mainly flexibility. Flexibility is a metric and part of a causal model that can link an IT investment with business performance measures. Given today’s fast paced business environment, flexibility has become a very important variable. It is seen as a key enabler of higher sales. One example is the ability to quickly shift production from a troubled product to a star. In the past, many CIOs had emphasized integration – hence the spectacular growth of the ERP business.

What methods have you developed to evaluate the added value that software generates?

Schlueter Langdon: Our research is focused on adapting ROI logic to make it work in IT, so that results are relevant, reliable and consistent. First, we work on better models. For example, to what extent does use of Web services standards increase a system’s flexibility? Secondly, we work on individual variables and metrics themselves. To stick with flexibility, we have worked on turning flexibility from a concept into an IT metric, complete with definition and measurement concepts, because why bother if you cannot measure it. For example, we can complement factual and perceptual measures to help a company benchmark its IT flexibility across business units and departments. We also work on agent-based models and systems. Many of today’s top management problems are about change, and therefore, inherently dynamic, such as technological disruptions, product convergence, and organizational transformation. Outcomes are the result of mutual adaptation. In a nutshell, your success depends on how I adapt to your adaptation of my strategy. Agent-based and complex adaptive system modeling tools have emerged to help solve these problems. For example, forecast values, such as for ROI projections, can be much more accurate and are not necessarily an extrapolation of the past as with traditional statistics or econometrics. I am currently chairing SIGABIS, the Special Interest Group on Agent-based IS of the Association of Information Systems, to advance science and practice in this field.

How does what you are doing differ from pure IT evaluation (ITEM)?

Schlueter Langdon: IT evaluation is nothing like a do-it-yourself-surgery. And ERP software packages are not plug-and-play solutions. You need to have trained individuals sort out the variables, identify the right model and choose measures that are actionable, meaning they can work with the data at hand. It takes experts to get it right the first time. The automaker Nissan, a fierce and highly profitable competitor, relies on a “quant group” staffed with modeling experts to construct M&Ms to increase precision of decision making throughout the business.

Why are existing workflow and simulation tools not sufficient for evaluating IT profitability?

Schlueter Langdon: Tools are often used prematurely – and without a model. Business has become more not less complicated over the years. It takes an engineer to develop a blueprint, it takes an architect to draw a plan and it takes an expert to separate variables and relationships. In order to avoid confusing cause and consequence this separation of relationships must be based on rules. Without rules there is no causal model, without theory there are no rules, and without research there is no theory.

Can you base your work on other metrics, for example on those focusing on the performance of business processes (Business Process Management/ Corporate Performance Management)?

Schlueter Langdon: Theoretically yes – and in fact, it is very desirable to do so. If management is using business process metrics such as key performance indicators, there may be an opportunity to link IT metrics with KPIs. KPIs often provide a positive culture of accountability, risk management and ultimately shareholder value orientation. But unfortunately, IT still suffers from a lack of standard industry metrics, and therefore, benchmarks. In IT there is no equivalent of the Harbour Consulting Report, which provides production performance benchmarks for the auto industry, or J.D. Power indices that provide customer satisfaction and appeal benchmarks. Without benchmarks, IT controlling often has to use the research literature. Our and others’ work on flexibility is a first, small step toward establishment of rigorous yet actionable IT benchmarks.

How can software manufacturers help customers understand the profitability of the products they offer?

Schlueter Langdon: Provide their customers, CIOs and IT executives with business case calculators or decision dashboards and cockpits like, for example, SAP Value Calculator. These calculators have an easy-to-use interface and are powered by an “engine” with a model and data inside. In other words, these calculators are pre-packed M&Ms that would need proper adaptation by experts. Many software vendors already provide reference business process diagrams, so why not offer dashboard toolkits for ROI analysis and project management?

What do you aim to achieve with your solutions? When will the methods and tools you are developing be ready for the market?

Schlueter Langdon: As a researcher I continuously innovate, while my advisory firm, Pacific Coast Research, puts innovation into practice to give companies an edge. For example, we have already constructed the first auto market simulation, which forecasts how disruptive events, such as the next recession, can affect competitors’ profits and strategies. We are also experts in the new field of vehicle relationship management – the IT-enabled automation of the interaction between a vehicle, driver and its environment. Automakers install black boxes to collect usage data. However, key to success is knowing how to use the data or in my language: M&Ms. I am currently planning a VRM research consortium at USC’s Center for Telecom Management.

Why are many CEOs, and even CIOs, hesitant when it comes to developing their own metrics to determine the profitability of their corporate software?

Schlueter Langdon: It’s simply the same as with other innovations: There are always leaders and followers. Just think about the use of email. It took a while for it to become a business tool. And M&Ms don’t make for a casual read. Knowing how and when to use them takes a bit of training.

How can you convince a project manager who focuses on short-term planning to invest in M&Ms for the long term?

Schlueter Langdon: It has been said that the former CEO of Boeing, Mr. Mullaly, ran the business using weekly meetings with key executives to discuss dashboard data. If Mr. Mullaly could run Boeing this way, you can run a project with M&Ms. In addition, let me mention three key findings. First, M&Ms tends to be a stealth weapon. For example, in the auto industry, it is difficult to detect that a competitor’s quality boost is due to M&Ms; furthermore, results are also company-specific, and therefore, difficult to imitate. Second, benefits go beyond a single project. A first M&M is typically only a starting point. Once you can fly with radar you do not want to go back, but you want a better radar. Third, M&Ms are probably the best opportunity to capture, codify and share knowledge. Many employees have valuable experience, which is lost when they leave. This knowledge can be captured in M&Ms. It can also easily be shared throughout the entire organization with M&M powered calculators or decision dashboards. Last, M&Ms provide CIOs and management what rapid prototyping has achieved for software developers. It is a reliable way to pursue a test-and-learn approach, which allows for faster-to-market action.

Can you give us any examples of enterprises that have used metrics successfully in practice?

Schlueter Langdon: We have advised many Global Fortune 500 firms, including leading automakers, telecoms, consumer electronics and entertainment companies. We have quantitatively linked IT and other investments with business performance targets across many different departments including corporate planning, business development, controlling, marketing and quality.