Measurement Issues in BI

Introduction

We are now firmly ensconced in the age of Big Data. Wal-Mart has passed the U.S. Government in the amount of data that they keep – the result of keeping the details of every one of more than 1 million transactions per hour. Add to that the millions of daily Facebook and LinkedIn postings, Twitter Tweets and GPS data maintained by cell phone companies and the ‘Big’ in Big Data may not express what we are experiencing in big enough terms. All of this data has made the question of what and how to measure (and how to use the results for decision making) even more important in contemporary information systems.

The last two decades of the 20^th century witnessed tremendous progress in systems for information support – flexible and adaptable systems to support decision makers and to accommodate individual needs and preferences. These model- or data-driven or hybrid systems, called Decision Support Systems (DSS), incorporated diverse data drawn from many different internal and external sources. These sources included sophisticated Enterprise Resource Planning (ERP) systems, Customer Relationship Management (CRM) systems, Supply Chain Management (SCM) systems, data warehouses and other enterprise-wide systems that often contain huge amounts of data and permit relatively easy access to that data by a wide variety of users at many different levels of the organization.

In our 21^st century age of Big Data, such systems have expanded greatly are now usually referred to as Business Intelligence (BI). Big Data has also ushered in Business Analytics (BA) tools, often statistically based, to find patterns and relationships in large amounts of data that will lead to better decision making, often automated decision-making. Clearly BI is moving beyond simply asking “what happened” and “where did it happen,” and pushing, with newer BA tools, into “how did it happen” and “why did it happen.”

These terms have entered our lexicon and are now common topics of discussion and development in large, and even in medium-sized, enterprises, but their definitions are often difficult to separate and clarify. Our goal is not to add to that discussion. We will talk in more general terms about the issues of measurement in such systems in terms that a manager, not a data analysis, will be interested in considering.

Increasingly the issues in BI revolve around measurement and metrics – what to measure, how to measure it, and how to separate out meaningful data and convert it in information that can be used to support decision-making. The problem of measurement is severely compounded by the massive databases that are now available – much of it unintelligible, just plain nonsense, or simply noise.

Background

Decision-making as we know it today, supported by computers and vast networked information systems, is a relatively recent phenomenon. But the concept has been around long enough to permit the methods and theories of decision-making to blossom into what was described a decade ago as “a plethora of paradigms, research schools, and competing theories and methods actively argued by thousands of scientists and decision makers worldwide” (Robins 2003).

Early computer systems focused primarily on accounting and financial data. It is said that information systems are about transforming data into information, perhaps even knowledge. We could say that early systems transformed data into aggregated or summarized data – for example, wage rates, hours worked, benefits and tax data etc. transformed into departmental or corporate payroll reports.

Decision support systems “evolved from the theoretical studies of organizational decision making done at the Carnegie Institute of Technology during the late 1950s and early '60s. By the end of the 1970s, it was clear that model-based decision support had become a practical, useful tool for managers.

A 1970 article by John Little of MIT clarified the concept of decision support (Little, 1970). In a 1979 paper he provided a definition that is paraphrased here: (Little, 1979):