Measurement Issues in BI

Measurement Issues in BI

William K. Holstein, Jakov Crnkovic
DOI: 10.4018/978-1-4666-5888-2.ch509
(Individual Chapters)
No Current Special Offers

Chapter Preview



Gut feel is great for everyday problems. But, it often leads us astray when we’re presented with complex streams of information. We can be blinded by the newest and nearest data point and miss the big picture.-Nate Silver, statistician, author and writer for The New York Times speaking at the November 2012 IBM Information on Demand Conference.

We are now firmly ensconced in the age of Big Data. Wal-Mart has passed the U.S. Government in the amount of data that they keep – the result of keeping the details of every one of more than 1 million transactions per hour. Add to that the millions of daily Facebook and LinkedIn postings, Twitter Tweets and GPS data maintained by cell phone companies and the ‘Big’ in Big Data may not express what we are experiencing in big enough terms. All of this data has made the question of what and how to measure (and how to use the results for decision making) even more important in contemporary information systems.

The last two decades of the 20th century witnessed tremendous progress in systems for information support – flexible and adaptable systems to support decision makers and to accommodate individual needs and preferences. These model- or data-driven or hybrid systems, called Decision Support Systems (DSS), incorporated diverse data drawn from many different internal and external sources. These sources included sophisticated Enterprise Resource Planning (ERP) systems, Customer Relationship Management (CRM) systems, Supply Chain Management (SCM) systems, data warehouses and other enterprise-wide systems that often contain huge amounts of data and permit relatively easy access to that data by a wide variety of users at many different levels of the organization.

In our 21st century age of Big Data, such systems have expanded greatly are now usually referred to as Business Intelligence (BI). Big Data has also ushered in Business Analytics (BA) tools, often statistically based, to find patterns and relationships in large amounts of data that will lead to better decision making, often automated decision-making. Clearly BI is moving beyond simply asking “what happened” and “where did it happen,” and pushing, with newer BA tools, into “how did it happen” and “why did it happen.”

These terms have entered our lexicon and are now common topics of discussion and development in large, and even in medium-sized, enterprises, but their definitions are often difficult to separate and clarify. Our goal is not to add to that discussion. We will talk in more general terms about the issues of measurement in such systems in terms that a manager, not a data analysis, will be interested in considering.

Increasingly the issues in BI revolve around measurement and metrics – what to measure, how to measure it, and how to separate out meaningful data and convert it in information that can be used to support decision-making. The problem of measurement is severely compounded by the massive databases that are now available – much of it unintelligible, just plain nonsense, or simply noise.



Decision-making as we know it today, supported by computers and vast networked information systems, is a relatively recent phenomenon. But the concept has been around long enough to permit the methods and theories of decision-making to blossom into what was described a decade ago as “a plethora of paradigms, research schools, and competing theories and methods actively argued by thousands of scientists and decision makers worldwide” (Robins 2003).

Early computer systems focused primarily on accounting and financial data. It is said that information systems are about transforming data into information, perhaps even knowledge. We could say that early systems transformed data into aggregated or summarized data – for example, wage rates, hours worked, benefits and tax data etc. transformed into departmental or corporate payroll reports.

Decision support systems “evolved from the theoretical studies of organizational decision making done at the Carnegie Institute of Technology during the late 1950s and early '60s. By the end of the 1970s, it was clear that model-based decision support had become a practical, useful tool for managers.

A 1970 article by John Little of MIT clarified the concept of decision support (Little, 1970). In a 1979 paper he provided a definition that is paraphrased here: (Little, 1979):

A coordinated collection of data, systems, tools, and techniques along with requisite software and hardware, by which an organization gathers and interprets relevant information from the business and environment and turns it into a basis for action.

Key Terms in this Chapter

Decision Support System (DSS): Decision support systems couple the intellectual resources of individuals with the capabilities of the computer to improve the quality of decisions. [They comprise] a computer-based system for management decision makers who deal with semi-structured problems (Gorry & Scott Morton, 1989).

Balanced Scorecard: An approach for measuring business and management results that goes well beyond financial metrics. Several ‘perspectives’ are suggested, e.g. financial, customer, internal processes and learning and innovation ( Kaplan & Norton, 1992 ).

Metric: A predetermined measure that will be used as the basis for a measurement process. For example, the percentage of customer calls answered within one minute.

Business Intelligence (BI): The process of gathering information in the field of business. Information is typically obtained about customer needs, customer decision making processes, the competition, conditions in the industry, and general economic, technological, and cultural trends. Business intelligence is carried out to gain sustainable competitive advantage, and is a valuable core competence in some instances. The term was first used by Gartner and popularized by Gartner analyst Howard Dresner. It describes the process of turning data into information and then into knowledge. The intelligence is claimed to be more useful to the user as it passes through each step.

Measurement: The process of determining values representing performance, or the results of the process. For example, the measurement process is now started , or the measurement of the order to delivery cycle indicated 35 days .

Benchmark: A standard, usually from outside sources and usually representing the best, or better than average, performance against which an activity’s metric is compared. For example, world-class competitors have 35 defects per unit within the first six months, we have 85.

Dashboard | Cockpit: A graphic display on a manager’s computer screen summarizing measures and metrics that are relevant to the business unit or responsibility area of the manager.

Complete Chapter List

Search this Book: