The use of information technology and decision support concepts at the operational business level were slow to take hold in the 20th century. In 2010, the authors documented the evolution and current state of the field of business intelligence and analytics (BIA). In the last decade, however, through the resurgence and mainstream use of artificial intelligence, machine learning algorithms, the development of inexpensive cloud-based mass storage, and the internet-of-things, business intelligence has evolved into data science. In this chapter, the authors trace this evolution across the diverse areas of data science and identify extremely useful advancements and best practices in the field.
TopIntroduction
Prior to 2010, significant growth occurred in the field of business intelligence (BI), a discipline more prevalent in practice than in theory. While there were a number of professional and consulting organizations, few universities had created or adapted their research at that time. BI was thought by some, to be the latest incarnation of management information systems (MIS), but a fundamental change has occurred in the organizational approach to the relevant disciplines. After witnessing the growth of BI, it is important to visit the changes that have taken place in the last decade. With the advent of cloud storage, the internet-of-things, and with the assistance of artificial intelligence and machine learning algorithms, the world has quickly progressed from business intelligence to data science.
Program offerings now span many schools and departments in universities, including business, computer science, statistics, and mathematics. Ten years ago, analytics programs were lumped into rankings with MIS programs. Currently, business analytics and data science programs abound in undergraduate and graduate settings, while online graduate programs have also proliferated, each with their own rankings.
By 2021, 98 colleges and universities were offering undergraduate programs and 627 colleges and universities were offering graduate programs and certificates in the many subfields of data science1. Programs focus in areas such as data analytics, data mining, cyber security, and marketing analytics to name a few. Online offerings have abounded.
For purposes of this article, “Data” is considered to encompass all of the traditional functional activities in business such as marketing, manufacturing, accounting, finance, distribution and the support operations provided by the transaction processing systems and other technology. “Science” includes artificial intelligence, machine learning, applied mathematics, data mining and statistical tools developed to solve business problems over this same period.
Assumptions
This article examines the ultimate confluence between efficiency-effectiveness and operational-tactical-strategic. Both dimensions are part of the decision-making process, that has been examined in many arenas from the psychological to the managerial. Some decisions support small issues while others have enormous impact. Theoretical approaches have always advocated for optimize some form of outcome (broadly defined as minimization of cost or maximization of profit) through algorithms or formulae. The practical approaches have usually tried to find “satisfactory” answers through simulation and heuristics. In the end, there are cost-effectiveness tradeoffs that contribute to the selection of the appropriate (or even feasible) decision-making approach. These tradeoffs will be examined on these two dimensions.
Efficiency is the measurement of Output as a function of Input. The formula is simple: Efficiency = Output/Input (e.g., miles/gallon). Operations are fine tuned to ensure the greatest impact of resources in accomplishing a goal. The time-motion expert charts the operation with a stopwatch and clipboard hoping to save precious seconds in highly repetitive operations, seconds which add up to dollars with the operation being repeated thousands or millions of times. On the other hand, effectiveness is not so easy to define. Traditional definitions like “successful in producing a desired or intended result” are less than definitive. What is the desired or intended result? How can it be determined what should be done? Perhaps the simplified “Efficiency is doing things right and Effectiveness is doing the right things” provides guidance. Business practitioners have typically used information and decision technology primarily for efficiency while the academics in the intelligence community were trying to determine what was effective.
Decisions are made at all levels within an organization. At the tactical level, decisions tend to be current in time and use highly detailed data. A clerk at a bank must decide whether to give an automobile loan to a customer based upon the details of an application. Decisions at this level tend to be highly structured, sometimes quite complex, but almost always, well defined by policy and algorithm. At the tactical level, the time horizon grows to weeks, months and maybe a year. Decisions do not require the level of detail but usually involve aggregated data. For example, a branch manager might use summary transaction activity data to make a staff planning decision to assure good service to customers. At the strategic level decisions are made for the long run. There is very little detail or even accuracy in the numbers that may be included in these important and usually very unstructured decisions.