Article Preview
TopBackground
In “Models of Man” (1957), eventual Nobel laureate Herbert Simon established something of a manifesto for the improvement of rational decision-making via the use of information technology and data stored in computer files. The problem as Simon saw it was not that humans were inherently irrational, but their rationality was bounded by their information processing limitations for which information technology offers some solution. Simon believed that the novel (even if by modern standards rudimentary at best) technologies of that era as tools that would enhance and perhaps even replace human decision-making with systems that would enable all relevant decision variables to be considered prior to decisions being made. In a later essay (Simon, 1988), Simon lamented the focus of early versions of information systems on the generation of reports that summarized the huge amounts of internal information within companies, including production, sales, and accounting data, but with little consideration for how the reports would be used to support actual decision making. He argued that the ultimate value of such systems would be based on the ability to collect, filter, and analyze both internal and external data, including natural-language and numerical information, in order to provide managers with sufficient information to make better decisions without inundating them with unnecessary information.
Many of Simon’s early ideas went largely unrealized for decades. However, as technologies for first analytical processing and then data storage improved, tentative first steps were made toward this end in the form of management reporting systems, and on limited data, Decision Support Systems. This emphasis was in place in a wide range of MIS, OR and Decision Sciences curricula for several years.
With the advent of significantly faster and cheaper data storage, and building on the efforts of E.F. Codd (1970), database management systems came to the forefront in the 1970’s. These types of systems made reporting, limited analysis and ready access to data throughout organizations easier, leading to a proliferation of summarized data for management reporting purposes. Advances in data storage eventually led to virtually unlimited storage capacity, at which point databases began being combined into vast data warehouses. These types of systems were devised specifically for the purpose of providing a mechanism by which transactional data could be ported into decision support environments. Hence, in addition to processing transactional data, these systems provided the ability to capture and process historical data (albeit with the development of purpose-built data models and routines for data cleansing) for the purposes of decision-making. Sperry-Univac introduced one of the earlier systems to embody this concept (albeit in a very rudimentary fashion) with their MAPPER (Maintain, Prepare, and Produce Executive Reports) system in the 1970s (cf. Gray &Smith, 2001); another pioneer in this area was TeradataTM, which introduced a database reporting system specifically for decision support in the early 1980s (cf. Pereira, 2010).
A key change that also drove the data warehousing movement was the distinction between dimensional and relational modeling of data. As opposed to relational modeling, which grouped data in tables by general subject (e.g., customer, supplier, product, order), dimensional models grouped data as facts (e.g. something was sold for X dollars) and dimensions (the date it was sold, or the quarter it was sold, or its association with a specific event such as Black Friday). The dimensional view had distinct advantages for data warehousing, because the dimensions tended to be concepts that had clear meaning to the business user, and because dimensions were stored in association with facts constructs such as “all sales of yellow polka-dot bikinis on Black Friday over the past three years” could be readily accessed. However, the dimensional model also has significant complexity and storage requirements as dimensions tend to proliferate (cf. Turban, Sharda, Delen & King, 2011). The relational model has the distinct advantage of being more flexible for storage, access and modification, which is largely why it is still fairly dominant for transaction processing and ad-hoc querying. The nature of dimensional versus relational modeling required a work-around to extract, translate and load transactional data into a data warehouse.