Mismanagement or Mismeasurement: The Application of DEA to Generate Performance Values and Insights from Big Data

Introduction

Data Envelopment Analysis (DEA) is an increasingly popular management tool to assess the relative productive efficiency (performance) of a set of decision making units (DMUs). It generates insights from big data to speak for themselves about the relationship between inputs and outputs with minimal assumptions unlike traditional approaches (Charnes et al., 1985). The new DEA modeling and analytical capabilities have been found extremely attractive by researchers and professionals leading to a large number of real life applications with positive impacts and values. Although the DEA method can be viewed as a production frontier approach, DEA is widely used for conducting performance evaluation and benchmarking best-practices to discover insights from peers (Cook, Tone and Zhu, 2014). Measuring the performance has never been a simple task; any change in the dataset or computed DEA model would lead to a significant difference in the generated results (Emrouznejad and De Witte, 2010). The availability of DEA software made the generation of DEA results a simple and fast process, with much focus on the DEA modeling process that is an essential step to design before the generation of the analysis. Due to DEA being a young field and attractive for applications; one can easily notice a faster growth in applications than theoretical research since its inception in the early 1980s. Unfortunately, in the literature, one can easily find a number of misguided applications with misguided insights that motivated us to develop this general framework to guide the DEA community; researchers, practitioners, and reviewers. Few published academic frameworks exist, and there are a few emerging guidelines to clarify rules and set academic standards for performance measurement, selection of DEA models, determination of the number of DMUs and their relationships to the number of inputs and outputs and their definitions (Cook, et al., 2014; Osman et al., 2011; Emrouznejad and De Witte, 2010; Brown, 2006; Dyson et al., 2001).

Cook et al (2014) provided some clarifications and directions on the use of DEA. They addressed several issues such as model orientation, input and output selection, the use of mixed and raw data, and the number of DMUs in relation to the total number of input and output variables. Osman et al. (2011) validated the previous practical recommendations on the number of DMUs in relation to the total number of input and output variables to find that the numbers of DMUs should be much greater than 3 to 5 times the total sum of the number of inputs and outputs. This number of DMUs was shown to depend on the types of orientation of the DEA models, and the desired spread of the bell-normality shape of the final-value scores from the perspective of the evaluator. They called for more research on such rule of thumb assumptions related to the number of DMUs and homogeneity of data and working environment.