Use of Qualitative Research to Generate a Function for Finding the Unit Cost of Software Test Cases

Use of Qualitative Research to Generate a Function for Finding the Unit Cost of Software Test Cases

Mark L. Gillenson, Thomas F. Stafford, Xihui “Paul” Zhang, Yao Shi
Copyright: © 2020 |Pages: 22
DOI: 10.4018/JDM.2020040103
(Individual Articles)
No Current Special Offers


In this article, we demonstrate a novel use of case research to generate an empirical function through qualitative generalization. This innovative technique applies interpretive case analysis to the problem of defining and generalizing an empirical cost function for test cases through qualitative interaction with an industry cohort of subject matter experts involved in software testing at leading technology companies. While the technique is fully generalizable, this article demonstrates this technique with an example taken from the important field of software testing. The huge amount of software development conducted in today's world makes taking its cost into account imperative. While software testing is a critical aspect of the software development process, little attention has been paid to the cost of testing code, and specifically to the cost of test cases, in comparison to the cost of developing code. Our research fills the gap by providing a function for estimating the cost of test cases.
Article Preview


This is a demonstration of case research used to generate an empirical function, and it is an unusual contribution to the literature in that respect. Mathematical expressions of functionality are usually developed statistically, supported with survey or simulation data (Akhavein et al., 1997; Kurfman et al., 2003). It is rare for a case study to be quantitative, yet mathematical functions can (conceivably) be derived in this way since deductions involving mathematical propositions are a subset of formal logic and nothing in formal logic is tied to a specific methodology (Lee, 1989). The evidence that is considered in case studies can be qualitative, quantitative, or both (Eisenhardt, 1989), but the primary purpose of a case study is to generate theory (Yin, 2013) and a theoretical representation of cause and effect (as is typically found in a statement of mathematical logic, such as a response function) is not out of the question in case research, just rare (Lee, 1989). It is, practically speaking, more likely that response equations would lead to case studies (Hengl et al., 2007), since many statistical techniques benefit greatly from a qualitative interpolation process. In contrast to the conventional wisdom on case study uses and outcomes, however, we utilize an in-depth case study of industry experts to directly induce an empirical function through qualitative generalization (e.g., Yin, 2013).

Our methodology demonstrates the derivation of the variables and interrelated functionality of a candidate empirical expression. This methodology can be used in any situation in which there is a need to develop an empirical expression from case-based, qualitative research. While we will describe our innovative methodology in general terms, we believe that a detailed exposition of its actual use is essential to both understanding the process and to demonstrating its worth.

The actual example we have chosen is that of developing a function to estimate the cost of developing and executing a test case in the software testing process. The outcome of this process results in the description of the cost of generating and utilizing a functional test case for software testing purposes; this outcome of our case research is much in the way that Yin (2013) and Eisenhardt (1989) each suggests extracting theoretical meaning from case research. In that sense, the contribution of this study lies in generalizing from qualitative inquiry to a theoretical construction, as Yin and Eisenhardt would put it. For clarity, this will necessitate our providing some explanation of the elements of generating and executing software test cases, and of the software testing process, itself.

Case studies, concentrated as they are on a focused and bounded phenomenon embedded in a context, are particularly useful for identifying context-specific meaning (Eisenhardt, 1989; Miles et al., 2013). We consider that case study methods are typically intended to lead to theoretical developments as their own unique modality of generality (Yin, 2013), rather than the broad extension of generality to other contextual areas as is so typically the case with quantitative research. In that manner, we consider this in-depth investigation of software testing cost factors to be a revelatory approach (Sarker et al., 2012; Yin, 2013) meant for specific and detailed understanding of a unique context.

In addition, one of the most important potential theoretical contributions that could be made in the software development literature would be a functional model useful for determining the costs components of software testing, and this is an additional contribution of this study: the derivation of an empirical cost function as a generalization from an interpretive case study of software testing engineers brought together in a cohort as part of a certification program on software testing.

Complete Article List

Search this Journal:
Volume 35: 1 Issue (2024)
Volume 34: 3 Issues (2023)
Volume 33: 5 Issues (2022): 4 Released, 1 Forthcoming
Volume 32: 4 Issues (2021)
Volume 31: 4 Issues (2020)
Volume 30: 4 Issues (2019)
Volume 29: 4 Issues (2018)
Volume 28: 4 Issues (2017)
Volume 27: 4 Issues (2016)
Volume 26: 4 Issues (2015)
Volume 25: 4 Issues (2014)
Volume 24: 4 Issues (2013)
Volume 23: 4 Issues (2012)
Volume 22: 4 Issues (2011)
Volume 21: 4 Issues (2010)
Volume 20: 4 Issues (2009)
Volume 19: 4 Issues (2008)
Volume 18: 4 Issues (2007)
Volume 17: 4 Issues (2006)
Volume 16: 4 Issues (2005)
Volume 15: 4 Issues (2004)
Volume 14: 4 Issues (2003)
Volume 13: 4 Issues (2002)
Volume 12: 4 Issues (2001)
Volume 11: 4 Issues (2000)
Volume 10: 4 Issues (1999)
Volume 9: 4 Issues (1998)
Volume 8: 4 Issues (1997)
Volume 7: 4 Issues (1996)
Volume 6: 4 Issues (1995)
Volume 5: 4 Issues (1994)
Volume 4: 4 Issues (1993)
Volume 3: 4 Issues (1992)
Volume 2: 4 Issues (1991)
Volume 1: 2 Issues (1990)
View Complete Journal Contents Listing