Article Preview
TopIntroduction
As software applications permeate everywhere in the world, people are becoming more sensitive to the validity and reliability of software applications (Juristo et al., 2006). Defects in software applications may result in tremendous financial loss and even innocent death (Felderer & Ramler, 2014). According to a report by Tricentis, a leading company providing software testing solutions, about 606 major software failures from 314 companies occurred throughout the world in 2017. These failures caused $1.7 trillion in financial losses, resulted in 268 years of cumulative unplanned downtime, and affected 3.6 billion people (Tricentis, n.d.). Billions of dollars are invested in software development every year around the world, and approximately 50 percent of the total elapsed time and more than 50 percent of the total costs are expended in testing the program or system being developed in a typical software development project (Boehm & Papaccio, 1988; Hailpern & Santhanam, 2002; Harrold, 2000; Myers et al., 2011). Despite the fact that software testing has been considered as an important phase in the software development life cycle to assure software quality, defects still cannot be entirely eradicated due to inadequate testing (Tricentis, n.d.; Whittaker, 2000).
Inadequate software testing usually results in defective applications and negative outcomes. Inspired by Whittaker’s (2000) study, we identified five primary software testing practical issues (untested code, untested combinations of input values, untested paths, untested operating environments, and defective testing procedures) that cause inadequate software testing. These five testing issues are rooted in resource constraints and technical constraints in software testing. On the one hand, a company might not have adequate resources such as budget, time, or personnel to run sufficient tests. On the other hand, a company might have enough resources but not have key technical support such as sophisticated algorithms, powerful testing tools, or expert testing engineers, resulting in defective testing procedures. Considering that most practical software testing issues result from resource constraints, we therefore focus on exploring the comparative value of software test cases in this study. The issue is that in a resource constrained testing environment, it is imperative for companies to choose the most valuable test cases to make their testing efforts as effective and efficient as possible. We believe that determining the most valuable test cases will help alleviate the resource constraints issues and thus dramatically improve the testing process.
Given that exhaustive software testing is impossible (Myers et al., 2011), maximizing the efficiency and effectiveness of software testing with limited resources becomes an important question in the software testing domain (Juristo et al., 2006). Although researchers have attempted to solve the problem of resource constraints in software testing from different perspectives in the last few decades (e.g., Biffl et al., 2006; Boehm, 2006; Felderer & Ramler, 2014; Wohlin & Aurum, 2006), the shortcomings of the prior studies indicate that the existing approaches cannot appropriately address the problem. This is because the software testing methods either lag behind software development methods or just take into account software engineering factors which cannot provide adequate guidance for improving software testing (Juristo et al., 2006; Talby et al., 2006).