Receive a 20% Discount on All Purchases Directly Through IGI Global's Online Bookstore

Renée C. Bryce (Utah State University, USA), Yu Lei (University of Texas, Arlington, USA), D. Richard Kuhn (National Institute of Standards and Technology, USA) and Raghu Kacker (National Institute of Standards and Technology, USA)

Source Title: Handbook of Research on Software Engineering and Productivity Technologies: Implications of Globalization

Copyright: © 2010
|Pages: 13
DOI: 10.4018/978-1-60566-731-7.ch014

Chapter Preview

TopSoftware systems are complex and can incur exponential numbers of possible tests. Testing is expensive and trade-offs often exist to optimize the use of resources. Several systematic approaches to software testing have been proposed in the literature. Category partitioning is the base of all systematic approaches as finite values of parameters are identified for testing. Each of these finite parameter-values may be tested at least once, in specified combinations together, or in exhaustive combination. The simplest approach tests all values at least once. The most thorough approach exhaustively tests all parameter-value combinations. While testing only individual values may not be enough, exhaustive testing of all possible combinations is not always feasible. Combination strategies are a reasonable alternative that falls in between these two extremes.

Consider an on-line store that has four parameters of interest as shown in Table 1. There are three log-in types; three types of member status; three discount options; and three shipping options. Different end users may have different preferences and will likely use different combinations of these parameters. To exhaustively test all combinations of the four parameters that have 3 options each from Table 1 would require 3^{4} = 81 tests.

Four parameters that have three possible settings each for an on-line store

Log-in Type | Member Status | Discount | Shipping |

New member - not logged in | Guest | None | Standard (5-7 day) |

New-member - logged in | Member | 10% employee discount | Expedited (3-5 day) |

Member - logged in | Employee | $5 off holiday discount | Overnight |

In this example, exhaustive testing requires 81 test cases, but pair-wise combinatorial testing uses only 9 test cases. Instead of testing every combination, all individual pairs of interactions are tested. The resulting test suite is shown in Table 2, and is contains only 9 tests. All pairs of combinations have been combined together at least once during the testing process. For instance, the first test from Table 2 covers the following pairs: (New member - not logged in, Guest), (New member - not logged in, $5 off holiday discount), (New member - not logged in, Standard (5-7 day)), (Guest, None), (Guest, Standard (5-7 day)), and (None, Standard (5-7 day)). The entire test suite covers every possible pairwise combination between components. This reduction in tests amplifies on larger systems - a system with 20 factors and 5 levels each would require 5^{20} = 95,367,431,640,625 exhaustive tests! Pairwise combinatorial testing for 5^{20} can be achieved in as few as 45 tests.

Covering Array Number (CAN): The size of a covering array, or a mixed-level covering array and is considered optimal when it is as small as possible.

Covering Array: CA?(N;t,k,v), is an N x k array. In every N x t subarray, each t-tuple occurs at least ? times. In combinatorial testing, t is the strength of the coverage of interactions, k is the number of components (degree), and v is the number of symbols for each component.

Pairwise Combinatorial Testing: An interaction test in which the strength, t, is equal to two. Pair-wise permutations of factors are executed during testing.

Pseudo-Exhaustive Testing: The term used when interaction testing is considered effectively exhaustive. Interaction testing at levels of 4-way to 6-way coverage have been suggested to be pseudoexhaustive (Kuhn, Reilly 2002)

n-way Combinatorial Testing: An interaction test in which the strength, t, is equal to a specified value n. Permutations of n-tuples of factors are executed during testing

Search this Book:

Reset

Copyright © 1988-2018, IGI Global - All Rights Reserved