T-Way Testing Strategies: Issues, Challenges, and Practices

T-Way Testing Strategies: Issues, Challenges, and Practices

Kamal Z. Zamli (Universiti Malaysia Pahang, Malaysia), AbdulRahman A. Alsewari (Universiti Malaysia Pahang, Malaysia) and Mohammed I. Younis (University of Baghdad, Iraq)
DOI: 10.4018/978-1-4666-6026-7.ch018
OnDemand PDF Download:


In line with the advancement of hardware technology and increasing consumer demands for new functionalities and innovations, software applications grew tremendously in term of size over the last decade. This sudden increase in size has a profound impact as far as testing is concerned. Here, more and more unwanted interactions among software systems components, hardware, and operating system are to be expected, rendering increased possibility of faults. To address this issue, many useful interaction-based testing techniques (termed t-way strategies) have been developed in the literature. As an effort to promote awareness and encourage its usage, this chapter surveys the current state-of-the-art and reviews the state-of-practices in the field. In particular, unlike earlier work, this chapter also highlights the different possible adoptions of t-way strategies including uniform interaction, variable strength interaction, and input-output-based relation, that is, to help test engineers make informed decision on the actual use of t-way strategies.
Chapter Preview

2. T-Way Testing Fundamentals

In general, t-way testing can be abstracted to a covering array. Throughout out this chapter, the symbols: p, v, and t are used to refer to number of parameters (or factor), values (or levels) and interaction strength for the covering array respectively. Earlier works suggested two definitions for describing the covering array (Cohen, 2004). The first definition is based on whether or not the numbers of values for each parameter are equal. If the number of values is equal (i.e. uniformly distributed), then the test suite is called Coverage Array (CA). Now, if the number of values in non-uniform, then the test suite is called Mixed Coverage Array (MCA). Unlike earlier work in the literature, this chapter unifies the CA and MCA notations in order to include that of variable strength covering array (VCA), and input output based covering array (IOR).

Normally, the CA takes parameters of N, t, p, and v respectively (i.e. CA(N,t, v p)). For example, CA (9, 2, 2 4) represents a test suite consisting of 9x4 arrays (i.e. the rows represent the size of test cases (N), and the column represents the parameter (p)). Here, the test suite also covers pairwise interaction for a system with 4 2 valued parameter (see Figure 1). When the CA is the most optimal result, the covering array can be rewritten as CAN (N,t, v p).

Figure 1.

CA (9,2,24)

Alternatively, MCA takes parameters of N, t, and Configuration (C). In this case, C captures the parameters and values of each configuration in the following format: v1p1 v2p2, ….. vnpn indicating that there are p1 parameters with v1 values, p2 parameters with v2 values, and so on. For example, MCA (8,3,2311) indicates the test size of 8 which covers 3-way interaction. Here, the configuration takes 4 parameters: 3 2 valued parameter and 1 1 valued parameter (see Figure 2).

Figure 2.

MCA (8,3,2311)

In the case of VCA, the parameter consists of N, t, C, and Set (S) (i.e. VCA (N,t,C,S)). Similar to MCA, N,t, and C carry the same meaning. Set S consists of a multi-set of disjoint covering array with strength larger t. For instance (see Figure 3), VCA (12, 2, 3222, (Zamli, et al., 2011)) indicates the test size of 12 for pairwise interaction (with 2 3 valued parameter and 2 2 valued parameter) and 3-way interaction (with 1 3 valued parameter and 2 2 valued parameter).

Key Terms in this Chapter

Cumulative/Variable Strength: More than one interaction strength is considered to different parameter interactions.

IO Based Relations: The interaction considered is based on the input output behavior.

Covering/Mixed Covering Array: Formal mathematical description for expressing the covering of parameters for testing consideration.

Strength: The interaction strength between the involved parameters (often denoted as t).

Combinations of Uniform, Variable Strength and IO Based Relations: Hybrid consideration of interaction strength.

Uniform Strength: The same strength are adopted throughout all the parameter interactions.

Complete Chapter List

Search this Book: