Teaching Property-Based Testing: Why and How

Introduction

Compared to other engineering, software engineering is a relatively new area. It has matured over the years as new trends and concerns have emerged. Testing, which is considered an essential activity to ensure software quality, is a reasonably established software development practice. However, not all testing methods are taught and, hence, known and used. For instance, model-based mutation and testing (MBT) approaches appear to be more popular in academia than in industry, where it is less widespread (Garousi & Felderer, 2017).

Model-based testing is a technique where test cases are generated from the requirements (Dalal et al., 1999). Property-based testing (PBT) is a type of MBT in which properties play a dominant role, and they are related to requirements that the system is expected to satisfy. Moreover, there are some other characteristics:

• •

  Random data is used to evaluate the previously defined properties.

• •

  The first property that encounters an error is returned by the PBT tool, or a simplified one (identified by shrinking).

• •

  Generators are used to produce input data. However, their writing may not be straightforward.

PBT acts as a complement to the software engineering lifecycle, preventing the introduction of defects in the code (Fink & Bishop, 1997). It is related to a tool developed to formulate and test properties expressed in Haskel programs, launched in 1999, which brought visibility to the approach: QuickCheck (Claessen & Hughes, 2000).

Agile methods are designed to ensure that software components are created quickly and respond to changing requirements without neglecting their quality. Testing is a noticeable quality assurance approach to decrease software defects. However, as software systems are becoming increasingly complex and interconnected, their proper testing is becoming more difficult and challenging, leading to an increase in the number and complexity of software defects.

One way to mitigate the undesired effects is to include PBT in the development life cycle (Derrick et al., 2009). Its adoption in the software development industry can reduce the cost of defects in software. Defects have associated costs, namely:

• •

  They have a visible cost related to the time taken to prevent and correct them.

• •

  There is also an invisible cost related to customer abandonment.

Thus, the time invested in testing properties and writing generators can be translated into a higher level of confidence of the source code reliability (Labuschagne, Inozemtseva, & Holmes, 2017) and can help detecting bugs, as happened for Ericsson (Arts, Hughes, Johansson, & Wiger, 2006), Klarna (Hughes, 2016), and Volvo Cars (Arts, Hughes, Norell, & Svensson, 2015).

Software engineering research has impacted industrial practice, such as the REST architectural style (Fielding, 2000) and its high usage. However, despite some notorious examples, the collaboration between industry and academy has not always been so fruitful (Basili et al., 2018). This gap seems to be even more pronounced in software testing (Beszédes & Vidács, 2016; Garousi, & Felderer, 2017).

PBT has not yet have been largely adopted by software developers. Developers will be inclined to use what they have learned about in subjects that have been introduced in their formal education. PBT has been integrated into bachelor’s and master’s courses and used to test functional code (Page, 2011; Waldmann, 2017). Lars Brünjes, Director of Education at IOHK, emphasized the importance of property-based testing for the company, especially for complex systems, to ensure their quality, but also the importance of learning the approach (Brünjes, 2018).

This chapter has a two-fold objective, namely: