Conformance Testing of Preemptive Real-Time Systems

Conformance Testing of Preemptive Real-Time Systems

Noureddine Adjir (Faculty of Exact and Applied Sciences, University of Oran, Oran, Algeria), Pierre de Saqui-Sannes (ISAE, University of Toulouse, Toulouse, France) and Kamel Mustapha Rahmouni (Computer Science Department, University of Oran, Oran, Algeria)
DOI: 10.4018/ijertcs.2013100101

Abstract

The paper presents an approach for model-based black-box conformance testing of preemptive real-time systems using Labeled Prioritized Time Petri Nets with Stopwatches (LPrSwTPN). These models not only specify system/environment interactions and time constraints. They further enable modelling of suspend/resume operations in real-time systems. The test specification used to generate test primitives, to check the correctness of system responses and to draw test verdicts is an LPrSwTPN made up of two concurrent sub-nets that respectively specify the system under test and its environment. The algorithms used in the TINA model analyzer have been extended to support concurrent composed subnets. Relativized stopwatch timed input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Assuming the modelled systems are non deterministic and partially observable, the paper proposes a test generation and execution algorithm which is based on symbolic techniques and implements an online testing policy and outputs test results for the (part of the) selected environment.
Article Preview

1. Introduction

The embedded real-time industry is changing fast – systems have become larger, more complex, and preemptive. For real-time systems, the timely reaction is just as important as the kind of reaction. Thus the system must not only produce correct result, but must do so at the correct time; neither too early nor too late. Fly-by-wire systems in modern airplanes are an example for such embedded systems. If a pilot hits the brakes, the breaking system should engage almost immediately to ensure secure travelling. Furthermore, real-time systems may be interruptible. They may be interrupted at any time while keeping the capacity to restart later on without losing their state information (think, e.g., of interrupting a washing machine in order to remove a pencil from a shirt, and closing the machine immediately after). Such systems need to be tested in order to check their reliability before use. In testing real-time systems, the tester must consider when to stimulate the system, when to interrupt or resume operations, when to expect responses to be issued and how to assign verdicts to any timed sequence it may observe and partly control. Further, the test cases must be executed in real-time.

Without automation and modeling tools, testing remains ad hoc, time-consuming and error prone. With the use of models in software/hardware design and development, timed model-based testing has received increasing attention from industry practitioners. Therefore, Timed Model-Based Testing uses timed models describing the desired behaviour of a system to automate the testing process. Using models to generate test cases and assign verdicts is cheaper and more effective than a completely manual approach. Conformance testing is a way of black-box testing in which common testing tasks such as test case generation and test result evaluation are based on a model of the system. Thus, no knowledge about the internal workings of the program to be tested is used and the tests are limited to the functional and timing properties. A survey of the literature indicates that those papers which address timed test sequence generation have extensively discussed reactivity and timeliness. So, much work on model based testing have considered as formal modelling techniques timed automata (TA) (Alur, 1994) or time Petri nets (Merlin, 1976). However, all this models cannot enable to model the suspension and resumption of a task or any kind of executable portion of code in real-time systems. Therefore a real-time specification model should include a suspend/resume capability. The paper addresses model-based black-box conformance testing of preemptive real-time systems. It checks a System Under Test (SUT) against its specification. This is typically achieved in a controlled environment where the SUT is executed and stimulated with inputs and delays according to a test specification, and the responses of the SUT are checked to conform to its specification. Precisely, the paper presents a technique for conformance testing of preemptive real-time systems based on Labelled Prioritized Time Petri Nets with Stopwatches models (LPrSwTPN). The test specification is given as an LPrSwTPN made up of two composed subnets that respectively model the expected behaviour of the SUT and the latter’s environment. The proposal implements an online testing approach and proposes a relativized conformance relation named rswtioco (Relativized Stopwatch Timed Input Output Conformance), between model and SUT which coincides with timed trace inclusion taking assumptions about the environment behavior explicitly into account. It is an extension of the rtioco relation (Hessel, 2008). In addition to allowing explicit and independent modelling of the environment, it also has some nice theoretical properties that allow testing effort to be reused when the environment or system requirements change. Unlike other approaches based on offline testing, we do accept unrestricted non-deterministic and partially observable specifications.

The rest of paper is organized as follows. Section 2 surveys related work. Section 3 shows what is new in the test of preemptive real-time systems. In section 4, we illustrate and compare the two different approaches to timed testing: offline and online testing. Section 5 describes the test specification. Section 6 introduces LPrSwTPN, their formal semantics in terms of timed labeled transition systems, and their use to model and specify the behavior of real-time systems. The rswtioco relation is presented in section 7. In section 8, we present an online testing algorithm of real-time systems from LPrSwTPN specifications allowing full non-determinism. This algorithm combines test cases generation and their execution. Finally, section 9 concludes the paper.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 10: 4 Issues (2019): Forthcoming, Available for Pre-Order
Volume 9: 2 Issues (2018)
Volume 8: 2 Issues (2017)
Volume 7: 2 Issues (2016)
Volume 6: 2 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing