Article Preview
TopIntroduction
Service Oriented Architecture (SOA) has been evolving as a mainstream software development paradigm where Web services are basic elements (Papazoglou, Traverso, Dustdar, & Leymann, 2008). A Web service often implements an application or part of an application, and is able to make a set of operations available to its consumers through the Web Service Description Language (WSDL) (Haas & Brown, 2004). In the context of SOA, Web services can be implemented and owned by one organization, and published as an independent resource that is consumed by other organizations. To implement complex applications, Web services have to be loosely orchestrated to fulfill a business goal (Peltz, 2003; Sun, Khoury, & Aiello, 2011).
Let us consider an e-bookstore system constructed by several Web services. Among them, a Web service is responsible for the electronic payment. Usually, such a Web service is developed and owned by a third-party organization, such as a software company, a bank, or an independent commercial office. Due to the fact that the consumer (i.e., the e-bookstore system) can access the Web service only through its description (namely WSDL) and cannot look into the source code of the Web service, this results in some inconsistency issues. For example, some faults may exist in the implementation regardless how many efforts are spent on testing. Also, the service owner may update the implementation due to changes of the payment policy (specification); however, the service consumer may not realize the changes happening to the implementation or specification. This may result in the situation where the consumer invokes the latest version of implementation while holds the old version of specification. All these cases bring us one question, namely “how should we assure the consistency between implementation and specification of Web services?”
Software testing provides a practical and feasible approach to the question. The basic idea of testing is to select some test cases for executing program under test, and then evaluate the output against the expected correct output (namely oracle). If the actual output equals to the oracle, such a test succeeds; otherwise the test detects a fault. In the context of SOA, new unique features pose challenges for testing Web services. For instance, the specification and implementation of Web services are completely separated, and often service consumers cannot access the source codes of Web services they invoke. The lack of source code and the restricted control of Web services not only limit the testability of Web services, but also make white-box testing techniques inapplicable. One may argue that the lack of source codes is also present in the use of any normal software library, but they are totally different in that the implementation of a Web service can be written in any programming language and its specification is written in the XML style. Moreover, the test oracle problem, which means in some situations it is impossible or practically too difficult to decide whether the program outputs on test cases are correct (Weyuker, 1982), is even amplified. In the example of the electronic payment service, the consumer, in some situations, may not know exactly how much should be charged for a given input (i.e., book price). The problem becomes even worse when the payment involves charges for transfer between accounts or currency exchange. Testing Web service under SOA calls for new techniques (Bartolini, Bertolino, Elbaum, & Marchetti, 2009; Canfora & Penta, 2009; Farooq, Georgieva, & Dumke, 2008; Sun, 2011).