How Much Automation can be done in Testing?

Introduction

Part of software project development, it is very important to make sure, before product delivery, that the software is working as expected without serious flaws or problems. Testing the software, like any other product, can be seen from several perspectives. It can be seen based on what we are testing. For example, each deliverable in the software project (e.g. the requirements, design, code, documents, user interface, etc) should be tested. Moreover, in the code, which is the main deliverable, we may test the code based on the user and functional requirements or specification (i.e. black box testing). In this level, we are testing the code as a black box to make sure that all services expected from the program are: existed, working as expected and with no problem. On this perspective, we may need also to test the internal structure of the code (i.e. white box testing) based on the code structure or any other element. Testing can be also divided based on the sub stages or activities that occur in testing. For example, the testing stage includes the main activities: test case generation and design, test case execution and verification, building the testing database or oracle, coverage evaluation and assessment, results reporting and so on. Software testing tries to ensure that software products are not developed heuristically and optimistically. This software engineering stage ensures that the developed software is, ultimately and hopefully, error free. However, in reality, no process can guarantee that the developed software is 100% error free). In all cases, it is widely agreed that, this stage takes a large percent of the overall project resources.

Software test automation is the process of achieving one or more of software testing activities mentioned earlier programmatically or automatically with no user intervention (or with minimal user intervention). Software test automation is expected to cost more at the beginning while save more eventually. There are extra costs for test automation framework at start-up. However, once correctly implemented, it is widely accepted that test automation will decrease the cost as it reduces the required number of human testers and hence reduces the cost of testing. There are several repetitive tasks in testing that take a long time and that occur very often (e.g. regression testing, reporting, test cases’ execution, etc.). For such activities, test automation is the best choice.

As explained at the beginning of this chapter, the focus of this chapter is in test automation for Graphical User Interfaces (GUIs). GUIs are getting more and more focus and roles in new applications. Earlier, the GUI has less important role and many program events can be executed with very simple Console commands. Developing rich GUIs that can handle complex types of application-user interactions became a trend in all new applications. A typical programming Integrated Development Environment (IDE) such as .NET, Eclipse or NetBeans includes lots of types of GUI components. Every time a new release is developed of those IDEs, one of the expected additions is new or richer GUI components. Such richness in GUI components add extra overhead to software testing where testing such components extensively require several different complex scenarios.