Performance is critical to the success of every software system. As a sub-area of software engineering, Software Performance Engineering (SPE) is a systematic and quantitative discipline to construct software systems that meet performance objectives. A family of SPE approaches that has become popular in the last decade is SPE based on models developed using the Unified Modeling Language (UML), referred to as UML-Driven Software Performance Engineering (UML-SPE). This particular research area has emerged and grown since late 1990s when the UML was proposed. More than 100 papers have been published so far in this area. As this research area matures and the number of related papers increases, it is important to systematically summarize and categorize the current state-of-the-art and to provide an overview of the trends in this specialized field. The authors systematically map the body of knowledge related to UML-SPE through a Systematic Mapping (SM) study. As part of this study, they pose two sets of research questions, define selection and exclusion criteria, and systematically develop and refine a systematic map (classification schema). In addition, the authors conduct bibliometric, demographic, and trend analysis of the included papers. The study pool includes a set of 90 papers (from 114 identified papers) published in the area of UML-SPE between 1998 and 2011. The authors derive the trends in terms of types of papers, types of SPE activities, and types of evaluations. They also report the demographics and bibliometrics trends in this domain and discuss the emerging trends in UML-SPE and the implications for researchers and practitioners in this area.
Top1. Introduction
Performance is critical to the success of every software system. Many software products fail to meet their performance objectives after they are released. Fixing these problems late in the software development life-cycle is costly and causes schedule delays, cost overruns, damaged customer relations, missed market windows, lost revenues, and a host of other difficulties (Liu, 2011).
To avoid such costs, a system’s performance characteristics must be considered throughout the whole software development life-cycle. Software Performance Engineering (SPE) is a systematic and quantitative discipline which provides a collection of methods to assure appropriate performance-related product quality throughout the entire development process (Liu, 2011).
The Unified Modeling Language (UML) is a standardized general-purpose modeling language in software engineering. Since UML’s inception in late 1990’s (Object Management Group [OMG]), many researchers have benefitted from powerful structural and behavioral modeling features of UML to model, test, analyze and predict software performance, and as a result, a joint area between UML and SPE has emerged, which is referred to as UML-driven Software Performance Engineering (UML-SPE) in this study.
More than 100 papers have appeared since 1998 in the area of UML-SPE. As this research area matures and the number of related papers increases, it is important to systematically classify the current state-of-the-art and to provide an overview of the trends in this specialized field (Kitchenham, et al., 2009; Kitchenham & Charters, 2007; Petersen, et al., 2008). Such categorized results provide many benefits to the broader community. For example, they are valuable resources for new researchers (e.g., PhD students) aiming to identify open research areas needing further work and/or to conduct additional secondary studies (Kitchenham, et al., 2009; Kitchenham & Charters, 2007; Petersen, et al., 2008). A secondary study is defined as a study of studies (Jill, et al., 2011).
We systematically analyze and classify the body of knowledge related to UML-SPE through a systematic mapping (SM) study (Petersen, et al., 2008). As part of this study, we pose two sets of research questions, define selection and exclusion criteria, and systematically develop and refine a systematic map (classification schema). In addition, we conduct a bibliometric and trend analysis of the included papers, to analyze the annual trend of papers, and identify the active researchers and most cited studies (a smaller-scale version of larger studies such as (Garousi & Varma, 2010; Tse, et al., 2006; W. Eric, et al., 2011)).
After a careful selection process (Section 4), our study pool includes a set of 90 papers (from the set of 114 identified papers) published in the area of UML-SPE between 1998 and 2011. The full version of our mapping data is available through a publicly-accessible online repository (Garousi, et al., Last accessed: Sept. 5, 2012). We derive the trends for instance in terms of types of papers (types of contribution and research facets), types of SPE activities (e.g., modeling, model transformation, and testing), and types of metrics and evaluations used in papers.
The main contributions of this article are four-fold:
- •
A systematic map (Section 5) developed for the area of UML-SPE which could be used for other secondary studies (e.g., surveys, systematic mappings or systematic literature reviews) in this field.
- •
Systematic mapping of the existing research in this area (Sections 6).
- •
An online article repository which has been created during this SM (Garousi, et al., Last accessed: Sept. 5, 2012).
- •
Bibliometric, trends and demographics data of the UML-SPE field (Section 7).