Quantitative Productivity Analysis of a Domain-Specific Modeling Language

Quantitative Productivity Analysis of a Domain-Specific Modeling Language

Joe Hoffert (Indiana Wesleyan University, USA), Douglas C. Schmidt (Vanderbilt University, USA) and Aniruddha Gokhale (Vanderbilt University, USA)
DOI: 10.4018/978-1-4666-6359-6.ch013
OnDemand PDF Download:
List Price: $37.50


Model-Driven Engineering (MDE), in general, and Domain-Specific Modeling Languages (DSMLs), in particular, are increasingly used to manage the complexity of developing applications in various domains. Although many DSML benefits are qualitative (e.g., ease of use, familiarity of domain concepts), there is a need to quantitatively demonstrate the benefits of DSMLs (e.g., quantify when DSMLs provide savings in development time) to simplify comparison and evaluation. This chapter describes how the authors conducted quantitative productivity analysis for a DSML (i.e., the Distributed Quality-of-Service [QoS] Modeling Language [DQML]). The analysis shows (1) the significant quantitative productivity gain achieved when using a DSML to develop configuration models compared with not using a DSML, (2) the significant quantitative productivity gain achieved when using a DSML interpreter to automatically generate implementation artifacts as compared to alternative methods when configuring application entities, and (3) the viability of quantitative productivity metrics for DSMLs.
Chapter Preview


Model-driven engineering (MDE) helps address the problems of designing, implementing, and integrating applications (Hästbacka, 2011)(Lukman, 2010)(Schmidt, 2006)(Hailpern, 2006)(Atkinson, 2003)(Kent, 2002). MDE is increasingly used in domains involving modeling software components, developing embedded software systems, and configuring quality-of-service (QoS) policies. Key benefits of MDE include (1) raising the level of abstraction to alleviate accidental complexities of low-level and heterogeneous software platforms, (2) more effectively expressing designer intent for concepts in a domain, and (3) enforcing domain-specific development constraints. Many documented benefits of MDE are qualitative, e.g., use of domain-specific entities and associations that are familiar to domain experts, and visual programming interfaces where developers can manipulate icons representing domain-specific entities to simplify development. There is a lack of documented quantitative benefits for domain-specific modeling languages (DSMLs), however, that show how developers are more productive using MDE tools and how development using DSMLs yields fewer bugs.

Conventional techniques for quantifying the benefits of MDE in general (e.g., comparing user-perceived usefulness of measurements for development complexity (Abrahao and Poels, 2007, 2009)) and DSMLs in particular (e.g., comparing elapsed development time for a domain expert with and without the use of the DSML (Loyall, Ye, Shapiro, Neema, Mahadevan, Abdelwahed, Koets, & Varner, 2004)) involve labor-intensive and time-consuming experiments. For example, control and experimental groups of developers may be tasked to complete a development activity during which metrics are collected (e.g., number of defects, time required to complete various tasks). These metrics also often require the analysis of domain experts, who may be unavailable in many production systems.

Even though DSML developers are typically responsible for showing productivity gains, they often lack the resources to demonstrate the quantitative benefits of their tools. One way to address this issue is via productivity analysis, which is a lightweight approach to quantitatively evaluating DSMLs that measures how productive developers are, and quantitatively exploring factors that influence productivity (Boehm, 1987) (Premraj, Shepperd, Kitchenham, & Forselius, 2005). This chapter applies quantitative productivity measurement using a case study of the Distributed QoS Modeling Language (DQML), which is a DSML for designing valid QoS policy configurations and transforming the configurations into correct-by-construction implementations. Our productivity analysis of DQML shows significant productivity gains compared with common alternatives, such as manual development using third-generation programming languages. While this chapter leverages DQML as a case study, in general the productivity gains and analysis presented are representative of DSMLs’ ability to reduce accidental complexity and increase reusability.

The remainder of this chapter includes the following objectives: highlighting related work; presenting an overview of DQML; outlining the DQML case study as a basis for DSMLs; describing productivity analysis for DSMLs leveraging the DQML case study; and presenting concluding remarks and lessons learned.

Key Terms in this Chapter

Quality of Service (QoS): Properties of a software system related to non-functional aspects ( e.g. , latency, reliability, usability).

DDS Benchmarking Environment: An open-source tool suite developed to examine and evaluate the QoS of various DDS implementations.

Middleware: A software abstraction layer that typically resides between the operating system and the application layer ( e.g. , the Java Virtual Machine).

Publish/Subscribe (Pub/Sub): An architectural paradigm for software systems that is data-centric (i.e., data is sent out and received) rather than call-centric ( i.e. , request/response typically done by invoking a function and receiving a response).

Productivity Analysis: Analyzing the use of various software system development techniques to show productivity gains or losses ( i.e. , typically concerned with the productivity of the software developers).

Domain Specific Modeling Language (DSML): A computer language (typically graphical in nature) developed for a particular domain ( e.g. , disease diagnosis, QoS configuration).

Data Distribution Service (DDS): An open standard for QoS-enabled pub/sub middleware supported by the Object Management Group (OMG).

Complete Chapter List

Search this Book: