Abstract
This chapter describes the evolution of approaches to empirical software engineering from goal and data-driven to the latest purchaser-centered approach. The new Japanese Software Traceability and Accountability for Global software Engineering (StagE) project is developing this approach to ensure the transparency of software development processes and products for software purchasers by “tagging” software with empirical software development data. Global software development raises unprecedented difficulties for developers, including the international and intercorporate coordination of development and distribution, the change to composition as the primary development approach, the shift to software everywhere talking to everything, and continuing upgrades and interaction with released software. To work effectively in this environment, empirical data collection, analysis, and feedback must extend throughout the software lifecycle including both production and usage data.
TopBackground
Top-Down, Bottom-Up, or Sideways: Getting the Data Right or Getting the Right Data?
One of the key questions in empirical software engineering is exactly what data do you want to collect. Empirical software engineering has used two basic approaches to decide what data to collect. The first, goal-driven metrics, typically starts with high-level goals or business directions and works down to specific data and metrics. One difficulty with this is that the data collection often is very specific to the environment and projects. The second approach has been to start with automated data collection and work upward to develop abstractions and analyses. Perhaps the leading example of this is the Hackystat project, which provides a wide array of data collection tools and a platform to tie them together. (Johnson, 2008) One of the difficulties with this approach has been linking the low-level data to business goals and abstractions.
Goal-driven metrics and data-driven approaches have shown the abilities of empirical software engineering to improve the software process. However, the managers and developers often have different interests from the purchasers of the software, suggesting that changing the stakeholders driving the selection and use of the data can provide a more effective process for selection and application of empirical measurements. But before we look at that new approach, let's take a brief look at the older approaches.
The Conventional Approach to Empirical Software Engineering: Goal-Driven Metrics
Today, there are several national projects on empirical software engineering in Japan (EASE Project, 2007; SEC, 2008), Australia (NICTA, 2008), Germany (IESE, 2008), and the USA (CeBASE, 2004). We can find many research papers concerning empirical topics in major conferences in software engineering.
In most conventional projects and papers, only software developers use empirical data about software development to improve software quality and productivity. However, developers’ needs for software quality and productivity are often too abstract to relate to the data collected in software projects. Models and techniques that derive the “data to be collected and analyzed” from the “goal to be achieved” play an important role. The GQM (Goal/Question/Metric) approach proposed by Prof. Basili, and Measurement Information Model defined in ISO/IEC 15939 may be helpful. In addition, real-time data collection, analysis, and feedback are also important. Postmortem analysis and feedback are not powerful enough to achieve various kinds of goals in software quality and productivity.
Key Terms in this Chapter
Goal-Driven Metrics: Data collection and analysis based on business goals.
STAGE: Software Traceability and Accountability for Global Software Engineering Project.
Traceability: providing information to allow tracing the origin of problems into the development process by purchasers.
Transparency: Providing visibility into the development process for purchasers.
Ubiquitous, Networked Software: Software in almost all consumer products, with data networking connecting it all.
Software tag: A collection of empirical data and abstracted information from the development process associated with a software product.
Composition: Software construction using components or parts.
Purchaser-Centered: Driving the development methods and decisions based on purchaser lifecycle benefits, rather than developer-centered ease.
Empirical Software Engineering: Software engineering based on empirical data.