A Framework for Collecting Experiences

11.1 The Rationale

An informed introduction to AMs requires the ability to determine whether and when AMs are better than traditional software development methodologies. The risk is that AMs are considered just like another tool. Altogether to accredit AMs we need to show the qualified evidence of their effectiveness, performance, productivity, in the different contexts where they can be introduced.

This analysis is difficult as such effectiveness varies with the development environment, depending on several aspects, such as skills, resources and culture. However, this analysis is a key ingredient for the creation of a comprehensive body of knowledge on AMs.

To achieve our objectives, we need to collect existence experience on AM and to formalize it. To such an end we need to define a common experience framework where we identify rules so that experience can be archived, compared, and used to create knowledge.

The experience framework for AMs would support such an initiative defining standards and general guidelines for an experimental process in AMs. Its structure would allow some degree of freedom to guarantee its adaptation to individual and working group cultures when used in a single specific experiment.

The experience framework focuses on different issues including the ones presented in the research roadmap:

• •

  Business

• •

  Management

• •

  Human factor

• •

  Infrastructure

• •

  Technical

Each AM consists of a set of operational practices and share a set of common features with all the other AMs. As AMs are emergent, their practices are often loosely defined and applied by a subjective selection of a few core ones. Therefore, there is a need for benchmarking on AMs that evaluates and identifies best practices, tools and approaches.

Furthermore, following Highsmith’s idea (2002) of an ecosystem, an evaluation system for AMs would help interested and expert people in building their own agile method.

There are two main kinds of analysis we may perform on AMs: per single agile practice or per common features.

If we identify standards and guidelines per single agile practice, we model our experimental process to reveal practice, use and efficiency across different agile methodologies. On the other hand if we focus on common features of AMs, we analyze analogies and differences across different AMs or compare AMs with traditional approaches.

11.2 Structure Of The Experience Framework

We have identified four main ingredients to build an experience framework (Figure 1),

• 1.

  Data collection

• 2.

  Data analysis

• 3.

  Set up of the experiment

• 4.

  Generalization and validation of the results

  Figure 1.

  The four phases of the experience framework

  

The four branches are strictly interconnected. Three of them, data collection, data analysis and generalization and validation of the results are independent from the single experiment. The fourth, set up of the experiment, models general standards to the single experiment to cope with the environment of the experiment.

The structure is intended to be agile in the sense that would allow easy access, collective ownership, automated tools, iterative processes, easy reuse, iterative modifications and refinements.

11.3 Standards For Data Collection

A solid analysis requires valid data. Data validation and collection is a continuous process following all the phases of software development. Data needs to be validated in the field, for example, interviewing experts of the project. Vice versa, one can increase accuracy of data validation providing the experts guidelines for data collection.