Examining the Quality of Evaluation Frameworks and Metamodeling Paradigms of Information Systems Development Methodologies

Examining the Quality of Evaluation Frameworks and Metamodeling Paradigms of Information Systems Development Methodologies

Eleni Berki (University of Jyväskylä, Finland)
DOI: 10.4018/978-1-60566-278-7.ch015
OnDemand PDF Download:


Information systems development methodologies and associated CASE tools have been considered as cornerstones for building quality in an information system. The construction and evaluation of methodologies are usually carried out by evaluation frameworks and metamodels - both considered as meta-methodologies. This chapter investigates and reviews representative metamodels and evaluation frameworks for assessing the capability of methodologies to contribute to high-quality outcomes. It presents a summary of their quality features, strengths and weaknesses. The chapter ultimately leads to a comparison and discussion of the functional and formal quality properties that traditional metamethodologies and method evaluation paradigms offer. The discussion emphasizes the limitations of both methods and meta-methods to model and evaluate software quality properties such as computability and implementability, testing, dynamic semantics capture, and people’s involvement. This analysis along with the comparison of the philosophy, assumptions, and quality perceptions of different process methods used in information systems development, provides the basis for recommendations about the need for future research in this area.
Chapter Preview


In traditional software engineering, the information systems development (ISD) process is defined as a series of activities performed at different stages of the system lifecycle in conformance with a suitable process model (method or methodology). In the fields of Information Systems and Software Engineering, the terms methodology and method are often used interchangeably (Nielsen, 1990; Berki et al., 2004). Increasingly, new methods, techniques and automated tools have been applied in Software Engineering (SE) to assist in the construction of software-intensive information systems. Quality frameworks and metamodels are mainly concerned with the evaluation of the quality of both the process itself and the resulting product at each stage of the life cycle including the final product (the information system).

Professional bodies such as IEEE and ISO have established quality standards and software process management instruments such as Software Process Improvement and Capacity dEtermination (SPICE) (Dorling, 1993) and Capability Maturity Model (CMM) (Paulk et al., 1993) have focused on the quality properties that the ISD process should demonstrate in order to produce a quality information system (Siakas et al., 1997). However, software quality assurance issues (Ince, 1995) such as reliability (Kopetz, 1979) and predictability, measurement and application of software reliability in particular (Myers, 1976; Musa et al., 1987), have long preoccupied software engineers, even before quality standards.

IS quality improvement can be achieved through the identification of the controllable and uncontrollable factors in software development (Georgiadou et al., 2003). ISD methodologies and associated tools can be considered as conceptual and scientific ways to provide prediction and control; their adoption and deployment, though, by people and organizations (Iivari & Huisman, 2001) can generate many uncontrollable factors. During the last thirty-five years, several methodologies, techniques and tools have been adopted in the ISD process to advance software quality assurance and reliability. A comprehensive and detailed coverage of existing information systems development methodologies (ISDMs) has been carried out by Avison & Fitzgerald (1995), with detailed descriptions of the techniques and tools used by each method to provide quality in ISD.

Several ISDMs exist. Berki et al., (2004) classified them into families, highlighting their role as quality assurance instruments for the software development process. They have been characterized as hard (technically oriented), soft (human-centered), hybrid (a combination of hard and soft), and specialized (application-oriented) (Berki et al., 2004). Examples of each include:

  • Hard methods - object-orientated techniques, and formal and structured families of methods;

  • Soft methods - Soft Systems Method (SSM) and Effective Technical and Human Implementation for Computer-based Systems (ETHICS)

  • Hybrid methods - Multiview methodology, which is a mixture of hard and soft techniques;

  • Specialized methods - KADS, extreme programming (XP) and other agile methods.

The contribution of these methods to the quality of the ISD process has been a subject of controversy; particularly so because of the different scope, assumptions, philosophies of the various methods and the varied application domains they serve. For example, it is believed that the human role in ISD bears significantly on the perception of the appropriateness of a method (Rantapuska et al., 1999); however, usability definitions in ISO standards are limited (Abran et al., 2003). There is empirical support for the notion that a methodology is as strong as the user involvement it supports (Berki et al., 1997).

Complete Chapter List

Search this Book:
Editorial Advisory Board
Table of Contents
Chapter 1
Terry Halpin
When modeling information systems, one often encounters subtyping aspects of the business domain that can prove challenging to implement in either... Sample PDF
Enriched Conceptualization of Subtyping
Chapter 2
Alessandro Artale, C. Maria Keet
This chapter focuses on formally representing life cycle semantics of part-whole relations in conceptual data models by utilizing the temporal... Sample PDF
Essential, Mandatory, and Shared Parts in Conceptual Data Models
Chapter 3
Peter Bollen
In this chapter the authors extend the ORM conceptual modeling language with constructs for capturing the relevant parts of an application ontology... Sample PDF
Extending the ORM Conceptual Schema Language and Design Procedure with Modeling Constructs for Capturing the Domain Ontology
Chapter 4
Janis Stirna, Anne Persson
This chapter presents experiences and reflections from using the EKD Enterprise Modeling method in a number of European organizations. The EKD... Sample PDF
EKD: An Enterprise Modeling Approach to Support Creativity and Quality in Information Systems and Business Development
Chapter 5
Anders Carstensen, Lennart Holmberg, Kurt Sandkuhl, Janis Stirna
This chapter discusses how an Enterprise Modeling approach, namely C3S3P1, has been applied in an automotive supplier company. The chapter... Sample PDF
Integrated Requirement and Solution Modeling: An Approach Based on Enterprise Models
Chapter 6
John Krogstie, Frank Lillehagen
Innovative design is the most important competitive factor for global engineering and manufacturing. Critical challenges include cutting lead times... Sample PDF
Methodologies for Active Knowledge Modeling
Chapter 7
Peretz Shoval, Mark Last, Avihai Yampolsky
In the analysis phase of the information system development, the user requirements are studied, and analysis models are created. In most UML-based... Sample PDF
Data Modeling and Functional Modeling: Examining the Preferred Order of Using UML Class Diagrams and Use Cases
Chapter 8
Mauri Leppänen
A large number of strategies, approaches, meta models, techniques and procedures have been suggested to support method engineering (ME). Most of... Sample PDF
OntoFrame: An Ontological Framework for Method Engineering
Chapter 9
Patrick van Bommel, Stijn Hoppenbrouwers, Erik Proper, Jeroen Roelofs
A process-oriented framework (QoMo) is presented that aims to further the study of analysis and support of processes for modeling. The framework is... Sample PDF
Concepts and Strategies for Quality of Modeling
Chapter 10
John Erickson, Keng Siau
This chapter presents the basic ideas underlying Service Oriented Architecture as well as a brief overview of current research into the phenomena... Sample PDF
Service Oriented Architecture: A Research Review from the Software and Applications Perspective
Chapter 11
Vítor Estêvão Silva Souza, Ricardo de Almeida Falbo, Giancarlo Guizzardi
In the Web Engineering area, many methods and frameworks to support Web Information Systems (WISs) development have already been proposed.... Sample PDF
Designing Web Information Systems for a Framework-Based Construction
Chapter 12
Tony Elliman, Tally Hatzakis, Alan Serrano
This paper discusses the idea that even though information systems development (ISD) approaches have long advocated the use of integrated... Sample PDF
Business Process Simulation: An Alternative Modelling Technique for the Information System Development Process
Chapter 13
Leandro Dias da Silva, Elthon Allex da Silva Oliveira, Hyggo Almeida, Angelo Perkusich
In this chapter a formal agent based approach for the modeling and verification of intelligent information systems using Coloured Petri Nets is... Sample PDF
An Agent Based Formal Approach for Modeling and Verifying Integrated Intelligent Information Systems
Chapter 14
Jan vom Brocke
With the design of reference models, an increase in the efficiency of information systems engineering is intended. This is expected to be achieved... Sample PDF
Design Principles for Reference Modelling: Reusing Information Models by Means of Aggregation, Specialisation, Instantiation and Analogy
Chapter 15
Eleni Berki
Information systems development methodologies and associated CASE tools have been considered as cornerstones for building quality in an information... Sample PDF
Examining the Quality of Evaluation Frameworks and Metamodeling Paradigms of Information Systems Development Methodologies
About the Contributors