Evaluating UML Using a Generic Quality Framework

Background

Earlier, we developed a framework for understanding and assessing quality of models and modeling languages (Krogstie & Sølvberg, 2003; Krogstie, Sindre, & Jørgensen, 2006).

The main concepts of the framework and their relationships are shown in Figure 1 and are explained next. Quality has been defined referring to the correspondence between statements belonging to the following sets:

Figure 1.

Framework for discussing the quality of models

• •

  G, the goals of the modeling task

• •

  L, the language extension (i.e., the set of all statements that are possible to make according to the graphemes, vocabulary, and syntax of the modeling languages used)

• •

  D, the domain (i.e., the set of all statements that can be stated about the situation at hand)

• •

  M, the externalized model itself

• •

  Ks, the relevant explicit knowledge of those being involved in modeling. A subset of these is actively involved in modeling, and their explicit knowledge is indicated by K_M.

• •

  I, the social actor interpretation (i.e., the set of all statements that the audience thinks that an externalized model consists of)

• •

  T, the technical actor interpretation (i.e., the statements in the model as “interpreted” by modeling tools)

The main quality types are indicated by solid lines between the sets and are described briefly next:

• •

  Physical quality: The basic quality goals on the physical level is that the knowledge K of the domain D has been externalized, and internalizeability, that the externalized model M is available.

• •

  Empirical quality deals with predictable error frequencies when a model is read or written by different users, coding (e.g., shapes of boxes) and HCI-ergonomics for documentation and modeling-tools. For instance, graph layout to avoid crossing lines in a model is a mean to address the empirical quality of a model.

• •

  Syntactic quality is the correspondence between the model M and the language extension L.

• •

  Semantic quality is the correspondence between the model M and the domain D. This includes validity and completeness.

• •

  Perceived semantic quality is the similar correspondence between the audience interpretation I of a model M and his or hers current knowledge K of the domain D.

• •

  Pragmatic quality is the correspondence between the model M and the audience’s interpretation and application of it (I). We differentiate between social pragmatic quality (to what extent people understand and are able to use the models) and technical pragmatic quality (to what extent tools can be made that interpret the models).