Ontology Alignment Quality: A Framework and Tool for Validation

Introduction

Ontologies are a promising technology for supporting data interoperability by providing a basis for integrating separate domains through the identification of logical connections or constraints between data schemas. Furthermore, ontologies are well suited for defining shared conceptualizations in order to support the interoperability of heterogeneous and inter-organizational sources of information. Through automated reasoning ontologies provide the flexibility required for navigating through different levels of abstraction and querying the overall body of knowledge about business processes. Interoperability problems arise from the fact that developers of information systems use different vocabularies to express the information contained in the systems even when describing the same domain. In many cases, applications require information from multiple information sources. However, an application can only use information from two software systems completely and accurately, if precise semantic correspondences among the data from the two systems can be made. Ontology alignment tools such as Chimæra (McGuinness, 2000), FOAM (Ehrig, 2005), OMEN (Mitra et al., 2005) and Prompt (Noy & Musen, 2003) help establish semantic correspondences between data entities contained in different software systems.

The overall research process that was followed is shown in Figure 1. The research questions we address in this paper are as follows.

Figure 1.

The research process

• 1.

  How can we define ontology alignment quality?

• 2.

  Does visualization improve ontology alignment quality and how?

To maintain quality throughout the research we applied the design-science research guidelines developed by Hevner et al. (2004). Design-science research requires the use of rigorous methods in both the development and evaluation of the designed artifact. Since artifact quality depends on the selection of appropriate techniques to develop or construct a theory, we based our framework for ontology alignment quality on a well established quality framework (Krogstie et al., 1995; Lillehagen & Krogstie, 2008).

The core subject of the paper is ontology alignment, whereby we develop two design science artifacts, a framework and a visualization tool. We define ontology alignment quality as a framework comprising: semantic, syntactic, pragmatic and social quality dimensions. According to Denning (1997) and Tsichritzis (1998) artifacts built in design science research are rarely full-grown information systems that are used in practice. Instead, artifacts are innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, and use of information systems can be effectively achieved.

It is well known that evaluation is an important component of the research process. Evaluation of a design artifact requires the definition of appropriate metrics and the gathering and analysis of appropriate data. We designed and undertook a controlled experiment to perform the evaluation of the visual ontology alignment tool. Hevner et al. (2004) claim that a controlled experiment is a relevant evaluation method for designed artifacts. We found that the evaluation phase provided essential feedback to what Hevner et al. (2004) refers to as the ‘construction’ phase. Accordingly a design artifact is complete and effective when it satisfies the requirements and constraints of the problem it was meant to solve. The evaluation of the prototype tool resulted in promising results for supporting ontology alignment tasks such as: checking candidate alignments and locating missing alignments.