Experiences on the Evaluation of DSSim: A Multi-Agent Ontology Mapping System

Experiences on the Evaluation of DSSim: A Multi-Agent Ontology Mapping System

Maria Vargas-Vera, Miklos Nagy
Copyright: © 2015 |Pages: 31
DOI: 10.4018/IJKSR.2015040102
(Individual Articles)
No Current Special Offers


This paper presents a comprehensive evaluation of DSSim (DSSim stands for Similarity based on Dempster-Shafer), our ontology alignment system. The authors participated several years in the annual evaluation defined by the Ontology Alignment Initiative (OAEI). Each year their DSSim was evolved and participated in more difficult tracks defined by the Ontology Alignment Initiative. In fact, DSSim obtained exceptional results in the OAEI-2008 Evaluation. In this evaluation (OAEI-2008), DSSim participated on all given tracks namely, benchmark, anatomy, fao, directory, mldirectory, library, very large crosslingual resources and conference. The challenges presented by each track were addressed by the DSSim team.
Article Preview

The Ontology Alignment Evaluation Initiative: Oaei

Euzenat et al. (Euzenat et. al., 2005) recognised early on that a common methodology is needed that facilitates the evaluation of ontology mapping approaches. In response to this need, the OAEI was established. The first comparison campaign was organised in 2004 and since then the evaluation takes place yearly. The establishment of the OAEI 1 has paved the way to a continuously improving common methodology for comparing ontology mapping systems. As a result, common practices of the past, involving self-evaluation and drawing conclusions from different data sets could be reduced, and ontology mapping approaches could be compared in an objective manner. Besides the general objectives, OAEI organisers have put forward some very specific objectives, like to establish the advantages and disadvantages of each proposed ontology mapping approach. For example, participation in the OAEI allows system developers to discover that a particular system performs better for class alignment than for individual alignments. Furthermore, there were other objectives such as establishing a direct and open forum between system developers. However, based on DSSim's participation in the OAEI events, these objectives were only secondary and received little emphasis between the years 2006-2009. Nevertheless, one remarkable trend was recognisable, namely the continuous improvement and increasing difficulty of the matching tracks throughout the years. In general, the success of the initiative, to bring a coherent methodology for evaluating and comparing systems, is unquestionable. In order to achieve its goals, the OAEI set up yearly evaluation workshops that were organised in parallel with the International Semantic Web Conference (ISWC). The tests and results of each evaluation were publicly accessible, and organisers published a yearly summary (Shvaiko et al., 2006; Euzenat et al., 2007; Caracciolo et al., 2008; Euzenat et al., 2009) of the evaluation and the comparison of the results with the previous years.

Complete Article List

Search this Journal:
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing