Article Preview
TopThe Ontology Alignment Evaluation Initiative: Oaei
Euzenat et al. (Euzenat et. al., 2005) recognised early on that a common methodology is needed that facilitates the evaluation of ontology mapping approaches. In response to this need, the OAEI was established. The first comparison campaign was organised in 2004 and since then the evaluation takes place yearly. The establishment of the OAEI 1 has paved the way to a continuously improving common methodology for comparing ontology mapping systems. As a result, common practices of the past, involving self-evaluation and drawing conclusions from different data sets could be reduced, and ontology mapping approaches could be compared in an objective manner. Besides the general objectives, OAEI organisers have put forward some very specific objectives, like to establish the advantages and disadvantages of each proposed ontology mapping approach. For example, participation in the OAEI allows system developers to discover that a particular system performs better for class alignment than for individual alignments. Furthermore, there were other objectives such as establishing a direct and open forum between system developers. However, based on DSSim's participation in the OAEI events, these objectives were only secondary and received little emphasis between the years 2006-2009. Nevertheless, one remarkable trend was recognisable, namely the continuous improvement and increasing difficulty of the matching tracks throughout the years. In general, the success of the initiative, to bring a coherent methodology for evaluating and comparing systems, is unquestionable. In order to achieve its goals, the OAEI set up yearly evaluation workshops that were organised in parallel with the International Semantic Web Conference (ISWC). The tests and results of each evaluation were publicly accessible, and organisers published a yearly summary (Shvaiko et al., 2006; Euzenat et al., 2007; Caracciolo et al., 2008; Euzenat et al., 2009) of the evaluation and the comparison of the results with the previous years.