The first contribution of this paper consists on a live catalogue of pitfalls that extends previous works on modeling errors with pitfalls resulting from an empirical analysis of numerous ontologies. Such a catalogue classifies pitfalls according to the Structural, Functional and Usability-Profiling dimensions. For each pitfall, we include the value of its importance level (critical, important and minor). The second contribution is the description of OntOlogy Pitfall Scanner (OOPS!), a widely used tool for detecting pitfalls in ontologies and targeted at newcomers and domain experts unfamiliar with description logics and ontology implementation languages. The tool operates independently of any ontology development platform and is available through a web application and a web service. The evaluation of the system is provided both through a survey of users' satisfaction and worldwide usage statistics. In addition, the system is also compared with existing ontology evaluation tools in terms of coverage of pitfalls detected.
TopIntroduction
The Linked Data (LD) effort has become a catalyst for the realization of the vision of the Semantic Web originally proposed by Berners-Lee et al. in (Berners-Lee, Hendler, & Lassila, 2001). In this scenario, a large amount of data, annotated by means of ontologies, is shared on the Web. Such ontologies enrich the published data with semantics and help their integration. In other cases, ontologies are used to model data automatically extracted from web sources, which can be noisy and contain errors. Therefore, ontologies not only must be published according to LD principles1, but they also must be accurate and of high quality from a knowledge representation perspective in order to avoid inconsistencies or undesired inferences.
The correct application of ontology development methodologies (e.g., METHONTOLOGY (Fernández-López et al., 1999), On-To-Knowledge (Staab et al., 2001), DILIGENT (Pinto, Tempich, & Staab, 2004), or the NeOn Methodology (Suárez-Figueroa et al., 2012)) benefits the quality of the ontology being built. However, such a quality is not totally guaranteed because ontologists face a wide range of difficulties and handicaps when modeling ontologies (Aguado de Cea et al., 2008; Blomqvist, Gangemi, & Presutti, 2009; Rector et al., 2004), and this fact may cause the appearance of anomalies in ontologies. Therefore, in any ontology development project it is vital to perform the ontology evaluation activity since this activity checks the technical quality of an ontology against a frame of reference.
In the last decades a huge amount of research and work on ontology evaluation has been conducted. Some of these attempts define a generic quality evaluation framework (Duque-Ramos et al., 2011; Gangemi et al., 2006; Gómez-Pérez, 2004; Guarino, & Welty, 2009; Strasunskas, & Tomassen, 2008); others propose evaluating an ontology depending on its final (re)use (Suárez-Figueroa, 2010); some others propose quality models based on features, criteria, and metrics (Burton-Jones et al., 2005); whereas others present methods for pattern-based evaluation (Djedidi, & Aufaure, 2010; Presutti et al., 2008).
As a consequence of the emergence of new methods and techniques, a few tools have been proposed. These tools ease the ontology diagnosis by reducing the human intervention. This is the case of XD-Analyzer2, a plug-in for NeOn Toolkit and Ontocheck3 (Schober et al., 2012), a plug-in for Protégé. The former checks some structural and architectural ontology features, whereas the latter focuses on metadata aspects. Moki4 (Pammer, 2010), a wiki-based ontology editor, also provides some evaluation features. Finally, Radon (Ji et al., 2009) is a NeOn Toolkit plug-in that detects and handles logical inconsistencies in ontologies.