Application of Multiple Criteria Decision Analysis and Optimisation Methods in Evaluation of Quality of Learning Objects

Application of Multiple Criteria Decision Analysis and Optimisation Methods in Evaluation of Quality of Learning Objects

Eugenijus Kurilovas (Vilnius University, Vilnius Gediminas Technical University, Lithuania), Irina Vinogradova (Vilnius University, Vilnius Gediminas Technical University, Lithuania) and Silvija Serikoviene (Vilnius University, Lithuania)
Copyright: © 2011 |Pages: 15
DOI: 10.4018/ijopcd.2011100105
OnDemand PDF Download:
No Current Special Offers


This paper analyses and presents the new scientific models and methods for the expert evaluation of quality of learning objects (LOs) paying special attention to LOs reusability level. Currently all existing approaches in the area are quite subjective and depend only on the experience of the decision-makers. The authors analyse several scientific methods and principles to minimise the subjectivity level in the expert evaluation of LOs quality. They are: (a) the principles of multi-criteria decision analysis for identification of quality criteria, (b) technological quality criteria classification principle, (c) fuzzy group decision making theory to obtain evaluation measures, (d) normalisation of the weights of criteria, and (e) scalarisation method for LOs quality optimisation. The authors demonstrate that the complex application of these approaches could significantly improve the quality of the expert evaluation of LOs and noticeably reduce the level of the expert evaluation subjectivity. The paper also presents the example of practical application of these approaches for evaluation of LOs for Mathematics subject.
Article Preview


Educational sector needs to provide the students with learning objects (LOs) of high quality. The main stakeholders in the educational sector are the educational institutions (schools, universities, etc.), educational authorities (such as ministries of education, regional and other agencies, etc.) and policy makers, and providers of learning content (e.g., publishers). Since educational institutions are interested to use the quality LOs, they need some kind of proper approaches, models and methods how to choose the quality LOs in the market or to find free-of-charge LOs of high quality. The publishers are interested to propose such LOs to the institutions, and the policy makers are interested to know these approaches in order to formulate the education policy (e.g., while implementing the public tenders on provision the educational sector with LOs).

All the education sector stakeholders need to know for sure what LOs are quality ones or what are not. Therefore, this problem is of very high practical relevance for the educational sector that needs clear and easy to use approaches to evaluate the quality of LOs, both proprietary and free ones. These quality evaluation models and methods have to fit all the stakeholders’ needs.

Therefore, the problem of evaluation of the quality of LOs is high on the agenda of the international research.

Currently a number of LOs available through search engines and repositories is rapidly increasing. The provision of LOs ensures better access to quality LOs and supports enhanced learning outcomes. According to Haughey and Muirhead (2005), the purpose of LOs is to increase the effectiveness of learning by making LOs more readily available, also by reducing the cost and effort to produce quality LOs, and by allowing LOs to be more easily shared.

One of the main criteria for achieving the high level of LOs effectiveness and efficiency is LOs reusability (Kurilovas, 2009c). According to McCormick et al. (2004), the need for reusability of LOs has at least three elements, namely:

  • 1.

    Interoperability – it means that LO is interoperable and can be used in different platforms.

  • 2.

    Flexibility in terms of pedagogic situations – it means that LO can fit into a variety of pedagogic situations.

  • 3.

    Modifiability to suit a particular teacher’s or student’s needs – it means that LO can be made more appropriate to a pedagogic situation by modifying it to suit a particular teacher’s or student’s needs.

The evaluation of LOs is a comparatively new concern because the quantity of LOs has grown and the development of their repositories has come about to allow for greater ease in finding and using LOs for blended learning (Kurilovas, 2009a). According to Haughey and Muirhead (2005), the growth in the number of LOs, the multiplicity of authors, their increasing diversity of design and their availability to educators has generated interest in how to evaluate them and which evaluation criteria to use to make judgments about their quality and usefulness.

Teachers, students, and instructional designers can access large LO repositories and a number of even larger meta-collections such as European Schoolnet’s (EUN – a network of 31 Ministries of Education in Europe) Learning Resource Exchange (LRE, 2010) which currently contains more than 130,000 LOs and assets from over 25 providers.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 12: 4 Issues (2022): Forthcoming, Available for Pre-Order
Volume 11: 4 Issues (2021): 3 Released, 1 Forthcoming
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing