Recommender systems have shown to be successful in many domains where information overload exists. This success has motivated research on how to deploy recommender systems in educational scenarios to facilitate access to the wide spectrum of information. Tackling open issues in their deployment is gaining importance as lifelong learning becomes a necessity of the current knowledge-based society. Although Educational Recommender Systems
(ERS) share the same key objectives as recommenders for e-commerce applications (i.e. helping users to select the most appropriate item from a large information pool), there are some particularities that should be considered before directly applying existing solutions from those applications. For instance, recommendations in the educational domain should not be guided only by the learners’ preferences but also by educational criteria. However, most ERS approaches have focused on applying traditional recommendation algorithms in order to find relevant resources for learners in learning scenarios. While this approach is pointing at interesting open issues, there are complementary views in this field to address the current challenges, which may clarify grounds to successfully deploy ERS.
The handbook aims to provide a comprehensive review of state-of-the-art practices for ERS as well as the challenges to achieve their actual deployment. Some topics discussed in the handbook are the following: 1) State of the art of ERS, 2) Modeling issues in developing ERS, 3) Techniques and algorithms to produce recommendations in e-learning settings, 4) Methodologies to develop ERS, 5) Architectures to support the recommendation process, 6) Management of educational issues when providing recommendations to learners, 4) Evaluation methods to measure the recommendations’ impact on users, 5) Challenges for providing suitable recommendations in the educational domain, 6) Application of ERS in real world scenarios, and 7) New educational related areas where ERS fit purpose.
The target audience
of this handbook covers researchers interested in recommendation strategies for educational scenarios and in evaluating the impact of recommendations in learning, as well as academics and practitioners in the area of technology enhanced learning interested in supporting learners and educators in their learning and teaching tasks though personalized intelligent learning environments. Furthermore, the handbook is also targeted at teachers who are interested in using learning technologies and/or in the potential of adaptive and intelligent learning technologies.
We received 31 chapter proposals
, which ended in 13 accepted chapters (42% acceptance rate) written by a total of 44 researchers working in institutions worldwide. In particular, there are papers from 3 continents and 13 countries, as follows: Netherlands, Germany, Cyprus, Romania, Spain, Norway, and Finland in Europe, Uruguay, USA, Argentina, and Brazil in the Americas, and Singapore and China in Asia. Moreover, 53 researchers have been involved in the reviewing process
from diverse institutions in Europe, America, and Asia (see reviewers’ list for details).
The reviewing process consisted of several phases. First, the chapter proposals were reviewed by the editors who provided feedback to the authors aimed to better fit the objectives of the handbook. The full chapters were reviewed by 4 reviewers, consisting of 1 from the Editorial Advisory Board (EAB), 2 from the Program Committee (PC), and 1 from the Additional Reviewers list. Editors also reviewed the chapters, made additional comments, highlighted critical issues, and asked authors of initially accepted papers to consider all comments in their revised version. The authors of papers who were considered valuable for the handbook had to submit their revised version and explain in detail how they addressed the reviewers’ comments. Afterwards, reviewers and editors read through the revised version and authors’ explanations. For some submissions, a second round of revision was needed. Final decisions of acceptance were taken based on the last received reviews. Additional final comments were given to authors for preparing the final version. The aim of this reviewing process was to obtain high quality chapters that properly cover relevant field issues.
With regards to the contents of the Handbook on Educational Recommender Systems and Technologies: Practices and Challenges, ERS are currently undergoing a rapid evolving process as reflected by the ongoing developments in the field. We have identified several practices taking place in two different directions. First, those focused on design time issues
, which involve knowledge modeling to feed the recommendation process. This issue is of relevance as educational aspects are to be taken into account when developing recommender systems for educational scenarios. Second, the runtime issues
, which deal with techniques, algorithms, and architectures that differ from those used in other domains to support the delivery of recommendations. The submissions received show that very often, design and runtime practices are still in the research arena and range from conceptual descriptions to the implementation and testing of prototypes, although gradually, recommender systems are being used in real world e-learning scenarios. Supporting this progress entails facing challenges both on addressing educational issues, and on taking advantage of increasing opportunities coming from the learner's context, such as those from mobile devices.
The contributions of this handbook are grouped into the following four sections:
- Section 1: Knowledge modeling for the recommendation process
- Section 2: Techniques, algorithms and architectures to support the recommendation process
- Section 3: Experiences in real world e-learning scenarios
- Section 4: Challenges for ERS
focuses on knowledge modeling for the recommendation process and presents different approaches aimed to model at design time the recommendation needs required in ERS. This modeling can be done following ad-hoc approaches, using existing modeling structures such as content or topic maps, or through ontologies. This section compiles the first four chapters of the handbook.
In Chapter 1
, Okoye et al.
outline a science ERS for high school students that can support student learning of science content by providing just-in-time personalized information retrieval of vetted science content to intentional learners in an informal setting. Intentional learners are learners that are intrinsically motivated to learn and seek out resources by themselves. The authors discuss a prototype system, the Customized Learning Service for Concept Knowledge (CLICK), which is an application designed to provide recommendations on digital library resources based on user’s concept knowledge derived from automated evaluation and approximation of their knowledge state from essay writing. CLICK employs recommendation techniques to suggest resources to learners aiming at addressing the misconceptions and incorrect knowledge that are found by comparing automatically generated domain and learner models. Three core components to personalization in the context of intentional learning were established: first, a learner model that accounts for the learner state over time; second, a designed approach built upon a foundation of strong resources that are expert-approved and vetted; third, a recommender technology approach that supports traditional recommender techniques as well as social-based techniques. By exploring this system, the authors aim to contribute to better understand the knowledge modeling requirements to make comprehensive recommendations that account for the learner’s knowledge state.
In Chapter 2
describes a recommender system that is intended to guide students to activities for which they are ready by using a framework for structuring digital learning activities in terms of knowledge, skills, and abilities. It uses a structured map of mathematics concepts and processes to power a recommender system called Metis. The Metis approach to recommender systems uses the content map to determine the knowledge of the learner and to select a set of activities the learner is ready for. Its reliability draws on a content map structured by prerequisite relations between knowledge, skills, and abilities. The approach is built on cognitively guided instruction, which has not been implemented in an automated system to date. In contrast to most tutoring systems, which take a procedural approach, Metis tries to find a good place to start rather than try to make corrections along the way. It is designed to assess in real-time how students are performing and to use this information, along with a content map, as a basis for recommending a set of activities that the learner is ready for. Metis is focused on pooling together all the digital learning content that is available on the web in order to help finding quality educational activities. This approach is intended to make activities more engaging, and thus encouraging learners’ interest in the content area.
In Chapter 3
, Sielis et al.
describe a method to support the creativity process through context-aware recommendations. The method uses ontologies for representing the context knowledge and the topic maps technology for storing, managing, and delivering content used as recommendations. The context in which ideas are developed and the context around the user task are considered to enhance creativity. Such context can be exploited in offering context-aware recommendations to the users on relevant resources, people, ideas, projects, et cetera. These recommendations are provided to users during the creativity process and the learning involved. These authors also present the software system that has been developed to support this method. The goal of their system is to recommend suitable patterns for the current problem and its context. The recommended pattern is ontology independent and is based on problem’s parameters the user is trying to solve (e.g. type of the problem, problem definition, problem complexity, if the problem is divisible, objectives, if expert knowledge is required, etc.). The evaluation carried out shows positive results.
In Chapter 4
, Diaz et al.
introduce an educational resource recommender that uses an ontology network to assist the modeling and execution of a system based on quality features for a given user and context. Authors show how different ontologies can be modeled as an ontology network in order to explicitly specify the relevant features for the domains identified. Such features are i) specific educational domain, ii) quality assurance domain, iii) user context domain, and iv) recommendation criteria domain. The ontology network can be tailored to specific domains and user profiles. The design aims to obtain a flexible model that is not dependent on any particular mechanism of content evaluation, such as a specific quality metric or educational domain. Whenever it is required to assess a different quality dimension or to consider another educational domain, new extensions of learning object metadata, educational resources quality and recommendation criteria ontologies might be added, keeping up the core model intact. This work shows how the ontology-based reasoning mechanism can be used to validate the recommendation criteria, thus looking for flexibility in tailoring the educational resource adequacy features called educational resource quality.
Once recommendations have been modeled for the educational domain, appropriate techniques, algorithms, and architectures have to be designed and implemented in order to provide the recommendations support in the educational system. Thus, Section 2
covers these issues. Firstly, there are techniques to deal with the reputation of the learners or predicting the student performance. Secondly, meta-search engines supported by ontology-based clustering algorithm or multi-agent architectures can be applied to provide flexible content-based retrieval that facilitates finding educational resources. Thirdly, meta-rules are used to define set of educationally oriented rules. Finally, architecture-centric approaches support designing ERS. Six chapters have been included in this section that deal with these issues.
In Chapter 5
, Hennis et al.
propose a reputation model intended to support knowledge sharing and management, quality assurance, and user engagement in peer-based online learning communities. Online communities have the potential to support learning as a creative process, and reputation systems can support the process in different ways, as for motivation and quality concerns. Recommender technologies have been proposed to automatically produce recommendations based on interest and interactions in online learning environments, which use either collaborative filtering techniques or content filtering techniques. Both techniques can be combined with reputation management to improve the recommendation process. Reputation derives from learners’ past, which relates to the sum of their interactions, actions, and contributions in that environment. Thus, the reputation information can be a useful input for ERS. The authors discuss that reputation management systems can be used: i) to make a contextualized picture of the individuals’ interests and capability to solve problems, and ii) to motivate individuals to share knowledge in online communities, to recognize learning activities carried out in peer-to-peer communities as a complement to formal certificates.
In Chapter 6
, Thai-Nghe et al.
focus on the problem of predicting student performance in the context of ERS by applying state-of-the-art factorization techniques to generate the most accurate rating (or performance prediction). Moreover, as a natural fact, the knowledge of learners improves over time, so factorization methods should and can take the temporal effect into account. Experimental results show that the proposed temporal extension, i.e. tensor factorization, can improve the prediction results. Student performance data from two real data sets from the KDD Challenge 2010 were used and compared with some other baselines. In this way, authors discuss how they can deliver recommendations for many problems, such as to recommend to the students some similar exercises with the appropriate difficulty level (i.e. not too hard nor too easy for them) in order to help them improve their knowledge by learning/doing similar tasks, or to recommend similar grammar structures, vocabularies, or a similar problem/section when the student is learning or doing exercises in an English course, et cetera. The proposed method opens an approach to tackle the need of considering recommendations features different from the preferred learning activities as they might not be the pedagogically the most adequate. This is entitled to facilitate recommendation of tasks based on students’ performance but not on their preferences.
In Chapter 7
, Bodea et al.
present a meta-search approach, meant to deliver bibliography from the Internet according to trainees’ results obtained from an e-assessment task. It uses an ontology-based clustering algorithm for results’ presentation that produces recommendations for the students who want to use the e-assessment tool in project management. The meta-search engine is part of an ERS, attached to an e-assessment application for project management knowledge. The procedure is included in a recommender engine, which fires recommendations each time an e-assessment application is used. Meta-search means that, for a specific query (or mistake made by the trainee), several search mechanisms for suitable bibliography (further reading) could be applied. The lists of results delivered by the standard search mechanisms are used to build thematically homogenous groups using an ontology-based clustering algorithm. Exploiting the clustering technologies improves the performance issues of the ERS attached to the e-assessment application, which is intended to gain a formative value and become a learning tool. Authors present experimentation results and discuss the efficiency of the proposed solution in the context of similar applications.
In Chapter 8
, Casali et al.
describe an ERS that is intended to help learners to find educational resources that are appropriate to their needs and preferences. The search is performed in different repositories of learning objects, where each object has descriptive metadata. A multi-agent architecture that includes several types of agents with different functionalities is used to support content-based retrieval, which provides an ordered list of the resources according to the user profile data. The framework is meant to be scalable and able to work with heterogeneous and distributed information from different learning object repositories, thus showing that the system architecture is feasible and can help people to find suitable objects. Authors present the design of the personalized search agent that takes a hybrid approach using content- and knowledge-based techniques. This agent is in charge of showing the retrieved items in the correct order. To evaluate the proximity between the recommended ranking and the user’s own ranking, authors carried out two experimentations with resources from the Ariadne repository. Six users evaluated whether the retrieved objects were interesting for them and if the recommended order was correct, according to his/her opinion, after browsing the retrieved resources.
In Chapter 9
, Romero et al.
present a solution to the problem of generating recommendations for educational applications where required amount of data are not available for the algorithms. To this, they introduce a new abstraction level, meta-rules. Rules can offer specific recommendations even with no usage information. However, large rule-sets are hard to maintain, reengineer, and adapt to user preferences. Meta-rules are used as rules that generate rules, and generalize a rule-set providing bases for adaptation, reengineering, and on-the-fly generation. Authors show the process of creating a recommender for a real situation. The process is described in detail from the data collection stage to the recommender definition.
In Chapter 10
, Brito et al.
present an architecture-centric approach for designing ERS where quality attributes are considered in a systematic manner. The underpinning of their approach is a simple process that touches two levels of software design: i) architectural design, which deals with quality attributes in a high-level way; and ii) detailed design, which deals with lower level modeling strategies for realizing the software architecture at the source code. Authors provide a case study that shows, in a step-by-step fashion, how the software architecture can be designed and gradually refined. The proposed approach relies on architectural styles and good practices for the structural definition of the system, and design patterns for the implementation of quality attributes related to ERS (e.g., adaptability, performance and reliability) by using object-oriented programming languages.Section 3
reflects the current situation in the field of ERS. The inclusion of this section is expected to motivate researchers to produce more user studies, as more real-world experiences are needed to understand how users perceive recommendation-based support in e-learning. One chapter was selected to illustrate this section.
In Chapter 11
presents a real-life application and discusses how recommenders can also be gamed in the context of e-learning. The author discusses his experiences of adding recommender features to additional reading materials listing page in an undergraduate-level course. The results reported show that students perceived the system as useful and did not resent compulsoriness, and that the perceived social presence promoted social behavior in many students. However, some misbehavior was also detected, as many students rated materials without viewing them, which points to the need for ERS to be robust against some students trying to get points without earning them. The lessons learned from this chapter may be applied in various e-learning settings. Moreover, this case study is a step in the direction of gathering and analyzing actual experiences of using recommender systems in e-learning to better augment the e-learning process on the learners’ terms and to create an engaging e-learning experience.
Finally, Section 4
includes a couple of papers that compile some of the existing challenges in the field, which cover two significant issues: 1) considering meta-cognitive functions in the recommendations process, and 2) taking advantage of the opportunities derived when the mobile is used for the learning process (i.e. in m-learning contexts).
In Chapter 12
, Zhou and Xu
analyze the challenges regarding the use of ERS to enhance meta-cognitive functioning in online learners. Meta-cognition plays a crucial role in the promotion of effective school learning, but in most of the e-learning environment designs, meta-cognitive strategies have generally been neglected. If the learning system were able to guide the student and intelligently recommend learning activities or strategies to facilitate monitoring and control of their own learning, it would favor and improve the learning process and performance. These authors identify the following five challenges: i) consider both learner attributes and the learning sequences during recommendations, ii) detect the learners’ motivation and social-emotion and their changes as the task unfolds and to inform what learning strategies to recommend, iii) detect meaningful learning actions and translate them into a strategy in order to evaluate the current learning strategy and subsequently formulate recommendations, iv) identify learning activities or strategies that are more complex with multiple features instead of recommending static learning resources and v) retrieve ratings or preferences to learning strategies/activities in advance by the system by considering learner profiles, learning goals, tasks, and contexts and adapt them according to the learner’s progress and performance at different learning stages. The authors propose to adopt data mining algorithms (i.e., content-based and sequence-based recommendation techniques) to meet the identified issues.
In Chapter 13
, Liu and Divitini
analyze the opportunities and challenges from context to provide suitable recommendation for mobile learning. Context, which can be used to characterize the situations of a learner in mobile learning environment, presents an opportunity to improve the learning efficiency of mobile learning. Recommendation approaches can provide appropriate recommendation by understanding the actual demands of learners with the help of context, which are 1) supporting new pedagogical theories, 2) satisfying changing requirements of learners, 3) tackling information overload, and 4) overcoming the shortcomings due to poor input and output capacity of current mobile devices. Context also features a challenge to recommendation approaches on input data, context modeling, algorithms, and evaluation such as 1) creating accurate learning context, 2) designing effective recommendation algorithms, 3) properly presenting recommendation results, and 4) including the context of a learner in the scope of evaluation methods. Based on this analysis from context, possible research directions and questions are identified. The application of context-based recommendation may facilitate the further development of mobile learning.
The above chapters compile practices and challenges in ERS. They intend to provide a valuable window from where to analyze the current status of the field. This handbook aims to contribute to advance the research and motivate further analysis and developments, which includes the execution of real world experiences with recommender systems in e-learning scenarios. Overall, the handbook attempts to provide alternative views on related subjects and to bring educators closer to using recommender systems to support the learning process and computer scientists developing algorithms to provide appropriate recommendation strategies.
Last but not least, we would like to deeply thank the more than 50 people that have been involved in the ERSAT project, as Editorial Advisory Board members, Program Committee members, or Additional Reviewers. They all have contributed their expertise to select the relevant contents for this handbook and have provided very valuable comments to improve the quality of the submissions received. Thanks to all of you!
Olga C. Santos and Jesus G. Boticario
aDeNu Research Group, UNED, Spain