An e-Learning Collaborative Filtering Approach to Suggest Problems to Solve in Programming Online Judges

An e-Learning Collaborative Filtering Approach to Suggest Problems to Solve in Programming Online Judges

Raciel Yera Toledo, Yailé Caballero Mota
Copyright: © 2014 |Pages: 15
DOI: 10.4018/ijdet.2014040103
(Individual Articles)
No Current Special Offers


The paper proposes a recommender system approach to cover online judge's domains. Online judges are e-learning tools that support the automatic evaluation of programming tasks done by individual users, and for this reason they are usually used for training students in programming contest and for supporting basic programming teachings. The proposal pretends to suggest problems assuming that a user must try to solve those problems already successfully solved by similar users. With this goal, the authors adopt the traditional collaborative filtering method with a new similarity measure adapted to the current domain, and the authors propose several transformations in the user-problem matrix to incorporate specific online judge's information. The authors evaluate the effect of the matrix configurations using Precision and Recall metrics, getting better results comparing with the authors method without transformations and with a representative state-of-art approach. Finally, the authors outline possible extensions to the current work.
Article Preview


Programming online judges are applications that automate the process of compiling and testing the solutions to problems that the same tool contains. Each year the presence in the Web of these kinds of applications increases. Worth of note are the ACM-ICPC1 Live Archive (, whose only purpose is the integration of the UVa in an effective educational environment. Also in the University of Informatics Sciences (UCI), in Cuba, there had been some successful experiences focused on the integration of online judges into undergraduate courses programs.

As web applications, online judges are subject to the information overload associated to the World Wide Web in the last few years. The data maintained by the aforementioned applications include a great number of problems and registered users, and lots of code in the submitted solutions, both correct and erroneous. In the UCI intranet currently reside two online judges, containing 850+ problems, 1800+ users, and tens of thousands of solutions submitted, and growing. This amount of data causes that users usually are not able to select the appropriate problem to solve according to their profile.

A classical solution to this information overload is the development of recommender systems (Resnick & Varian, 1997) focused on specific domains. In this paper, we propose the application of traditional recommendation techniques to the new online judge domain, to obtain an online judge recommender system. Firstly, we make a brief overview of online judges, specifying their main functionalities and purposes. After this, we justify the novelty of the online judge recommender scenario over other current state-of-art e-learning recommenders. Then, we develop a new approach to recommend problems to solve in this scenario using their specific data, and we design an experiment to measure its accuracy against baseline methods. Finally, we show our conclusions and the future works.

Complete Article List

Search this Journal:
Volume 22: 1 Issue (2024): Forthcoming, Available for Pre-Order
Volume 21: 2 Issues (2023)
Volume 20: 4 Issues (2022): 1 Released, 3 Forthcoming
Volume 19: 4 Issues (2021)
Volume 18: 4 Issues (2020)
Volume 17: 4 Issues (2019)
Volume 16: 4 Issues (2018)
Volume 15: 4 Issues (2017)
Volume 14: 4 Issues (2016)
Volume 13: 4 Issues (2015)
Volume 12: 4 Issues (2014)
Volume 11: 4 Issues (2013)
Volume 10: 4 Issues (2012)
Volume 9: 4 Issues (2011)
Volume 8: 4 Issues (2010)
Volume 7: 4 Issues (2009)
Volume 6: 4 Issues (2008)
Volume 5: 4 Issues (2007)
Volume 4: 4 Issues (2006)
Volume 3: 4 Issues (2005)
Volume 2: 4 Issues (2004)
Volume 1: 4 Issues (2003)
View Complete Journal Contents Listing