This chapter examines the literature on evaluation methods within e-learning with respect to their applicability to evaluate e-portfolio systems within higher education as evaluation criteria for reviewing e-portfolio provisions do not currently exist in the literature. The approach taken is to initiate two extensive literature searches and reviews. The first search was undertaken in 2009 involved reviewing over 600 articles by abstract dating from 1995 to 2010 to develop evaluation criteria suitable for Blackboard LMS e-portfolio systems evaluation. The second search undertaken in 2013 involved extending the search criteria to include further terminology and databases and returned over 4107 articles, which were read by title and abstract dating from 2009 to 2013, in order to systematically map evaluation methods used within e-learning to assess their quality and applicability for evaluating e-portfolio systems. The implications of the research undertaken provide a starting-point for further research into the development of robust e-portfolio evaluation models and frameworks. The lack of evidence uncovered in the 2009 and 2013 searches of the literature justify the need for further research into the design, development, and testing of evaluation methods for the evaluation of e-portfolio systems.
Top1. Introduction
Not all improvements in technology equate to significant advances in educational outcomes as was the case with the introduction of electronic learning (e-learning) in relation to the implementation of educational policies and effective practice (Weaver, 2002). The institutional infrastructure of tertiary educational establishments provide the support needed for this new form of learning to function, as well as, access to technologies which can in many cases be beyond the control of the teacher and is dependent on a wider range of technological domain based services than courses that do not use e-Learning. The technological domain based services were identified by Deepwell (2007, p. 40) and comprise of five aspects which are governed by the institutions ICT department they are: (1) IT support; (2) Access (on and off site); (3) virtual learning environments (VLE) (third-party software can function within the VLE and is user-friendly); (4) software availability and expertise availability; and (5) licensing conditions. Further and higher educational institutions are now providing more support for this type of learning as well as more training for teaching staff who are required to engage with new teaching technologies.
With the UK government’s aim of every student in the UK having a ‘Personal Development Plan’ (PDP) e-portfolio as part of the criteria for the completion of a certificate, diploma, or higher degree (Burgess, 2007, p. 9) it is crucial that educators, institutions, and other stakeholders have evidence in relation to the performance of e-portfolios as a learning medium. This chapter reviews the available literature of e-portfolio success within education to: (1) identify which aspects of e-portfolios have been evaluated as a learning tool; (2) describe the evaluation strategies used; (3) synthesise the findings; (4) compare evaluation criteria and methods applicable for assessing LMS e-portfolio systems; and (5) discuss educational implications and future research directions.
The research has highlighted the need for an e-portfolio effective practice framework based on the findings evidenced in the evaluation of an e-portfolio project for a higher educational institution within the UK. As well as evaluating the existing ‘Learning Management System’ (LMS) from which the e-portfolio was situated the evaluation also involved building a Personal Development Planning (PDP) portfolio using the postgraduate templates provided by the institution, as well as linking the institutional PDP e-portfolio to external Web 2.0 social networking sites such as: Bebo, MySpace, and Wix. This evaluation report produced an LMS e-portfolio evaluation framework based on the findings of other studies during the period of 2009. The academic literature available on the subject area of e-portfolio systems evaluation was revisited in 2013 to identify if any effective practice frameworks have been developed and tested in the literature since the publication of our previous findings in 2009.
This study will be of interest to educational policy and programme developers, teaching staff, e-Learning and e-portfolio developers, faculty heads, Continuing Professional Development ‘CPD’ and Lifelong Learning partners as it will provide a means for assessing the quality of the user experience of the e-portfolio system in relation its effectiveness with respect to human-computer interaction. It is our intention that stakeholders will enhance and develop the framework further by building on our initial research findings and e-portfolio evaluation framework and report their findings to the wider academic community. The evaluation framework developed for this study was adapted from previous studies of e-learning best practices and virtual campuses review frameworks (ANSI, 2004; Cann, Ball and Sutherland, 2003; CATEA, 2006; Gallagher and Smith, 1989; Guilbert, 1984; Harrison, 1999; Hillesheim, 1998; Iahad, Dafoulas, Kalaizakis and Macaulay 2004; Mager, 1975; Marshall, 2005; Marshall and Mitchell, 2003; QAA, 2004; Ragan, 1999; Wright, 2003). With respect to contribution to knowledge within the areas of PDP and e-portfolio learning systems the adapted model was assembled for the purposes of improving monitoring and development processes to enhance the overall quality assurance of e-portfolios for supporting effective human-computer interaction and usability of e-portfolio systems.