Abstract
Accountability pressures on higher education have increased in recent decades (Dill, 1999; Garvin, 2000; Jacob & Hellström, 2003). Pressure for financial accountability stacks up alongside pressures to assess student learning outcomes. Student satisfaction has always been a factor in institutional success, but added mobility and growth in the for-profit educational market have increased the impact of student satisfaction. Further, citizens, parents of students, alumni, taxpayers, and, for the for-profit institutions, shareholders constitute powerful external forces. Online programs sit in a special spot in these pressures. As relatively new entrants in the educational repertoire, questions about quality of instruction and learning share center stage with questions about costs (The Learning Alliance, 2004). Online programs require an investment in technology, staff, course development, and marketing. Many campus stakeholders doubt that such investments will produce high-quality instruction and significant student learning. Even some early adopters are now starting to question the role of online learning (The Learning Alliance, 2004).
TopIntroduction
Accountability pressures on higher education have increased in recent decades (Dill, 1999; Garvin, 2000; Jacob & Hellström, 2003). Pressure for financial accountability stacks up alongside pressures to assess student learning outcomes. Student satisfaction has always been a factor in institutional success, but added mobility and growth in the for-profit educational market have increased the impact of student satisfaction. Further, citizens, parents of students, alumni, taxpayers, and, for the for-profit institutions, shareholders constitute powerful external forces.
Online programs sit in a special spot in these pressures. As relatively new entrants in the educational repertoire, questions about quality of instruction and learning share center stage with questions about costs (The Learning Alliance, 2004). Online programs require an investment in technology, staff, course development, and marketing. Many campus stakeholders doubt that such investments will produce high-quality instruction and significant student learning. Even some early adopters are now starting to question the role of online learning (The Learning Alliance, 2004).
A recent Sloan-C™ survey of chief academic affairs officers at degree-granting institutions in the U.S. demonstrates both the divide and the changing nature of the debate on the quality of online education (Allen & Seaman, 2003). While only 12.3% of respondents feel that online education is currently superior to face-to-face education, 44.9% believe that learning outcomes are the same. The remainder, 42.8% or almost half, believes that online learning outcomes are currently inferior to face-to-face learning outcomes. However, the percentage of those who believe that online learning outcomes will be inferior to face-to-face outcomes in three years drops to 25.2%, “a sea change in the perception of the quality of online learning” (Allen & Seaman, 2003, p. 13). Interestingly, in spite of the differences in perceptions of quality, Allen and Seaman report that a total of 66.8% of all respondents state that online education is critical to long-term strategy. The great majority (85.7%) of public institutions believe that online learning is a critical element of long-term strategy.
Given the accountability pressures and the long-term strategic importance of online learning, how should institutions of higher education evaluate online programs? A robust framework seems necessary to handle the variety of interests affecting online learning. The Sloan-C™ framework of five elements or pillars of quality (Moore, 2002) is arguably the best-known framework, and its comprehensive nature lends itself to robust evaluation. With several years of use and study, the Sloan-C™ framework has what is probably the best researched as well. Yet, other frameworks exist, not the least of which is that posed by post-secondary accrediting bodies (North Central Association Commission on Institutions of Higher Education, 2000). The Pew project on redesigning courses has also provided important insight into what makes for quality teaching and learning (Twigg, 2004).
A robust, yet comprehensible evaluation methodology that acknowledges the multiple perspectives held by many different stakeholders is also needed to encompass the complex tapestry of online learning within the wide variety of institutions and consortia involved in online learning. The Balanced Scorecard (BSC), first described by Kaplan and Norton (1992), has the potential to be just such a methodology. A strategic management tool organized around four distinct perspectives (financial, customer, internal, and innovation and learning), the BSC provides a balance “between short- and long-term objectives, between financial and non-financial measures, between lagging and leading indicators, and between external and internal performance perspectives” (Kaplan & Norton, 1996, p. viii). First developed for for-profit organizations, the BSC has since become a tool for government and not-for-profit organizations as well (Rohm, n.d.).
Key Terms in this Chapter
Perspective: The combination of interests, values, and viewpoints reflected in a Balanced Scorecard (BSC). The sum of the different perspectives in a given BSC should reflect all of the critical stakeholder views of an organization.
Distance Education: “A formal educational process in which the majority of the instruction occurs when student and instructor are not in the same place. Instruction may be synchronous or asynchronous” (North Central Association Commission on Institutions of Higher Education, 2000).
Accountability: Being called to account or held responsible for discharging higher education mission and goals, generally understood in terms of higher education’s responsibility to ensure student learning at a reasonable cost.
Online Program: That portion of distance education offered over the Internet by an accredited educational institution.
Balanced Scorecard (BSC): A tool that reflects “a carefully selected set of measures derived from an organization’s strategy” (Niven, 2002, p. 12). These measures reflect the four to five perspectives that an organization judges to be critical to achieve its mission and strategic objectives.
Pillar: As used in the Sloan-C™ quality framework for online learning, a pillar is a base on which quality online learning rests. The five pillars are: learning effectiveness, cost effectiveness, access, student satisfaction, and faculty satisfaction.