Hershey, Pennsylvania
New York, New YorkBeijing, China
Overcoming Size and Subject Bias in Rankings: A Review of Various Trends and Bias in Data Commonly Used in Rankings and Methods to Overcome Them

Overcoming Size and Subject Bias in Rankings: A Review of Various Trends and Bias in Data Commonly Used in Rankings and Methods to Overcome Them

Simon Michael Pratt (Thomson Reuters, Canada)
Copyright: © 2017 |Pages: 11
DOI: 10.4018/978-1-5225-0819-9.ch014
OnDemand:
(Individual Chapters)
Available
$37.50
Current Special Offers
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

Although it is impossible to distill the complexity of a university down to a single ranking position that represents the multiple missions of a university or the different stakeholders of ranking user, rankings are here to stay. It is essential that rankings publishers are clear about their objectives and use techniques to overcome bias. This paper discusses in detail some of the trends and bias found in data used in rankings and methods that can be used to overcome bias. It also discusses the motivations behind rankings and the influence of institution size on ranking outcome.
Chapter Preview
Top

Introduction

This chapter reflects my thoughts and opinions that I have developed over the last five years working on various rankings, in particular the Times Higher Education World University Rankings, the U.S. News Best Global Universities Rankings and the Reuters Innovative Universities rankings, for all of which I was the lead production manager. Although I am confident in the results of my work I recognize that my methods are not the only way to create a meaningful ranking and that other people may have contrary opinions. I hope that my insights will have use. My past experiences are primarily concerned with the production of “World” rankings of universities and thus my knowledge is somewhat limited to that context. I have little experience with domestic league tables, but recognize that there are many valuable resources such as league tables and university guidebooks that may be of more use, particularly for potential students or parents of students, than world university rankings. Some of the resources that I use from time to time are the US Department of Education’s “College Navigator” (https://nces.ed.gov/collegenavigator/) which is a rich resource of factual information and statistics such as on campus crime rates, accreditations and varsity athletics teams. This information may help inform a decision about which university to study at. There are of course many other resources in the US and other countries. If you happen to be the parent of a student or a potential student yourself I strongly recommend that you look first to see what domestic resources might be available.

Rankings and University Missions

As discussed previously (Pratt, 2013), university rankings do not reflect the varying missions of a university. They do not take into account the different objectives of the stakeholders, and a single ranking position cannot represent the highly complex nature of universities. University rankings continue to be popular and therefore rankings publishers have an obligation to make their rankings robust and to be clear about the objectives of their rankings. In this chapter I will discuss the nature of bias in rankings and how the objective of the ranking publisher may influence their methodological choices. I will also discuss methods that can be applied to overcome bias that might be caused by a university’s size, subject mix, or geographical location.

Complete Chapter List

Search this Book:
Reset