Journal Rankings: Comparing Reputation, Citation and Acceptance Rates

Journal Rankings: Comparing Reputation, Citation and Acceptance Rates

E. Susanna Cahn (Department of Management & Management Science, Pace University, Pleasantville, NY, USA)
DOI: 10.4018/ijisss.2014100106
OnDemand PDF Download:
No Current Special Offers


Research productivity is important in school reputation as well as individual faculty evaluation. In order to evaluate research productivity, the quality of research is often measured by proxy through the number of journal articles and ratings of the journals in which they appear. Because of this there is significant pressure on faculty to publish in the “top journals”. There are several metrics for evaluating and ranking journals, each of them with its own merits and limitations. Some commonly used quantitative measures of research quality are citation analyses, acceptance rates, and whether or not a journal is peer reviewed. Alternatively, journals can be ranked qualitatively into stratified groups based on reputation. Reputation, in turn, may be correlated with perceived values of quantitative measures, and thus is more subjective. The purpose of this research is to examine the extent of correlation between various measures of journal quality, in particular between quantitative and qualitative measures. The various measures are compared to examine the extent to which they are similar. Comparisons were also made among business departments. For this sample, overall journal rank was correlated with citation rate but not with acceptance rate. However, quantitative measures were not consistent among academic departments, indicating that journal rank can not be reliably used to make interdepartmental comparisons.
Article Preview

2. Measures Of Quality Scholarship

2.1. Citation Rate

Several measures of publication quality are studied in the journal ranking literature, each with its own strengths and shortcomings. Mainly, the measures fall into three categories: those based on citations, those based on acceptance rates, and those based on rankings of journals by professionals with some relevant expertise. Each is a proxy measure for journal quality; each is used to develop ordinal rankings or groups of “top” journals. Journal rank may in turn be used as a proxy for article quality, or author quality, or institution quality. How well does each type of measure reflect value? Are they consistent?

Citation is an indication that one scholar’s work is valuable to someone else. More citations should indicate higher value. MacRoberts & MacRoberts (1989) review some problems of citation analysis. Different rates of citation are customary in different fields. Multiple authors of an article are not always credited. There may be biased citing particularly when secondary sources are cited rather than initiators. Informal sources and tacit knowledge are usually not cited because they are not in papers and books. There is self-citation. Some citations are negative while others are positive. Not all literature is included in the SSCI which is the main source of citation impact factors. New journals and journals in specialized areas have fewer citations than general purpose and older, better known journals. When analyzing citation patterns, different authors use different length publication year windows for impact factors, making comparisons difficult. The most important papers are often not cited because their results are taken for granted to be part of the norm (Ritzberger, 2008); his example, users of the Nash equilibrium rarely cite Nash (1950). New developments are often in new journals which take a long time to get into citation indexes. The peer review system of journals is biased against authors that are not affiliated with top universities or are employed at non-academic institutions (Blank, 1991). Anthony F. J. Van Raan (2005) discusses some bibliometric problems of using citation index as a proxy for research quality, enumerating various possible sources of error in collection of the data used to compute a citation index. Citation is a complex process, practice differs among fields; he suggests citation is improper for evaluating research even aggregated at the level of large institutions. He advocates for improving bibliometric indicators and using them as support for, rather than replacement of, a peer-based evaluation procedure.

Citation rate can be computed for a journal as well as for an individual paper. Thus, citation rates are sometimes used in studies that compare journal quality and article quality.

Complete Article List

Search this Journal:
Open Access Articles
Volume 14: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 13: 4 Issues (2021)
Volume 12: 4 Issues (2020)
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing