Citation Based Indicators in Evaluation of Quality and Performance of Research and Researchers

Citation Based Indicators in Evaluation of Quality and Performance of Research and Researchers

Vahideh Zarea Gavgani (Tabriz University of Medical Sciences, Iran) and Fahime Abbasi (Tabriz University of Medical Sciences, Iran)
Copyright: © 2015 |Pages: 10
DOI: 10.4018/978-1-4666-5888-2.ch644
OnDemand PDF Download:
List Price: $37.50

Chapter Preview



Scientific citation to the paper is one of the indicators for identifying the rank/impact of the article among other publications in its field, as of the most significant criteria for assessment of scientific productions. Citation analysis is the bibliometric measures to assess the scientific literature based on citation count (Ziman, 1968). Citation analysis can help to determine the basic and fundamental production of a scientific field, identifying effective ideas and works by tracking published materials and analyzing the correlation between the citations.

A citation occurs when a published work ‘cites’ or directly refers to another published work, including the full reference of the latter within a reference list. These citations enable authors to acknowledge their intellectual debts (Van Noorden, 2010). Although citations are normally considered to originate in a journal article, they can also be found in so-called ‘gray literature’ including books, government publications, professional body documents, MSc and PhD theses, web articles, podcasts, newspapers and magazine articles. The presence and number of citations are frequently used to assess the influence of a particular article, author, journal or field of research. While it is acknowledged that the number of citations do not necessarily correlate with article quality, nevertheless a high number of citations for a particular article is suggestive of utility by other researchers and as such is an example of an academic impact measure (Nightingale & Marshall, 2012).

A wide range of research metrics (objective indicators) are now available to quantify notions of academic impact, profile and scientific quality, yet researchers from a range of disciplines are increasingly questioning the validity and reliability of these analytical tools (Van Noorden, 2010).

Citation analysis is an integral component of journal ranking criteria and is increasingly used to assess the impact of individual researchers and their institutions. It is important to have an understanding of how citation behaviors can be influenced. Seglen, an expert in information science has studied these behaviors and identified that higher citation rates are consistent with:

  • Articles in the English language.

  • Generalist areas rather than specific applied disciplines.

  • Review articles rather than original research.

  • Cutting edge articles with a short lifespan.

  • Longer rather than shorter articles.

  • Articles regarding established rather than emerging disciplines.

  • ISI-indexed journals (Seglen, 1997).

However, citation analysis is not a perfect tool for assessing the quality of research. Citation analysis depends not on the quality of research but also on the scientific articles and citation behavior. Mooij asserted that citation analysis leads the researchers with poor researches to withdraw from doing research or to strengthen their work and change to good researchers, because poor researches receive no citation (Mooij, 2006).

Key Terms in this Chapter

Science Citation Index (SCI): See ISI.

Discipline Impact Factor (DIF): DIF is similar to the impact factor, which measures the average number of times a paper in given journal is cited, except that the DID measures the number of times a paper in a journal is cited in core literature of the given discipline (Hirst, 1978).

Altmetrics: New metrics proposed as an alternative to the widely used journal impact factor and personal citation indices like the h-index. The term altmetrics was proposed in 2010 (“Altmetrics”, 2013).

H-Index: An index that attempts to measure both the productivity and impact of the published work of a scientist or scholar. The index is based on the set of the scientist's most cited papers and the number of citations that they have received in other publications (“H-index”, 2013).

Web of Science (WoS): See ISI.

Crown Indicator: The indicator aims to normalize citation counts for differences among fields (Waltman et al., 2010).

Immediacy Index: A measure of how topical and urgent work published in a scientific journal is. Along with the better known impact factor measure, it is a calculated each year by the Institute for Scientific Information (ISI) for those journals which it indexes; both impact factors and immediacy indices are published annually in the Journal Citation Reports (“Immediacy Index”, 2013).

Cite: To quote or refer to an authority outside oneself, usually in support of a point or conclusion or by way of explanation or example (Reitz, 2004).

G-Index: An index for quantifying scientific productivity based on publication record. It was suggested in 2006 by Leo Egghe. The index is calculated based on the distribution of citations received by a given researcher's publications (“G-index”, 2013).

Institute for Scientific Information (ISI): The Institute for Scientific Information (ISI) was founded by Eugene Garfield in 1960. It was acquired by Thomson Scientific & Healthcare in 1992, became known as Thomson ISI and now is part of the Intellectual Property & Science business of Thomson Reuters. ISI offered bibliographic database services. Its specialty: citation indexing and analysis, a field pioneered by Garfield. It maintains citation databases covering thousands of academic journals, including a continuation of its longtime print-based indexing service the Science Citation Index (SCI), as well as the Social Sciences Citation Index (SSCI), and the Arts and Humanities Citation Index (AHCI). All of these are available via ISI's Web of Knowledge database service (“Institute for Scientific Information”, 2013).

Literature Obsolescence: See Cited half-life.

Mathew Value: The term “Matthew effect” was introduced by Robert K. Merton .He described a psychosocial mechanism that led to misallocation of credit in the reward system of science (Pislyakov & Dyachenko, 2010).

Impact Factor: A quantitiative measure of the frequency with which the “average article” published in a given scholarly journal has been cited in a particular year or period, developed by the Institute for Scientific Information (ISI) for use in Journal Citation Reports, a multidisciplinary tool for ranking, evaluating, and comparing journals within subject categories. The indicator is used by serials librarians in collection management, journal publishers in marketing, information analysts in bibliometric research, and authors to identify journals in which to publish. Caution is advised in using the indicator as a measure a journal's prestige for purposes of academic evaluation for tenure or promotion (Reitz, 2004).

Research Assessment Exercise (RAE): An exercise undertaken approximately every 5 years on behalf of the four UK higher education funding councils to evaluate the quality of research undertaken by British higher education institutions (“Research Assessment Exercise”, 2013).

Citation Analysis: A bibliometric technique in which works cited in publications are examined to determine patterns of scholarly communication, for example, the comparative importance of books versus journals, or of current versus retrospective sources, in one or more academic disciplines. The citations in student research papers, theses, and dissertations are also examined by librarians for purposes of collection evaluation and development (Reitz, 2004).

Scientometrics: The study of measuring and analysing science research. In practice, scientometrics is often done using bibliometrics which is a measurement of the impact of (scientific) publications. Modern scientometrics is mostly based on the work of Derek J. de Solla Price and Eugene Garfield. The latter founded the Institute for Scientific Information which is heavily used for scientometric analysis. Methods of research include qualitative, quantitative and computational approaches (“Scientometrics”, 2013).

Cited Half-life: A measure developed by the Institute for Scientific Information (ISI) of the number of publication years from the current year which account for 50% of the current citations received by a journal (Reitz, 2004).

Complete Chapter List

Search this Book: