Article Preview
TopIntroduction
How does one evaluate the “worth” and the influence of a scholar? This is an important question for all academic stakeholders. Promotion and tenure committees need broadly applicable evaluation criteria allowing them to compare the work and influence of scholars across multiple disciplines. One commonly used evaluative criterion is the number of publications achieved in venues considered to publish only the best quality scholarship over a given period of time – the publication count. Publication counting provides simple-to-compute metrics, but the technique requires making value assumptions that are problematic and citation counting has been extensively criticized as biased. The choice, the ranking, and the weighting of the publication (journal) venue is considered problematic for many reasons (Chua, Cao, Cousins, & Straub, 2002; Walstrom, Hardgrave, & Wilson, 1995). Five critiques of the concept are discussed more fully in the following literature review. Moreover, the presumption that top journals always identify and publish “top” articles or that influential and important articles do not appear in other non premier publication venues has been challenged (Singh, Haddad, & Chow, 2007).
Recent research argues that rather than relying on a single simplistic metric (such as a raw publication count), a scholar’s ability is better measured in terms of a profile of metrics that assess the uptake of the scholar’s ideas in a field. The Hirsch family of statistics is argued to provide a fairer and more balanced way of assessing scholarly ability (Truex et al., 2009). Use of these metrics allows comparisons across different fields of study and provides a measure of both productive output and impact. In effect, these metrics measure the ideational impact of the scholar, i.e., the impact of their ideas on the field.
This paper extends the aforementioned research program by proposing the addition of another profile of statistics to the composite basket of scholarly evaluation tools. This approach is one that assesses the networks of a scholar’s social influence in the field. We argue that as a scholar interacts with other scholars, his/her ideas are shared with others and molded by these interactions. Thus his/her influence grows through the sharing and joint development of ideas in networks of discursive exchanges, interactions, and transactions called co-authoring. For this we use Social Network Analysis (SNA) on a co-author network, an approach also utilized by Vidgen, Hannenberg, and Naude (2007) and Xu and Chau (2006).
By adding SNA to the concept of scholarly influence, we are now examining the degree and frequency of connectedness between persons with whom a scholar works, creates, and publishes her scholarly work. In combining the H-family of bibliometrics with SNA we are providing ways to examine two parts of an extended discourse we call scholarship. That is, if one considers the inclusion of citations as an indication of ‘listening’, or acknowledging that a party to the discourse as having been heard, and one also considers the patterns, strength, and centrality measures of the networks in which ideas are created and published to be an indication of ‘who is talking with whom’ in that discourse, then we have a fuller picture of the nature of the construct called ‘scholarly influence.’
The objective of this research is to examine how SNA may be used to evaluate scholars. The research questions that we are investigating are therefore:
RQ1: How can social networks and network components be used meaningfully to compare and evaluate scholars?
RQ2: What are the key structural characteristics of a social network?
RQ3: How do the SNA and H-family metrics jointly provide a clearer picture of the construct ‘scholarly influence’?