Search:
Sort by:		Search Reset

Dr. Shalin Hai-Jew discusses the peer review process

The Art of Academic Peer Reviewing

By IGI Global on Sep 17, 2014

Dr. Shalin Hai-Jew is one of IGI Global's most prolific editors and contributors. She works as an instructional designer at Kansas State University (K-State) and teaches for WashingtonOnline (WAOL). She has taught at the university and college levels for many years, and reviews for several publications including Educause Quarterly and MERLOT’s Journal of Online Learning and Teaching. Dr. Hai-Jew recently took a minute to discuss the art and importance of the academic peer review with IGI Global Acquisitions Editor Kayla Wolfe.

IGI Global: Tell us a little about yourself. Why did you become interested in the topic of the academic peer review process in particular?

Dr. Hai-Jew: Let’s see. I’m an American-born academic, with decades of work teaching in higher education. For the past 8 years, I’ve been working as an instructional designer at Kansas State University, where I work with faculty to teach online. Some of my work involves research in educational technology and data analysis; other works go a bit farther afield.

Academic peer review has been something that has been part of my life since I was 15. I tested into the University of Washington when I was 14 and started studying as a full-time university student there a few months after. I started publishing poetry and other light works when I was 15, which meant going through academic peer review. So academic peer review has been part of my professional life now for many years. I’ve published broadly in a number of venues, not just academic ones. Then, too, I’ve worked as an editor on various publication projects. I’ve reviewed for various publishers and journals. I’ve been able to look at academic peer review from various angles.

I think when I first started writing and striving to publish, I looked at academic peer review as a more objective and hallowed process than it really is. As I’ve experienced more, I see it as a very human process with strengths and weaknesses. It’s not just about quality work but a whole lot of other practical considerations as well.

Recently, there have been several controversies surrounding the academic peer review process, academic fraud, and peer-review circles. What are your thoughts on these events?

Yes, I’ve been following these stories through the popular media with interest. By your question, I’m assuming that you mean the multiple cases of stem cell studies that were found to have involved some measure of fraud with multiple researchers. At about the same time, there were also stories about how 120 papers that were machine generated (using SCIgen) appeared in information technology journals and databases. When these issues came to the fore, many asked what controls were in place to detect potential research fraud and to ensure the quality of published research. After all, there are weighty decisions that may be made from research findings, particularly from findings in the health and medical sciences.

In certain parts of academia, there has always been a lot of pressure to perform. By “perform,” I mean to put out original and impactful research and to train talented researchers and contribute positively to society. In every field of academia, there are differing standards for research rigor and ways of knowing and understanding “truth”. There are gatekeepers who strive to ensure quality: these are the people who serve on committees to decide whose work should get funded, the editors who decide which works get published, the administrators who decide whom to hire, and so on.

Ironically enough, in the present day, we have more ways to tell whether something was fraudulently created. There is the application of statistical methods to identify anomalies in research trajectories…which may encourage closer attention. There are oversight committees (who were seated in response to federal laws on ethical research) that oversee pretty much all academic research (and who decide which types of research may be exempt from their purview). With the ease of accessing publications through electronic means, questionable research findings may be identified much more quickly—given that there are more eyes on a topic. People are competitive, and they are interested in making sure that their colleagues do not benefit from fraudulent claims and fraudulent deeds. (People who would lie on their electronic bios and resumes seem not to have gotten the memo about how easy it is to test assertions and find out deceits.)

On the other hand, people tend to behave in ways that align to their interests. Those grant funders, publishers, editors, and administrators want the research to be successful and accurate. They don’t want to believe that they are working with people who may be dishonest. They do the “trust” part without the “but verify”. The pressures to achieve apply to everyone. It’s sometimes too easy to let a question go by unanswered. Sometimes, it’s too easy to let slide on standards. Sometimes, it’s too easy to corner-cut and be tempted by “vanity” endeavors (whether that involves research or publishing or anything else).

The human tendencies towards cognitive biases are well established. It’s easier to think that one is surprisingly lucky and skilled to be working with super talented people than that one is being thwarted through others’ deception. To be an effective peer reviewer though, you have to be able to “go there” and imagine the full range of possibilities and to assess in effective ways. In game theory, a “dominated strategy” is an a priori losing hand. Making assumptions about the soundness of a work, prior to actually being thoroughly evaluated, is a given losing hand. In academic research (and elsewhere), people need to check themselves; they also need to check each other. The fallout from the events listed in your question is on-going. We have certainly seen people’s professional careers end for fraudulent research. That’s not a given in all circumstances. For some, their careers may be tarnished but not ended.

What steps do you believe can be taken to detect and prevent peer review fraud?

So in the above cases, the SCIgen one involved huge carelessness and no real oversight—for machine-written nonsense papers to make it into some very respected databases and publications. There was inordinate trust put into conference organizers who seemed to have the sole purpose of plumping up publications linked to conferences, without any sense of actual standards. I wonder if that occurred in part because of the acceptance of works written in English by non-native speakers without any effort to bring those works to full readability. I am all for accepting quality works from anywhere in the world, but the writing has to be fully fluent. In some of the technology journals I read, that correctness is not always assured—which means a lack of editorial work and likely a lack of actual copy-editing.

Those in academia are as hierarchical in thinking as the next person. They’ll put publications and publishers on a hierarchical ladder based on the big-name editors, the quality of publications, the acceptance rate (the lower the better), the name of the funding organizations, and other factors. The top-level journals have unwavering standards, strong leadership, tough editorial boards, and a long record of quality publications.

What can be done to prevent peer review fraud? One aspect of fraud may come about if there is no actual double-blind review, where people are aware of others’ identities…and reviewers are acting on their own subjectivities and alliances. Peer review fraud happens when the peer review doesn’t actually happen—such as when an editor loads up the evaluation team with confederates who do not actually review a work or provide any oversight or apply any actual rigor. It happens when a review board is seated who do not have the necessary expertise and education. So to head off academic peer review fraud, it’s critical to have oversight to the process. It’s important to have a rigorous applied process to surface concerns in a systematic way. It’s important to have members trained to look at issues of research method, statistical analysis, research documentation, logical reasoning, and other variables. It’s important to have reviewers who are open-minded and supportive but also critical. (You don’t want reviewers who close off to others simply because they’re acting on personal biases…which can happen at times, too.) The respectful assumption is that the researcher is an expert on the topic because only he or she or the team has spent that much time and effort exploring the issue. (This is a common assumption for doctoral students as they go in to their dissertation defense. In the same way, their committee generally has made sure that the literature review was thorough and correctly presented, that the research methodology was appropriate and properly followed, that the data was accurately attained and handled and recorded, that the analysis was logical and supportable, and so on. Unless there are identifiable problems, the core findings are generally accepted. A doctoral committee’s strength is in the combined and complementary expertise of the committee.) This is why a reviewer generally does not assess based on outcomes or issues beyond their own expertise.

Finally, I do think academic peer review is an imprecise and difficult process. There are limits to what is knowable. Much research is difficult or expensive to replicate. Often, such review is unpaid—and as such—people may be less willing to put their shoulder into it. People do not have the time or the inclination to run everything to ground.

What tips do you have for novice peer reviewers to ensure that they are properly reviewing manuscripts prior to publication?

I can’t speak to this generically because my experiences have been in education and educational technologies.

One thing I would say is that they should read broadly and continually learn to evolve their own knowledge and skill sets. They should not be afraid to call out any issue in any work (politely). There is no benefit in being silent or timid. The review process should be securely double-blind. The reviewers should not have any incentive to decide one way or another. They should be aware of their own cognitive biases—which is one of the toughest things to learn and then to train against. People just want to believe that they are infallible and that what they think to be true is truth. Reviewing well is an evolving process. I would also advise that novice peer reviewers expose themselves to a broad range of editors and publishers…and that they learn continually. The approach should always be respectful but candid.

As an author and a researcher, I can say categorically that I would much rather be called out and have my work ultimately strengthened than to be rushed into publication with all my native mistakes and shortcomings.

We understand that you have a new book in the works on the topic of academic peer reviewing. Please tell us more about that.

“Book” in this context would not be quite accurate. This is more of a light article. “The Art of Academic Peer Reviewing” is a small e-book on the open-source Scalar platform. (The Alliance for Networking Visual Culture, ANVC, created Scalar to enhance the amount of scholarly work applied to online digital repositories and collections.) This e-book integrates multimedia and includes multiple Q&A’s about various professors’ and researchers’ experiences with academic peer review. It’s a kind of evolving book. My initial motivation was to use this e-book project to learn the Scalar tool (to benefit a regional professional organization that I have been a member of for years). I needed a topic that I was familiar with but also that had some interest for others. I spend a lot of time learning new technologies (most way more complicated than Scalar), and it helps to have a project, and it sometimes helps to have the “threat” of an audience—so I learn a tool well.

How do you believe this new work will impact the academic community?

This new work will be one small way to foreground the issue for a short moment, nothing more. This is a light e-book with light research. What’s much more interesting about this work are the different cases with the different voices and perspectives, and with multimedia integrated. Another much more interesting aspect is Scalar itself, which is a fun and promising tool.

I think one important point that sometimes gets lost in the discussion of academic peer reviewing work is the importance not only of rigor but of nurturance. There are new researchers and authors coming online all the time, and they need support and encouragement. They need to experience a range of types of work. Dishonesty, malice, and/or a lack of willingness to improve are deal breakers for me, but if those are not apparent issues, then there has to be support to evolve a work to full quality (if possible). This work of a reviewer is not just about making demands.

Maybe one other impact is the realization that a published work is not perfect…ever, no matter how much work goes into it. An e-book is evolving and evolvable, which is one of the great benefits of this form on a flexible platform. I’m hoping that there are communications around the ideas in the text. The Scalar platform sure enables commenting on the various pages and artifacts. I’m still thinking of ways to evolve this book further.

Do you have any other projects in the works that you would like to mention?

I’m currently working on Design Strategies and Innovations in Multimedia Presentations, which is coming together very well.

Let me mention a book that is going to be published shortly. Enhancing Qualitative and Mixed Methods Research with Technology is slated for publication in September. This book means a lot to me because it’s a topic that has entranced me for many years—that mix of research with technology. Every book is very much a product of its time. Ideally, it would be the best that could be done given the limits of the moment.

Thank you for taking the time to speak with us, Dr. Shalin Hai-Jew, we wish you all the best as you continue to work on this new publication!

Dr. Shalin Hai-Jew's research has been published in dozens of IGI Global publications, dating back as far as 2009. Her newest book Enhancing Qualitative and Mixed Methods Research with Technology explores the integration of new digital tools into the research process. Including current information on data visualization, research design, information capture, and social media analysis, this publication serves as an ideal reference source for academicians, scientists, information specialists, business managers, and upper-level students involved in interdisciplinary research. This book is available in the Advances in Knowledge Acquisition, Transfer, and Management (AKATM) Book Series, a collection that brings together research on emerging technologies and their effect on information systems as well as the knowledge society. AKATM will provide researchers, students, practitioners, and industry leaders with research highlights surrounding the knowledge management discipline, including technology support issues and knowledge representation. For more information on Dr. Hai-Jew and her research, visit her IGI Global biography page.

Browse for more posts in:

Education Instructional Design Book Series Books & E-Books Chapters InfoSci-Books Events & Collaborations Presentations North America Resources for Instructors Resources for Librarians Resources for Researchers

No comments Comments

Be the first to comment!

More from IGI Global

Apr 18, 2024 Uniting Nations with Multinational Electronic Health Records to Stop Lassa Fever's Pandemic Threat!

The World Health Organization (WHO) has reported a surge in Lassa fever cases in Nigeria, emphasizing the urgent need for containment measures.

IGI GlobalRead More

Medical, Healthcare, and Life Sciences Healthcare Information Systems Books & E-Books Research Trends

Apr 16, 2024 IGI Global Announces the 2024 Journal Reviewers Award Winners

IGI Global congratulates the winners of this year's Journal Reviewer Award

IGI GlobalRead More

Journals Awards & Recognition Open Access

Apr 15, 2024 What Defines Authorship and Who Should Be Included in a Manuscript’s Byline?

For decades, academic publishing has been plagued with discrepancies surrounding authorship of scholarly research...

IGI GlobalRead More

Books & E-Books Acquisitions

Apr 15, 2024 Doody’s Selects IGI Global Books for Special Topics List

Two IGI Global publications have been recognized by Doody's for their excellence and niche topic focus.

IGI GlobalRead More

Medical, Healthcare, and Life Sciences Media and Communications Books & E-Books Awards & Recognition

Apr 11, 2024 Technology is Making Disparities Worse

Digital Inclusion Week underscored the urgent need for a national digital equity plan in the US due to disparities in internet access and digital skills.

IGI GlobalRead More

Apr 8, 2024 Nearly 90% of IGI Global’s Frontlist Books Now Indexed in Scopus

The majority of IGI Global's books Frontlist is now indexed by Scopus. Learn what this prestigious recognition means for the publisher and the experts behind these books.

IGI GlobalRead More

Books & E-Books Awards & Recognition Reviews & Indexing

Apr 4, 2024 Navigating Neglected Maintenance and Safety Risks of Flight Travel

A United Airlines Flight 837 bound for Tokyo from San Francisco could not take off due to a mechanical issue.

IGI GlobalRead More

Business and Management Computer Science and Information Technology Science & Engineering Hospitality, Travel & Tourism Management Aerospace Engineering Books & E-Books Research Trends

Apr 3, 2024 Introducing the New IGI Book Topic Brochures

IGI Global is excited to introduce a new initiative to provide detailed insights into various fascinating subjects. Our brochures offer a comprehensive overview of timely publications covering the latest research topics...

IGI GlobalRead More

Education Business and Management Computer Science and Information Technology Social Sciences and Humanities Books & E-Books

Apr 3, 2024 Interview with a Global Guru in Education

Hear from Dr. Velliaris, who was voted as a Top 30 Global Guru in Education.

Donna VelliarisRead More

Education Books & E-Books Interview Author News

Mar 29, 2024 Cocoa Prices Skyrocket

Cocoa prices surging to $10,000 per metric ton challenge chocolate manufacturers and consumers, risking supply chain stability and higher retail prices...

IGI GlobalRead More

Business and Management Environmental, Agricultural, and Physical Sciences Sustainable Development Books & E-Books Research Trends

First Previous 1 2 3 4 5 6 7 8 9 10 ... Next Last