Tracer Studies: A Concrete Approach to a Virtual Challenge

Introduction

The purpose of this chapter is to introduce an evaluation methodology called Tracer Studies, a relatively straightforward qualitative technique that can be used to follow (or trace) the spread of information from its original source through its full spectrum of impact, determining at every step of the way who used the information, how they used it, and with what effect. We will illustrate the utility of Tracer Studies in the evaluation of virtual communities of practice (VCoPs) and demonstrate how Tracer Studies can address the three most important questions of funders and sponsors of such communities:

• •

  What knowledge do VCoP members gain from their participation?

• •

  In what ways do members use their knowledge?

• •

  How and with whom do they share knowledge beyond the VCOP?

The chapter contains two historical discussions. The first follows the evolution and previous use of Tracer Studies. Then, in order to situate Tracer Studies appropriately, we examine the nature of the VCoP phenomenon. Based on our review of the literature, we make the argument that a VCoP is a dynamic, continuously evolving entity that requires an evaluation approach that focuses on process and outcomes rather than characteristics and structure. The specific goals of the chapter are to:

• •

  Define Tracer Study methodology and show how the Tracer Study framework translates into an evaluation plan;

• •

  Explicate the unique nature of VCoPs; and

• •

  Demonstrate the application of Tracer Studies to a specific VCoP evaluation and the value of the results.

Background And Previous Use Of Tracer Studies

Tracer Studies were developed at Abt Associates (Cambridge, MA) as part of a federally funded study of the dissemination and use of research findings in education (Louis et al. 1985). Tracer Studies were intended to accomplish several goals:

• •

  To understand how knowledge is exchanged and used;

• •

  To explore the degree to which different strategies result in more or less dissemination of knowledge beyond the direct recipients;

• •

  To understand the outcomes and use of knowledge exchange events; and,

• •

  To determine the factors that affect how knowledge is exchanged and used.

Key to understanding Tracer Studies is the notion of a “knowledge exchange event”. According to Louis et al., (1984), such an event occurs when a set of purposively prepared information is communicated to a set of recipients. A knowledge exchange event includes the message (the information itself), a sender, a receiver, a strategy or channel for communicating (e.g. workshops, written products, phone meetings), and a social context for processing the information. All of these are important to the outcome of the knowledge exchange process, which is use. Translating this framework into an evaluation plan involves, determining the message or knowledge to be traced, selecting one or a number of dissemination events, and conducting interviews with a sample of primary and secondary recipients of the information. Ideally, the flow of information is traced from original recipients of information to others with whom information is shared. A Tracer Study follows the message through as many levels within and beyond the original recipients as possible.