A Case Study in Data Mining for Automated Building of Teams

Introduction

Data mining of customer information is now widespread throughout the business world, however systematic data mining of employee information is less common. Thus, while the opinions of consumers have driven much of the effort to protect their data, the opinions of employees have often been ignored. The attitude of employees towards the use of their personal data is greatly affected by their perceptions of the company’s intentions. If the employer makes the effort to obtain permission from individual employees and carefully explain the goals of the data collection, negative perceptions may be reduced (Long & Troutt, 2003). Even when legal restrictions are observed, data mining within one’s own organization can have far-reaching social and ethical implications. Any process which can be considered invasive to personal data requires sensitivity to all stakeholders, including both the employees and the company itself (Saban, 2001).

This chapter highlights the approach taken by one company in gathering, storing, and using employee data to automatically staff teams for new projects. The company described herein is a privately owned research and development firm with approximately 100 employees. The majority of the staff is comprised of scientific researchers with graduate degrees in psychology, cognitive science, human-system engineering, modeling and simulation, and computer science.

Much of the business performed by the company is supported by dozens of small government contracts obtained by responding to quarterly requests for proposals (RFPs) and winning follow-on work. Currently, teams for writing the proposals and carrying out the project work are created by word of mouth. This chapter focuses on one such project, TeamBuilder, conducting research into the science of building teams automatically from employee data. The goal of the project is to make the process of staffing teams more efficient and the projects more successful by automatically finding people that both have the requisite skills and would work best together as a team. It was decided by the TeamBuilder research team to use their own company as a test bed, due to familiarity with team processes within the organization and availability of data. It was determined early in the project that, in order to establish whether each given candidate possesses the necessary skills and abilities that are required for a specific team, large amounts of personal data would be needed.

Using employee data for any purpose has both drawbacks and benefits. Clearly, the employer would like to use all available information about employees that would make them more efficient, productive and successful. A well-conducted data analyses could also help build better relationships between employees and the organization, increase the opportunities and choices available to individuals, and in general help build a better understanding of the organization as a whole. Additionally, automated interpretation of the data could help management reach conclusions beyond the ability of human analysis and avoid inefficient word-of-mouth processes. On the other hand, there is the ever-present concern about invasion of privacy. Exploiting personal information also brings the risks of false conclusions being drawn, abuse of information to the detriment of individuals or organizations, and misapplication of erroneous data. These risks must be weighed against the benefits (Cook & Cook, 2003).

The organization of the chapter is as follows. We will first discuss the scientific and theoretical issues of staffing teams that are the basis for the TeamBuilder project. Then we will describe how we gathered the data for our case study while addressing privacy, ethical, legal and security concerns of the organization (both employees and management). Because the goal of the project is to create teams automatically, the integrity of the data must be high in order to obtain the trust of both management for using the tool and employees for understanding their assignments. We therefore describe a number of processes undertaken to ensure data integrity. Finally, we briefly discuss some of the computational methods we use to operationally define team theoretical constructs and employee abilities, in order to automatically staff teams with people who both have the skills to do the task and will work well together. There are issues with applying any automated process to assess human behavior and we provide some details of how we mitigate these concerns.