Development of a Computerized Adaptive Test for a Large-Scale Testing Program: A Case Study of the Development of the North Carolina Computerized Adaptive Testing System

Abstract

In 1997, the Individuals with Disabilities Education Act (IDEA) amendments stated that “children with disabilities must be included in general state-and district-wide assessment programs, with appropriate accommodations, where necessary.” Where accommodations alone could not make the testing program accessible, the amendment required that the agencies develop alternative assessments so that every child would be included in the accountability programs. In response to the IDEA Amendments, the North Carolina Department of Public Instruction (NCDPI) reviewed its testing program and identified areas where additional accommodations were necessary. Based on this review, a research plan was developed for a computerized adaptive testing (CAT) accommodation, where computers and the Internet are used for administration of an adaptive test. This chapter explores several of the issues and lessons learned in the development of a computerized adaptive test in a case study of the design, development, and delivery of such a test in a large-scale testing environment. Findings from the case study have implications for item-pool development, curriculum alignment, and comparability to the paper and pencil tests, scoring and scaling of the computerized adaptive test accommodation, test reliability, validity, programming for the computer-adaptive test, and state and local technology infrastructure. The chapter concludes with lessons learned and future directions.