Test and Evaluation for Weapon Systems: Concepts and Processes

Introduction

Decisions made for weapon systems acquisition are hard in nature due to high technology and costs. Number of decision variables in weapon systems increases the complexity in acquisition since holistic evaluation is not possible with human brain. One way of taming the complexity dragon is to divide and conquer: portioning the acquisition problem into manageable pieces and solving each piece of problem using analytical techniques. Operations Research/Management Science (OR/MS) provide many analytical methods and techniques to solve complex decision problems.

In today’s world where countries tend to cut defense budgets, weapon system acquisition decisions turn out to be more compelling since one need to meet defense capability requirements with minimum cost. Hence, among various alternatives having different costs, determining which particular system meets capability requirements requires thorough evaluation of system alternatives.

There is more than one way of procuring a weapon system. One weapon system may be developed by a country itself while another system might be procured as Commercial Off-The Shelf (COTS). Without loss of generality, we can classify types of weapon system acquisition as;

• •

  Non-Developmental Item (NDI) or COTS (Developed or produced by another country)

• •

  NDI-COTS with some custom modifications

• •

  Co-developing with another country

• •

  Another country’s development and co-production

• •

  Country’s full development and production

Regardless of the way of procurement, the main questions to be answered remain to be the same: “How much a particular system alternative meets mission or capability requirements?” and “Which weapon system is superior in terms of meeting capability requirements?”.

Acquisition of NDI or COTS weapon systems involves some risks thereby needs rigorous evaluation. Most of the NDI/COTS systems are developed based on developing country’s needs, threat evaluation and capability requirements. These systems might be designed for different doctrine, user skills and combat environment. Moreover, they may not be compatible and interoperable with buyer’s systems that are already in inventory. Even though custom modifications might be considered to overcome aforementioned problems, these modifications may turn out to be infeasible and unaffordable.

Developing and producing a weapon system without another country’s involvement poses different challenges. At each phase of the development, the system needs to be evaluated to verify that it meets designated system specifications. Even though a holistic evaluation after all development process completed seems to be plausible and practical, this might generate greater problems since it may not be possible to solve or fix design problems, which are caused by early design flaws, after the system becomes a COTS.

The need for evaluating weapon systems is not only limited to acquisition process. For instance, in order to provide input to new concept development process, we may desire to measure performance of a weapon system that is already in inventory for some particular operational environment. We may also wish to verify weapon system performance after some years of service. This evaluation may help us to decide whether to put particular system out of service or to continue using it. In another case, it might be required to compare two or multiple in-service weapon systems’ performances for operational planning. This evaluation provides insight for weapon system allocation and new operational tactic development.

All these aforementioned needs for observing, verifying or measuring weapon system performance requires extensive test and evaluation planning and implementation. A test can be defined as a procedure to obtain, verify, or provide data for determining performance, capability, suitability, vulnerability, and lethality of systems in terms of meeting stated objectives. Evaluation corresponds to assembling, analyzing and comparing qualitative or quantitative data obtained from the test process to provide support to decision makers. In the context of weapon system analysis, the combined process of test and evaluation is a process where a system or components are tested and related data are evaluated in order to compare them against requirements and specifications or alternative system solutions.

There are several test and evaluation types defined in the literature, however, all test and evaluation procedures can be classified in two main categories;