Although policy evaluation has always been important, today there is a rising attention for policy evaluation in the public sector. In order to provide a solid base for the so-called evidence-based policy, valid en reliable data are needed to depict the performance of organisations within the public sector. Without a solid empirical base, one needs to be very careful with data mining in the public sector. When measuring performance, several unintended and negative effects can occur. In this chapter, the authors focus on a few common pitfalls that occur when measuring performance in the public sector. They also discuss possible strategies to prevent them by setting up and adjusting the right measurement systems for performance in the public sector. Data mining is about knowledge discovery. The question is: what do we want to know? What are the consequences of asking that question?
TopIntroduction
Policy aims at desired and foreseen effects. That is the very nature of policy. Policy needs to be evaluated, so that policy makers know if the specific policy measures indeed reach – and if so, how, how efficient or effective, with what unintended or unforeseen effects, etc. – these intended results and objectives. However, measuring policy effects is not without disadvantages. The policy evaluation process can cause side effects.
Evaluating policy implies making fundamental choices. It is not an easy exercise. Moreover, policy actors are aware of the methods with which their activities – their (implementation of) policy – will or could be evaluated. They can anticipate the evaluation, e.g. by changing the official policy goals – a crucial standard in the evaluation process – or by choosing only these goals that can be met and avoiding more ambitious goals that are more difficult to reach. In this context, policy actors behave strategically (Swanborn, 1999). In this chapter, we focus on these and other side effects of policy evaluation. However, we also want to bring them in a broader framework.
Within the public sector, as elsewhere, there is the need to have tools in order to dig through huge collections of data looking for previously unrecognized trends or patterns. Within the public sector, one often refer to “official data” (Brito & Malerba, 2003, 497). There too, knowledge and information are cornerstones of a (post-) modern society (Vandijck & Despontin, 1998). In this context data, mining is essential for the public sector. Data mining can be seen as part of the wider process of so called Knowledge Discovery in Databases (KDD). KDD is the process of distillation of information from raw data, while data mining is more specific and refers to the discovery of patterns in terms of classification, problem solving and knowledge engineering (Vandijck & Despontin, 1998).
However, before the actual data mining can be started, we need a solid empirical base. Only then the public sector has a valid and reliable governance tool (Bouckaert & Halligan, 2008). In general, the public sector is quit well documented. In recent decades, huge amounts of data and reports are being published on the output and management of the public sector in general. However, a stubborn problem is the gathering of data about the specific functioning of specific institutions within the broad public sector.
The use of data and data mining in the public sector is crucial in order to evaluate public programs and investments, for instance in crime, traffic, economic growth, social security, public health, law enforcement, integration programs of immigrants, cultural participation, etc.Thanks to the implementation of ICT, recording and storing transactional and substantive information is much easier. The possible applications of data mining in the public sector are quite divers: it can be used in policy implementation and evaluation, targeting of specific groups, customer-cantric public services, etc. (Gramatikov, 2003).
A major topic in data mining in the public sector is the handling of personal information. The use of such information balances between respect for the privacy, data integrity and data security on the one hand and maximising the available information for general policy purposes on the other (cf. Crossman, G., 2008). Intelligent data mining can provide a reduction of the societal uncertainty without endangering the privacy of citizens.