Data Preparation for Big Data Analytics: Methods and Experiences

Context

Know-how about the production process is crucial, especially in case the production facility reaches an unexpected operation mode such as a critical situation. When the production facility is about to reach a critical state, the amount of information (so called shower of alarms) can be overwhelming for the facility operator, eventually leading to loss of control, production outage and defects in the production facility. This is not only expensive for the manufacturer but can also be a threat to humans and the environment. Therefore, it is important to support the facility operator in a critical situation with an assistant system using real-time analytics and ad-hoc decision support.

The objective of the BMBF-funded research project “Frühzeitige Erkennung und Entscheidungsunterstützung für kritische Situationen im Produktionsumfeld”¹ (short FEE) is to detect critical situations in production environments as early as possible and to support the facility operator with a warning or even a recommendation how to handle this particular situation. This enables the operator to act proactively, i.e., before the alarm happens, instead of just reacting to alarms.

The consortium of the FEE project consists of several partners, including application partners from the chemical industry. These partners provide use cases for the project and background knowledge about the production process, which is important for designing analytical methods. The available data was collected in a petrochemical plant over many years and includes a variety of data from different sources such as sensor data, alarm logs, engineering- and asset data, data from the process-information-management-system as well as unstructured data extracted from operation journals and operation instructions (see Figure 1). Thus, the dataset consists of various different document types. Unstructured / textual data is included as part of the operation instructions and operation journals. Knowledge about the process dependencies is provided as a part of cause-effect-tables. Information about the production facility is included in form of flow process charts. Furthermore, there is information about alarm logs and sensor values coming directly from the processing line.

Figure 1.

In the FEE project, various data sources from a petrochemical plant are preprocessed and consolidated in a big data analytics platform in order to proactively support the operator with an assistant system for an automatic early warning