An Industrial Environment Augmented Reality System

An Industrial Environment Augmented Reality System

Srihathai Prammanee (VTT Technical Research Centre of Finland, Finland), Pekka Siltanen (VTT Technical Research Centre of Finland, Finland), Marko Luukkainen (VTT Technical Research Centre of Finland, Finland) and Tommi Karhela (VTT Technical Research Centre of Finland, Finland)
DOI: 10.4018/978-1-61520-655-1.ch033


Simantics Mobile is a toolset for nomadic use of industrial facilities’ information. It supports the interoperability between mobile services and user interactions to facilitate an augmented reality data access for mobile field service operations taking place throughout a process plant lifecycle. The provision beyond the classic mobility and ubiquity (i.e. anytime and anywhere service) is extended. Realising a relationship matrix of contexts, each of which has a unique perspective on knowledge, Simantics Mobile retrieves the right information at the right time and in the right place. Its underlying technological infrastructure encompasses the collaborative functionalities of augmented reality (AR), semantics plant model, and location-based service (LBS). This chapter documents the works in the Simantics Mobile. It also discusses its strategies beyond the implementations. At the end of the chapter, the use case, carried out at Stora Enso Kvarnsveden – a paper mill in Sweden, is demonstrated.
Chapter Preview


In manufacturing, the wide range of field service operations, including installation, operation, maintenance, and reparation, is involved. The maintenance and repair of any complex machinery is a vital process for its efficient operations. The proficiencies of the field service technicians critically affect a business confidence. Any mistake of operative personnel impinges on the quality of products, leading to the potentially long-term financial consequences. Such a case, the necessity of utilising a tool that assists the personnel in accomplishing their tasks is becoming increasing.

Generally, the plant equipment is physically distributed over a place. Thus, the field service operations are mobile by nature and the operating activities require an auxiliary functionality of location-dependent information access. To satisfy the industry requirement, the Simantics Mobile tool, developed as part of Nomadic Use of a Plant Model (Nose) project, opens a new opportunity for emergent mobile applications, supporting the manufacturing performances in a process plant. The tool makes an innovation contribution to the corporate information management. It implements the efficient methodology and application integration platform for accessing the plant-related information ubiquitously and pervasively.

Literally, the Simantics Mobile development is tightly bundled with the diverse technical aspects of mobile embedded software engineering. In principle, the typical software engineering involves the methods of requirement analysis, preliminary design, implementation, assessment, maintenance, and renovation. In spite of sharing the principle, the mobile embedded software engineering is conceptually different owing to the characteristics intrinsic to mobile computing. The reports on (Satyanarayanan, 1996; Yamakami, 2008) exposed various technological characteristics that have raised the challenges for the application interoperability between mobile services and software engineers. Based on those reports, Simantics Mobile takes account that a mobile computation is generally resource-poor, relying on the poor battery lifetime, the small size of memories, and the limited capabilities of physical interfaces. As a result, an ergonomic design is needed.

Enhancing an ergonomically interfacing environment, the concept of multimodal interaction is applied, following the strategy of a sequential combination. Theoretically, the sequential combination characterises a chronological use of modalities. It allows a number of modalities to be active, but only a single modality can function at a time (Niklfeld, 2005). Basically, the possible actions of the sequential combination use one modality after another. Simantics Mobile utilises the sophisticated modality technique of augmented reality (AR) as an alternative of GUI-based input widgets, such as textboxes and event buttons. AR enriches a user interaction by assigning the meaningful and contextual information to a concrete object (The EDUCAUSE Learning Initiative, 2005). A video stream based user interface is facilitated, supplemented with a marker-based identification. Instead of entering explicit input texts, an input request is deduced by the continuously augmented views of a user. Afterwards, the corresponding information is displayed on a mobile screen as conventional texts.

Due to mobility, the personnel have a freedom of roaming around a process plant. A location-based service (LBS) capitalises on the geographical information of the personnel and subsequently relates such information to the conditions and constructions in the plant. A physical route is then displayed on a mobile terminal to guide the personnel possible directions. Two LBS approaches, i.e. visual tracking and Nose’s positioning technique, are applied in Simantics Mobile. The visual tracking gives the real-time pose estimation. It is the navigation method based on a hybrid user interaction in an AR-enabled application. On the other hand, Nose’s positioning technique takes the advantages of available standardised positioning methods and then consolidates those methods for the maximum usability and functionality.

Key Terms in this Chapter

Simantics Mobile: Product concept developed at VTT for mobile field service operation.

Industrial Augmented Reality: Augmented reality (AR) is a term for a live direct or a indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound or graphics. Industrial augmented reality applies this technology in industrial surroundings.

Semantic Plant Model: Data model of a plant expressed using semantic data modelling techniques e.g. W3C semantic web standards.

Camera-Based ‘Input’ Interface: A camera-based interface in which a set of techniques involving images or live video stream are used as an user interface to real objects.

Mobile Field Service Operation: Set of techniques where mobile worker uses mobile terminals for interacting with the production process.

Complete Chapter List

Search this Book: