Supporting the Assembly Process by Leveraging Augmented Reality, Cloud Computing, and Mobile Devices

Introduction

To date, drawings still serve as the main mean for assembly guidance. However, the use of drawing guidelines is not enough for the complexity of assembling large-scale apparatus such as computer systems or other intricate components. The challenge involves: (a) providing non-intrusive feedback during the assembly process; (b) evaluating the user’s progress in real-time by comparing user’s assembly with a model stored in a database of correct assemblies and actions (including recognizing the current state of the assembly, a complex problem that fits in the image recognition area); (c) providing a correct hint or feedback about the current state of the assembly is also a complex problem since there are many possible paths to achieve a correct assembly; and (d) defining standard formats to describe the assembly process as well as defining a common storage method and a repository for assembly documentation to which recur to when needed.

The Augmented Reality Product Assembler (ARPA) is a solution to the described challenges that leverages the combination of three current and emerging technologies: Augmented Reality (AR), cloud computing, and mobile devices. Due to its embedded characteristics and features, AR is envisaged to afford great potential and be an alternative for traditional means to improve the creation of guidelines that illustrate the assembly processes. AR is used due to its ability to provide a dynamic exhibition of consistent context via animation segments that could be displayed for each assembly step. Observers then could detect the actual dimensions of the real already-positioned components, as well as the registered/stored ones for the virtual to-be-positioned components.

The inclusion of Cloud technology and modern data storage techniques create an opportunity to propose a common repository to standardize assembly guidelines. In particular, information retrieval for just-in-time assembly guidance generates the necessity of high performance computing options for pattern recognition, data mining, and mobility. This functionality is provided by cloud computing.

The use of mobile devices is also important for the ARPA system to be able to maintain mobility. Mobile technology in the form of laptops, tablets, and in particular smartphones, is revolutionizing the way people access information, which is an essential first step in the acquiring knowledge (Peng, Su, Chou, & Tsai, 2009). With the profound rate of increased processing power and portability, the masses are gravitating to mobile devices to “work, learn and study whenever and wherever they want” (Johnson, Smith, Willis, Levine, & Haywood, 2011). With increased access to web-based networks (e.g., the cloud), the process of learning is extended beyond other viable solutions (Johnson, Smith, Willis, Levine, & Haywood, 2012; Peng et al., 2009).

Even though mobile devices and wireless networks have evolved to offer consumers visually rich content, multimedia applications, and responsive high-speed data networks, the value that consumers receive from mobile devices is still limited by the tiny viewing experience provided by the LCD screens. Our interest is to incorporate and improve upon mobile devices with the use of eyewear devices, extending the user’s vision. Another key element of this proposed system is the addition of AR eyewear. Currently in the initial development phase, in conjunction with mobile devices, eyewear would combine new display engine technology with special optics that are embedded into a fashionable or protective eyeglasses.

In addition to leveraging modern hardware technology, the solution also relies on established software standards. Since this kind of project requires the integration of complex algorithms, diverse devices, intrinsic data structures, and high quality implementation (such as robustness, adaptability, and extensibility), the construction of a system to support creating, sharing, and using product assembly guidelines must follow a systematic, disciplined, and quantifiable approach of development, operation, and maintenance of the software involved. The proposed platform is described and fundament on standard software architecture, engineering models, and patterns.