Machine Learning Algorithms for Solving Linear Systems of Equations

Introduction

We are in the center of a significant transformation regarding the way production with the arrival digitization of manufacturing. This transformation is called Industry 4.0 to represent the fourth revolution that has occurred in manufacturing. In Industry 3.0, automation started with the adoption of computers. Industry 4.0 enhances it with smart and autonomous systems fed by data. Computers are connected and communicate to make decisions without human involvement ultimately. A combination of cyber-physical systems, the Internet of Things, and the Internet of Systems make Industry 4.0 possible and the smart factory a reality. With the support of intelligent machines getting smarter as they access to more data, factories will become more efficient and productive and less wasteful.

As moving into the Fourth Industrial Revolution with the rapidly digital transformation, there is a huge volume of data to be handled in each industry. Since connected machines create and share a tremendous volume of data, analyzing the data to identify patterns and insights would be impossible for a human to do in a sensible time. Industrial simulations generally produce data including the inputs and outputs of linear systems with several million unknowns. For example, the data in such domains (civil engineering, mechanical engineering, chemistry, biology, economics, etc.) arises as sparse matrices in which most of entries are zero. Determining a suitable method to solve these applications is a major problem requiring knowledge in the specific discipline like mathematics, and certain degree of expertise. Moreover, there is not a certain knowledge about which method the most suitable for many numerical problems is, or what the relationship between the kind of problem and the behavior of the solution methods.

Industrial data collected from sensors in its equipment or stored in the cloud environment can be in many forms such as sparse matrices as stated previous paragraph. Processing of these data requires some matrix computations and solution of the linear systems. Solution methods are generally grouped into two categories which are direct and iterative. Iterative methods are utilized for solving large sparse linear systems since they require less memory and operation count especially low accuracy is desired. Determining the best method to solve the linear systems through execution time and accuracy is challenging task. Up to date studies use machine learning (ML) techniques for identifying the best solver and preconditioner for a given problem. ML enable to process vast amounts of data to create predictive models. This supports researchers in developing classifiers which take a set of linear system attributes as input and returning preconditioned solver as the output.

Many real-world problems require solution of linear systems. There are several methods for solving such systems, and it is critical to choose the method that best suits the problem area. The state-of-the-art methods use machine learning methods for determining the best method for the problem solution. In this chapter, we review machine learning techniques for determining optimum linear solver and preconditioner for solving such systems. The chapter is organized as follows: In section I, we introduce the problem of the solution of the linear systems and application areas. In section II, some background information about preconditioning for solving the linear systems is given. In section III, existing linear solvers are explained and compared regarding their performance and complexities. Determining good set of features is a fundamental problem in forming good classifiers. In section IV, we overview domain specific features, sparsity pattern features, and generic features, which give important clues for understanding sparse matrices and provide guidelines for choosing appropriate linear solver and preconditioner. Finally, in section V, we survey the state-of-the-art machine learning approaches for choosing optimum linear solver and preconditioners.