Article Preview
TopIntroduction
During last ten years of our research, we investigated nonconventional neural architectures and their applications to modeling of dynamical systems, control, prediction, and novel evaluation of dynamic systems. In this paper, we review some achievements with quadratic neural unit (QNU) (Gupta et al., 2003; Bukovsky et al., 2009) in order to highlight its aspects for real applications. Generally from nowadays point of view, QNU can be considered a special class of polynomial neural network or as a special unit of higher order neural networks that are getting very popular today (Ivakhnenko, 1971; Zhang, 2010). The important aspects of QNU discussed in this paper are not unknown, but they are unnecessarily shadowed by focusing on many various too nonlinear neural networks (NN). Today, we can notice enthusiasm about theoretical validations for stability and convergence of powerful cognitive systems such as NN, fuzzy systems, to name but a few in real applications. The theoretical achievements can be difficult to understand and can be costly, at least timely to be applied without thorough mathematical background and without rich experiences in the relevant fields. For example, multilayer perceptron NN (MLP) are very often called universal function approximators; however, it is not usually considered (stated) that it is valid rather from the theoretical point of view; the local minima problem and the related overfitting can disqualify a trained neural model for its real application with new testing data. Thus, managers and process engineers still may not always have full trust or interest in applying the relatively complicated theoretical NN, especially in contrast to the uniqueness of the solution of linear systems and their solvability. Nevertheless, we do not have to use always complicated and too nonlinear NN to achieve superior results with real applications for many (industrial) processes.
Studies of nonconventional neural units have been running on investigation of QNU and Cubic Neural Unit (CNU), mainly, denoted also by higher order (nonlinear) neural units (HONU) (Gupta et al., 2003; Bukovsky et al., 2009). In Ivakhnenko (1971) author deals with polynomial networks. In Gupta et al. (2003) one of the first notations of higher order neural units was introduced. In Bukovsky et al. (2003), Quadratic Neural Unit and Cubic Neural Unit were presented for fast state feedback control of nonlinear systems; for unstable and unknown nonlinear dynamic systems, indicating the capability of the neural units for faster response than a liner state feedback controllers. Also, the stability criteria of a non-linear control loop that include QNU and CNU were analyzed. Later, a new classification of nonconventional neural units was proposed by Bukovsky et al. (2007) where a very comprehensible mathematical structure was remained. These neural architectures were settled to an understandable description with respect to conventional neural networks that usually are seen as black box or gray box. However, not only the last advantages were obtained, also a minimum number of neural elements were needed for maximum approximation capabilities. In Bukovsky et al. (2008), it was shown that a low dimensional quadratic neural unit can be used for adaptive evaluation of higher dimensional chaotic systems. Here, static and dynamic neural units were applied by the gradient descent based on back-propagation learning rule. Recently, in Bukovsky et al. (2010a), it was presented a Discrete Dynamic QNU applied to predict lung respiration dynamics using the real time recurrent learning for dynamic QNU. The prediction time considered was long term and the results obtained were very promising comparing with the recent literature. In Bukovsky et al. (2010b) a novel Quadratic Neural Network (QNN) based on the Quadratic Neural Unit to model measured data in energetic processes was presented. The authors used a sequential learning to reduce computational time with the large number of inputs and show nonlinear approximation for the real process. A comparison between QNU, QNN and Multilayer Perceptron trained by Levenberg-Marquard algorithm was also presented.