Non Linear and Non Gaussian States and Parameters Estimation using Bayesian Methods-Comparatives Studies

Non Linear and Non Gaussian States and Parameters Estimation using Bayesian Methods-Comparatives Studies

Majdi Mansouri (Texas A&M University at Qatar, Qatar), Moustafa Mohamed-Seghir (Gdynia Maritime University, Poland), Hazem Nounou (Texas A&M University at Qatar, Qatar), Mohamed Nounou (Texas A&M University at Qatar, Qatar) and Haitham A. Abu-Rub (Texas A&M University at Qatar, Qatar)
DOI: 10.4018/978-1-4666-4450-2.ch024
OnDemand PDF Download:


This chapter deals with the problem of non-linear and non-Gaussian states and parameters estimation using Bayesian methods. The performances of various conventional and state-of-the-art state estimation techniques are compared when they are utilized to achieve this objective. These techniques include the Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), and Particle Filter (PF). In the current work, the authors consider two systems (biological model and power system) to perform evaluation of estimation algorithms. The results of the comparative studies show that the UKF provides a higher accuracy than the EKF due to the limited ability of EKF to accurately estimate the mean and covariance matrix of the estimated states through lineralization of the nonlinear process model. The results also show that the PF provides a significant improvement over the UKF because, unlike UKF, PF is not restricted by linear-Gaussian assumptions which greatly extends the range of problems that can be tackled.
Chapter Preview


Many process operations, such as modeling, monitoring, and control, require the availability of state and/or parameter measurements. However, due to the difficulty of, or cost associated with, obtaining these measurements, state and/or parameter estimators are often used to overcome this problem. For example, in process monitoring and control, sometimes it is challenging to measure some of the key variables. In such cases, estimates of these variables can be obtained using state estimation. Also, in modeling, several model parameters need to be estimated. Estimating these parameters usually requires several experimental setups that can be challenging and expensive. Hence, estimating such parameters using state estimation can be of a great value. In this chapter, the objective is to compare the performances of various state-of-the-art state estimation techniques in estimating the state variables using different kinds of observation models (i.e., biological model and power system) and their abilities to estimate some of the key system parameters, which are needed to define the process model. Several estimation techniques, such as the extended Kalman filter, unscented Kalman filter and more recently the Sequential Monte Carlo method have been developed and utilized in many applications. Several estimation techniques, such as the extended Kalman filter, unscented Kalman filter and more recently the Sequential Monte Carlo method have been developed and utilized in many applications. The classical Kalman Filter (KF) was developed in the 1960s (Kalman, 1960), and has been widely applied in various engineering and science areas, including communications, control, machine learning, neuroscience, and many others. In the case where the model describing the system is assumed to be linear and Gaussian, the KF provides an optimal solution (Simon, 2006; Grewal & Andrews, 2008). KF has also been formulated in the context of Takagi-Sugeno fuzzy systems, which can be described by a convex set of multiple linear models (Chen et al., 1998; Simon, 2003). It is known that KF is computationally efficient; however, it is limited by the non-universal linear and Gaussian modeling assumptions. To relax such assumptions, the Extended Kalman Filter (Simon, 2006; Grewal & Andrews, 2008; Julier et al., 1997; Ljung et al., 1979; Kim et al., 1994) and the Unscented Kalman Filter (Simon, 2006; Grewal & Andrews, 2008; Wan et al., 2000; Wan et al., 2001; Sarkka et al., 2001; Sarkka et al., 2007) have been developed. In extended Kalman filtering, the model describing the system is linearized at every time sample (which means that the model is assumed to be differentiable). Therefore, for highly nonlinear models, EKF does not usually provide a satisfactory performance. The UKF, on the other hand, instead of linearizing the model to approximate the mean and covariance matrix of the state vector, uses the unscented transformation to approximate these moments. In the unscented transformation, a set of samples (called sigma points) are selected and propagated through the nonlinear model to improve the approximation of these moments and thus the accuracy of state estimation. Other state estimation techniques use a Bayesian framework to estimate the state and/or parameter vector (Beal et al., 2003). The Bayesian framework relies on computing the probability distribution of the unobserved state given a sequence of the observed data in addition to the state evolution model. Consider an observed data set y, which is generated from a model defined by a set of unknown parameters (Smidl et al., 2005). The beliefs about the data are completely expressed via the parametric probabilistic observation model, The learning of uncertainty or randomness of a process is solved by constructing a distribution called the posterior distribution, which quantifies our belief about the system after obtaining the measurements. According to Bayes rule, the posterior can be expressed as

(1) where is the conditional distribution of the data given the model parameter vector, which is called the likelihood function, and is the prior distribution, which quantifies our belief about before obtaining the measurement. Thus, Bayes rule specifies how our prior belief, quantified by the priori distribution, is updated according to the measured data Unfortunately, for most nonlinear systems and non-Gaussian noise observations, closed-form analytic expression of the posterior distribution of the state is untractable (Kotecha & Djuric, 2003). To overcome this drawback, a nonparametric Monte Carlo sampling based method called Sequential Monte Carlo method (SMC) (also known as Particle Filtering (PF)) (Doucet & Tadic, 2003; Poyiadjis et al., 2005) has recently gained popularity. The Particle Filter (PF) approximates the posterior probability distribution by a set of weighted samples, called particles. Since real-world problems usually involve high-dimensional random variables with complex uncertainty, the nonparametric and sample-based estimation of uncertainty (provided by the PF) has thus become quite popular to capture and represent the complex distribution in nonlinear and non-Gaussian environments (Arulampalam et al., 2005; Mansouri et al., 2009; Lindenmeyer et al., 2002). The PF has the ability to accommodate nonlinear and multi-modal dynamics, but at the cost of more computational complexity and storage requirements. Each of the above estimation techniques has its advantages and disadvantages. The PF can be applied to large parameter spaces, has better convergence properties and easier to implement than the UKF, and both of them can provide improved accuracy over the EKF. Some practical challenges, however, can affect the accuracy of estimated states and/or parameters. Such challenge include the large number of states and parameters to be estimated, the presence of measurement noise in the data, and the availability of small number of measured data samples.

Key Terms in this Chapter

Cad System: Cad system is one of the conditional stress response modules in E.coli, which is induced only at low pH and a lysine-rich environment

Induction Motor (IM): Induction motor is an AC motor in which all electromagnetic energy is transferred by inductive coupling from a primary coiling to a secondary coiling, where the two coilings are separated by an air gap. In three-phase induction motors, energy is transferred from the stator to either a wound rotor or a short-circuited squirrel cage rotor.

Power System: Power system is a system of high tension cables by which electrical power is distributed throughout a region. It is also known as a network of electrical components used to supply, transmit and use electric power.

Unscented Kalman filter (UKF): UKF is an estimation technique that uses a deterministic sampling technique known as the unscented transform to approximate the mean and the covariance of the state and parameter vector.

Extended Kalman Filter (EKF): EKF is an estimation technique that is applicable to nonlinear and non-Gaussian models. As the Kalman filter is applicable to linear systems, the EKF can be viewed as an extension of the Kalman filter that is applied to a linearized version of the nonlinear model.

Particle Filter (PF): PF, which is based on Bayesian estimation, is a sequential Monte Carlo state estimation method for nonlinear and non-Gaussian systems. The EKF and UKF algorithms do not always provide a satisfactory performance, especially for highly nonlinear processes as model linearization does not necessarily provide good estimates of the mean of the state vector and the covariance matrix of the estimation error, which are used in state estimation. These issues are addressed by the PF.

Biological System: Biological system is an organ or group of organs within the body that work collectively to achieve a certain task.

States and Parameters Estimation: State and parameter estimation is the process of obtaining the estimators of state variables and model parameters based on a dataset.

Complete Chapter List

Search this Book: