A New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks

A New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks

Zhao Lu (Tuskegee University, USA), Leang-san Shieh (University of Houston, USA) and Guanrong Chen (City University of Hong Kong, China)
Copyright: © 2009 |Pages: 12
DOI: 10.4018/978-1-59904-897-0.ch019
OnDemand PDF Download:
$37.50

Abstract

Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activation function for the nodes in the hidden layer. The new network is equivalent to a HONN; however, due to the underlying connections with polynomial kernel support vector machines, the weights and the structure of the network can be determined simultaneously using structural risk minimization. The advantage of the topology of the polynomial kernel network and the use of a support vector kernel expansion paves the way to represent nonlinear functions or systems, and underpins some advanced analysis of the network performance. In this chapter, from the perspective of network complexity, both quadratic programming and linear programming based training of the polynomial kernel network are investigated.
Chapter Preview
Top

Introduction

As an important neural processing topology, artificial higher order neural networks (HONNs) have demonstrated great potential for approximating unknown functions and modeling unknown systems (Kosmatopoulos et al., 1995; Kosmatopoulos and Christodoulou, 1997). In particular, HONNs have been adopted as basic modules in the construction of dynamic system identifiers and also controllers for highly uncertain systems (Rovithakis, 1999; Lu et al., 2006). Nevertheless, as an important factor that affects the performance of neural networks, the structure of a network is usually hard to determine appropriately in any specific application. It is possible to reduce modeling errors by increasing the complexity of the network; however, increasing the complexity may overfit the data leading to a degradation of its generalization ability. As a consequence, in practice the choice of network structure is often a compromise between modeling errors and the network complexity. Some efforts have been made in the attempt to determine the optimal topological structure of HONN by using for example genetic algorithms (Rovithakis et al., 2004).

Recently, there has been a trend in the machine learning community to construct a nonlinear version of a linear algorithm using the so-called ‘kernel method’ (Schölkopf and Smola, 2002; Vert et al., 2004). As a new generation of learning algorithms, the kernel method utilizes techniques from optimization, statistics, and functional analysis to achieve maximal generality, flexibility, and performance. The kernel machine allows high-dimensional inner-product computations to be performed with very little overhead and brings all the benefits of the mature linear estimation theory. Of particular significance is the Support Vector Machine (SVM) that forms an important subject in the learning theory. SVM is derived from statistical learning theory (Evgeniou et al., 2000; Cristianini and Shawe-Taylor, 2000), which is a two-layer network with inputs transformed by the kernels corresponding to a subset of the input data, while its output is a linear function of the weights and kernels. The weights and the structure of the SVM are obtained simultaneously by a constrained minimization at a given precision level of the modeling errors. For all these reasons, the kernel methods have become more and more popular as an alternative to neural-network approaches. However, due to the fact that SVM is basically a non-parametric technique, its effective use in dynamical systems and control theory remains to be seen.

Actually, SVM includes a number of heuristic algorithms as special cases. The relationships between SVM and radial basis function (RBF) networks, neuro-fuzzy networks and multilayer perceptron have been accentuated and utilized for developing new learning algorithms (Chan et al., 2001; Chan et al., 2002; Suykens and Vandewalle, 1999). Of particular interest is a recent observation that Wiener and Volterra theories, which extend the standard convolution description of linear systems by a series of polynomial integral operators with increasing degrees of nonlinearity, can be put into a kernel regression framework (Franz and Schölkopf, 2006).

Inspired by the unifying view of Weiner and Volterra theories and polynomial kernel regression, provided by (Franz and Schölkopf, 2006), and by the fact that the Wiener expansion decomposes a signal according to the order of interaction of its input elements, in this chapter a new topology for HONN, called the polynomial kernel network, is proposed and investigated, which bridges the gap between the parametric HONN model and the non-parametric support vector regression model.

Complete Chapter List

Search this Book:
Reset
Dedication
Table of Contents
Acknowledgment
Ming Zhang
Chapter 1
Ming Zhang
This chapter delivers general format of Higher Order Neural Networks (HONNs) for nonlinear data analysis and six different HONN models. This chapter... Sample PDF
Artificial Higher Order Neural Network Nonlinear Models: SAS NLIN or HONNs?
$37.50
Chapter 2
Adam Knowles, Abir Hussain, Wael El Deredy, Paulo G.J. Lisboa, Christian L. Dunis
Multi-Layer Perceptrons (MLP) are the most common type of neural network in use, and their ability to perform complex nonlinear mappings and... Sample PDF
Higher Order Neural Networks with Bayesian Confidence Measure for the Prediction of the EUR/USD Exchange Rate
$37.50
Chapter 3
Da Shi, Shaohua Tan, Shuzhi Sam Ge
Real-world financial systems are often nonlinear, do not follow any regular probability distribution, and comprise a large amount of financial... Sample PDF
Automatically Identifying Predictor Variables for Stock Return Prediction
$37.50
Chapter 4
John Seiffertt, Donald C. Wunsch II
As the study of agent-based computational economics and finance grows, so does the need for appropriate techniques for the modeling of complex... Sample PDF
Higher Order Neural Network Architectures for Agent-Based Computational Economics and Finance
$37.50
Chapter 5
Yuehui Chen, Peng Wu, Qiang Wu
Forecasting exchange rates is an important financial problem that is receiving increasing attention especially because of its difficulty and... Sample PDF
Foreign Exchange Rate Forecasting Using Higher Order Flexible Neural Tree
$37.50
Chapter 6
Yuehui Chen, Peng Wu, Qiang Wu
Artificial Neural Networks (ANNs) have become very important in making stock market predictions. Much research on the applications of ANNs has... Sample PDF
Higher Order Neural Networks for Stock Index Modeling
$37.50
Chapter 7
Ming Zhang
This chapter develops a new nonlinear model, Ultra high frequency Trigonometric Higher Order Neural Networks (UTHONN), for time series data... Sample PDF
Ultra High Frequency Trigonometric Higher Order Neural Networks for Time Series Data Analysis
$37.50
Chapter 8
Panos Liatsis, Abir Hussain, Efstathios Milonidis
The research described in this chapter is concerned with the development of a novel artificial higher order neural networks architecture called the... Sample PDF
Artificial Higher Order Pipeline Recurrent Neural Networks for Financial Time Series Prediction
$37.50
Chapter 9
Abir Hussain, Panos Liatsis
The research described in this chapter is concerned with the development of a novel artificial higherorder neural networks architecture called the... Sample PDF
A Novel Recurrent Polynomial Neural Network for Financial Time Series Prediction
$37.50
Chapter 10
David R. Selviah, Janti Shawash
Generalized correlation higher order neural network designs are developed. Their performance is compared with that of first order networks... Sample PDF
Generalized Correlation Higher Order Neural Networks for Financial Time Series Prediction
$37.50
Chapter 11
Godfrey C. Onwubolu
Real world problems are described by nonlinear and chaotic processes, which makes them hard to model and predict. This chapter first compares the... Sample PDF
Artificial Higher Order Neural Networks in Time Series Prediction
$37.50
Chapter 12
Rozaida Ghazali, Dhiya Al-Jumeily
This chapter discusses the use of two artificial Higher Order Neural Networks (HONNs) models; the Pi- Sigma Neural Networks and the Ridge Polynomial... Sample PDF
Application of Pi-Sigma Neural Networks and Ridge Polynomial Neural Networks to Financial Time Series Prediction
$37.50
Chapter 13
Edgar N. Sanchez, Alma Y. Alanis, Jesús Rico
In this chapter, we propose the use of Higher Order Neural Networks (HONNs) trained with an extended Kalman filter based algorithm to predict the... Sample PDF
Electric Load Demand and Electricity Prices ForecastingUsing Higher Order Neural Networks Trained by Kalman Filtering
$37.50
Chapter 14
Shuxiang Xu
Business is a diversified field with general areas of specialisation such as accounting, taxation, stock market, and other financial analysis.... Sample PDF
Adaptive Higher Order Neural Network Models and Their Applications in Business
$37.50
Chapter 15
Jean X. Zhang
This chapter proposes nonlinear models using artificial neural network models to study the relationship between chief elected official (CEO) tenure... Sample PDF
CEO Tenure and Debt: An Artificial Higher Order Neural Network Approach
$37.50
Chapter 16
Christian L. Dunis, Jason Laws, Ben Evans
This chapter investigates the soybean-oil “crush” spread, that is the profit margin gained by processing soybeans into soyoil. Soybeans form a large... Sample PDF
Modelling and Trading the Soybean-Oil Crush Spread with Recurrent and Higher Order Networks: A Comparative Analysis
$37.50
Chapter 17
Madan M. Gupta, Noriyasu Homma, Zeng-Guang Hou, Ashu M. G. Solo, Takakuni Goto
In this chapter, we aim to describe fundamental principles of artificial higher order neural units (AHONUs) and networks (AHONNs). An essential core... Sample PDF
Fundamental Theory of Artificial Higher Order Neural Networks
$37.50
Chapter 18
Jinde Cao, Fengli Ren, Jinling Liang
This chapter concentrates on studying the dynamics of artificial higher order neural networks (HONNs) with delays. Both stability analysis and... Sample PDF
Dynamics in Artificial Higher Order Neural Networks with Delays
$37.50
Chapter 19
Zhao Lu, Leang-san Shieh, Guanrong Chen
Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and... Sample PDF
A New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks
$37.50
Chapter 20
David R. Selviah
This chapter describes the progress in using optical technology to construct high-speed artificial higher order neural network systems. The chapter... Sample PDF
High Speed Optical Higher Order Neural Networks for Discovering Data Trends and Patterns in Very Large Databases
$37.50
Chapter 21
Zidong Wang, Yurong Liu, Xiaohui Liu
This chapter deals with the analysis problem of the global exponential stability for a general class of stochastic artificial higher order neural... Sample PDF
On Complex Artificial Higher Order Neural Networks: Dealing with Stochasticity, Jumps and Delays
$37.50
Chapter 22
Lei Zhang, Simeon J. Simoff, Jing Chun Zhang
This chapter introduces trigonometric polynomial higher order neural network models. In the area of financial data simulation and prediction, there... Sample PDF
Trigonometric Polynomial Higher Order Neural Network Group Models and Weighted Kernel Models for Financial Data Simulation and Prediction
$37.50
About the Contributors