Artificial Higher Order Neural Network Nonlinear Models: SAS NLIN or HONNs?

Artificial Higher Order Neural Network Nonlinear Models: SAS NLIN or HONNs?

Ming Zhang (Christopher Newport University, USA)
Copyright: © 2009 |Pages: 47
DOI: 10.4018/978-1-59904-897-0.ch001
OnDemand PDF Download:
$37.50

Abstract

This chapter delivers general format of Higher Order Neural Networks (HONNs) for nonlinear data analysis and six different HONN models. This chapter mathematically proves that HONN models could converge and have mean squared errors close to zero. This chapter illustrates the learning algorithm with update formulas. HONN models are compared with SAS Nonlinear (NLIN) models and results show that HONN models are 3 to 12% better than SAS Nonlinear models. Moreover, this chapter shows how to use HONN models to find the best model, order and coefficients, without writing the regression expression, declaring parameter names, and supplying initial parameter values.
Chapter Preview
Top

Introduction

Background of Higher-Order Neural Networks (HONNs)

Although traditional Artificial Neural Network (ANN) models are recognized for their great performance in pattern matching, pattern recognition, and mathematical function approximation, they are often stuck in local, rather than global minima. In addition, ANNs take unacceptably long time to converge in practice (Fulcher, Zhang, and Xu 2006). Moreover, ANNs are unable to manage non-smooth, discontinuous training data, and complex mappings in financial time series simulation and prediction. ANNs are ‘black box’ in nature, which means the explanations for their output are not obvious. This leads to the motivation for studies on Higher Order Neural Networks (HONNs).

HONN includes the neuron activation functions, preprocessing of the neuron inputs, and connections to more than one layer (Bengtsson, 1990). In this chapter, HONN refers to the neuron type, which can be linear, power, multiplicative, sigmoid, logarithmic, etc. The first-order neural networks can be formulated by using linear neurons that are only capable of capturing first-order correlations in the training data (Giles & Maxwell, 1987). The second order or above HONNs involve higher-order correlations in the training data that require more complex neuron activation functions (Barron, Gilstrap & Shrier, 1987; Giles & Maxwell, 1987; Psaltis, Park & Hong, 1988). Neurons which include terms up to and including degree-k are referred to as kth-order neurons (Lisboa and Perantonis, 1991).

Rumelhart, Hinton, and McClelland (1986) develop ‘sigma-pi’ neurons where they show that the generalized standard BackPropagation algorithm can be applied to simple additive neurons. Both Hebbian and Perceptron learning rules can be employed when no hidden layers are involved (Shin 1991). The performance of first-order ANNs can be improved by utilizing sophisticated learning algorithms (Karayiannis and Venetsanopoulos, 1993). Redding, Kowalczy and Downs (1993) develop a constructive HONN algorithm. Zhang and Fulcher (2004) develop Polynomial, Trigonometric and other HONN models. Giles, Griffin and Maxwell (1988) and Lisboa and Pentonis (1991) show that the multiplicative interconnections within ANNs have been used in many applications, including invariant pattern recognition.

Others suggest groups of individual neurons (Willcox, 1991; Hu and Pan, 1992). ANNs can simulate any nonlinear functions to any degree of accuracy (Hornik, 1991; and Leshno, 1993).

Zhang, Fulcher, and Scofield (1997) show that ANN groups offer superior performance compared with ANNs when dealing with discontinuous and non-smooth piecewise nonlinear functions. Compared with Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN), Neural Adaptive Higher Order Neural Network (NAHONN) offers more flexibility and more accurate approximation capability. Since using NAHONN the hidden layer variables are adjustable (Zhang, Xu, and Fulcher, 2002). In addition, Zhang, Xu, and Fulcher (2002) proves that NAHONN groups are capable of approximating any kinds of piecewise continuous function, to any degree of accuracy. In addition, these models are capable of automatically selecting both the optimum model for a particular time series and the appropriate model order.

Complete Chapter List

Search this Book:
Reset
Dedication
Table of Contents
Acknowledgment
Ming Zhang
Chapter 1
Ming Zhang
This chapter delivers general format of Higher Order Neural Networks (HONNs) for nonlinear data analysis and six different HONN models. This chapter... Sample PDF
Artificial Higher Order Neural Network Nonlinear Models: SAS NLIN or HONNs?
$37.50
Chapter 2
Adam Knowles, Abir Hussain, Wael El Deredy, Paulo G.J. Lisboa, Christian L. Dunis
Multi-Layer Perceptrons (MLP) are the most common type of neural network in use, and their ability to perform complex nonlinear mappings and... Sample PDF
Higher Order Neural Networks with Bayesian Confidence Measure for the Prediction of the EUR/USD Exchange Rate
$37.50
Chapter 3
Da Shi, Shaohua Tan, Shuzhi Sam Ge
Real-world financial systems are often nonlinear, do not follow any regular probability distribution, and comprise a large amount of financial... Sample PDF
Automatically Identifying Predictor Variables for Stock Return Prediction
$37.50
Chapter 4
John Seiffertt, Donald C. Wunsch II
As the study of agent-based computational economics and finance grows, so does the need for appropriate techniques for the modeling of complex... Sample PDF
Higher Order Neural Network Architectures for Agent-Based Computational Economics and Finance
$37.50
Chapter 5
Yuehui Chen, Peng Wu, Qiang Wu
Forecasting exchange rates is an important financial problem that is receiving increasing attention especially because of its difficulty and... Sample PDF
Foreign Exchange Rate Forecasting Using Higher Order Flexible Neural Tree
$37.50
Chapter 6
Yuehui Chen, Peng Wu, Qiang Wu
Artificial Neural Networks (ANNs) have become very important in making stock market predictions. Much research on the applications of ANNs has... Sample PDF
Higher Order Neural Networks for Stock Index Modeling
$37.50
Chapter 7
Ming Zhang
This chapter develops a new nonlinear model, Ultra high frequency Trigonometric Higher Order Neural Networks (UTHONN), for time series data... Sample PDF
Ultra High Frequency Trigonometric Higher Order Neural Networks for Time Series Data Analysis
$37.50
Chapter 8
Panos Liatsis, Abir Hussain, Efstathios Milonidis
The research described in this chapter is concerned with the development of a novel artificial higher order neural networks architecture called the... Sample PDF
Artificial Higher Order Pipeline Recurrent Neural Networks for Financial Time Series Prediction
$37.50
Chapter 9
Abir Hussain, Panos Liatsis
The research described in this chapter is concerned with the development of a novel artificial higherorder neural networks architecture called the... Sample PDF
A Novel Recurrent Polynomial Neural Network for Financial Time Series Prediction
$37.50
Chapter 10
David R. Selviah, Janti Shawash
Generalized correlation higher order neural network designs are developed. Their performance is compared with that of first order networks... Sample PDF
Generalized Correlation Higher Order Neural Networks for Financial Time Series Prediction
$37.50
Chapter 11
Godfrey C. Onwubolu
Real world problems are described by nonlinear and chaotic processes, which makes them hard to model and predict. This chapter first compares the... Sample PDF
Artificial Higher Order Neural Networks in Time Series Prediction
$37.50
Chapter 12
Rozaida Ghazali, Dhiya Al-Jumeily
This chapter discusses the use of two artificial Higher Order Neural Networks (HONNs) models; the Pi- Sigma Neural Networks and the Ridge Polynomial... Sample PDF
Application of Pi-Sigma Neural Networks and Ridge Polynomial Neural Networks to Financial Time Series Prediction
$37.50
Chapter 13
Edgar N. Sanchez, Alma Y. Alanis, Jesús Rico
In this chapter, we propose the use of Higher Order Neural Networks (HONNs) trained with an extended Kalman filter based algorithm to predict the... Sample PDF
Electric Load Demand and Electricity Prices ForecastingUsing Higher Order Neural Networks Trained by Kalman Filtering
$37.50
Chapter 14
Shuxiang Xu
Business is a diversified field with general areas of specialisation such as accounting, taxation, stock market, and other financial analysis.... Sample PDF
Adaptive Higher Order Neural Network Models and Their Applications in Business
$37.50
Chapter 15
Jean X. Zhang
This chapter proposes nonlinear models using artificial neural network models to study the relationship between chief elected official (CEO) tenure... Sample PDF
CEO Tenure and Debt: An Artificial Higher Order Neural Network Approach
$37.50
Chapter 16
Christian L. Dunis, Jason Laws, Ben Evans
This chapter investigates the soybean-oil “crush” spread, that is the profit margin gained by processing soybeans into soyoil. Soybeans form a large... Sample PDF
Modelling and Trading the Soybean-Oil Crush Spread with Recurrent and Higher Order Networks: A Comparative Analysis
$37.50
Chapter 17
Madan M. Gupta, Noriyasu Homma, Zeng-Guang Hou, Ashu M. G. Solo, Takakuni Goto
In this chapter, we aim to describe fundamental principles of artificial higher order neural units (AHONUs) and networks (AHONNs). An essential core... Sample PDF
Fundamental Theory of Artificial Higher Order Neural Networks
$37.50
Chapter 18
Jinde Cao, Fengli Ren, Jinling Liang
This chapter concentrates on studying the dynamics of artificial higher order neural networks (HONNs) with delays. Both stability analysis and... Sample PDF
Dynamics in Artificial Higher Order Neural Networks with Delays
$37.50
Chapter 19
Zhao Lu, Leang-san Shieh, Guanrong Chen
Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and... Sample PDF
A New Topology for Artificial Higher Order Neural Networks: Polynomial Kernel Networks
$37.50
Chapter 20
David R. Selviah
This chapter describes the progress in using optical technology to construct high-speed artificial higher order neural network systems. The chapter... Sample PDF
High Speed Optical Higher Order Neural Networks for Discovering Data Trends and Patterns in Very Large Databases
$37.50
Chapter 21
Zidong Wang, Yurong Liu, Xiaohui Liu
This chapter deals with the analysis problem of the global exponential stability for a general class of stochastic artificial higher order neural... Sample PDF
On Complex Artificial Higher Order Neural Networks: Dealing with Stochasticity, Jumps and Delays
$37.50
Chapter 22
Lei Zhang, Simeon J. Simoff, Jing Chun Zhang
This chapter introduces trigonometric polynomial higher order neural network models. In the area of financial data simulation and prediction, there... Sample PDF
Trigonometric Polynomial Higher Order Neural Network Group Models and Weighted Kernel Models for Financial Data Simulation and Prediction
$37.50
About the Contributors