Dynamics of Neural Networks as Nonlinear Systems with Several Equilibria

Dynamics of Neural Networks as Nonlinear Systems with Several Equilibria

Daniela Danciu (University of Craiova, Romania)
DOI: 10.4018/978-1-59904-996-0.ch018
OnDemand PDF Download:


Neural networks—both natural and artificial, are characterized by two kinds of dynamics. The first one is concerned with what we would call “learning dynamics”. The second one is the intrinsic dynamics of the neural network viewed as a dynamical system after the weights have been established via learning. The chapter deals with the second kind of dynamics. More precisely, since the emergent computational capabilities of a recurrent neural network can be achieved provided it has suitable dynamical properties when viewed as a system with several equilibria, the chapter deals with those qualitative properties connected to the achievement of such dynamical properties as global asymptotics and gradient-like behavior. In the case of the neural networks with delays, these aspects are reformulated in accordance with the state of the art of the theory of time delay dynamical systems.
Chapter Preview


Neural networks are computing devices for Artificial Intelligence (AI) belonging to the class of learning machines (with the special mention that learning is viewed at the sub-symbolic level). The basic feature of the neural networks is the interconnection of some simple computing elements in a very dense network and this gives the so-called collective emergent computing capabilities. The simple computing element is here the neuron or, more precisely, the artificial neuron – a simplified model of the biological neuron.

Artificial Neural Networks structures are broadly classified into two main classes: recurrent and non-recurrent networks. We shall focus on the class of recurrent neural networks (RNN). Due to the cyclic interconnections between the neurons, RNNs are dynamical nonlinear systems displaying some very rich temporal and spatial qualitative behaviors: stable and unstable fixed points, periodic, almost periodic or chaotic behaviors. This fact makes RNN applicable for modeling some cognitive functions such as associative memories, unsupervised learning, self-organizing maps and temporal reasoning.

The mathematical models of neural networks arise both from the modeling of some behaviors of biological structures or from the necessity of Artificial Intelligence to consider some structures which solve certain tasks. None of these two cases has as primary aim stability aspects and a “good” qualitative behavior. On the other hand, these properties are necessary and therefore important for the network to achieve its functional purpose that may be defined as “global pattern formation”. It thus follows that any AI device, in particular a neural network, has to be checked a posteriori (i.e. after the functional design) for its properties as a dynamical system, and this analysis is performed on its mathematical model.

A common feature of various RNN (automatic classifiers, associative memories, cellular neural networks) is that they are all nonlinear dynamical systems with multiple equilibria. In fact it is exactly the equilibria multiplicity that gives to all AI devices their computational and learning capabilities. As pointed out in various reference books, satisfactory operation of a neural network (as well as of other AI devices) requires its evolution towards those equilibria that are significant in the application.

Let us remark here that if a system has several isolated equilibria this does not mean that all these equilibria are stable – they may be also unstable. This fact leads to the necessity of a qualitative analysis of the system’s properties. Since there are important both the local stability of each equilibrium point and also (or more) the global behavior of the entire network, we shall discuss here RNN within the frameworks of the Stability Theory and the Theory of Systems with Several Equilibria.

The chapter is organized as follow. The Background section starts with a presentation of RNN from the point of view of those dynamic properties (specific to the systems with several equilibria), which make them desirable for modeling the associative memories. Next, there are provided the definitions and the basic results of the Theory of Systems with Several Equilibria, discussing these tools related to the Artificial Intelligence domain requirements. The main section consists of two parts. In the first part it is presented the basic tool for analyzing the desired qualitative properties for RNN as systems with multiple equilibria. The second part deals with the effect of time-delays on the dynamics of RNN. Moreover, one will consider here the time-delay RNN under forcing stimuli that have to be “reproduced”(synchronization). The chapter ends with Conclusions and comments on Future trends and Future research directions. Additional reading is finally suggested.

Complete Chapter List

Search this Book:
Table of Contents
Alfonso Araque Almendros
Ana B. Porto Pazos, Alejandro Pazos Sierra, Washington Buño Buceta
Chapter 1
Eduardo Malmierca, Nazareth P. Castellanos, Valeri A. Makarov, Angel Nuñez
It is well know the temporal structure of spike discharges is crucial to elicit different types of neuronal plasticity. Also, precise and... Sample PDF
Corticofugal Modulation of Tactile Responses of Neurons in the Spinal Trigeminal Nucleus: A Wavelet Coherence Study
Chapter 2
Didier Le Ray, Morgane Le Bon-Jego, Daniel Cattaert
Computational neuroscience has a lot to gain from invertebrate research. In this chapter focusing on the sensory-motor network that controls leg... Sample PDF
Neural Mechanisms of Leg Motor Control in Crayfish: Insights for Neurobiologically-Inspired Autonomous Systems
Chapter 3
Oscar Herreras, Julia Makarova, José Manuel Ibarz
Neurons send trains of action potentials to communicate each other. Different messages are issued according to varying inputs, but they can also mix... Sample PDF
Forward Dendritic Spikes: A Mechanism for Parallel Processing in Dendritic Subunits and Shifting Output Codes
Chapter 4
Gheorghe Paun, Mario J. Perez-Jimenez
This chapter is a quick survey of spiking neural P systems, a branch of membrane computing which was recently introduced with motivation from neural... Sample PDF
Spiking Neural P Systems: An Overview
Chapter 5
Juan Ramón Rabuñal Dopico, Javier Pereira Loureiro, Mónica Miguélez Rico
In this chapter, we state an evolution of the Recurrent ANN (RANN) to enforce the persistence of activations within the neurons to create activation... Sample PDF
Simulation of the Action Potential in the Neuron's Membrane in Artificial Neural Networks
Chapter 6
Ana B. Porto Pazos, Alberto Alvarellos González, Alejandro Pazos Sierra
The Artificial NeuroGlial Networks, which try to imitate the neuroglial brain networks, appeared in order to process the information by means of... Sample PDF
Recent Methodology in Connectionist Systems
Chapter 7
José A. Fernández-León, Gerardo G. Acosta, Miguel A. Mayosky, Oscar C. Ibáñez
This work is intended to give an overview of technologies, developed from an artificial intelligence standpoint, devised to face the different... Sample PDF
A Biologically Inspired Autonomous Robot Control Based on Behavioural Coordination in Evolutionary Robotics
Chapter 8
Enrique Mérida-Casermeiro, Domingo López-Rodríguez, J.M. Ortiz-de-Lazcano-Lobato
In this chapter, two important issues concerning associative memory by neural networks are studied: a new model of hebbian learning, as well as the... Sample PDF
An Approach to Artificial Concept Learning Based on Human Concept Learning by Using Artificial Neural Networks
Chapter 9
Enrique Fernández-Blanco, Julian Dorado, Nieves Pedreira
The artificial embryogeny term overlaps all the models that try to adapt cellular properties into artificial models. This chapter presents a new... Sample PDF
Artificial Cell Systems Based in Gene Expression Protein Effects
Chapter 10
Computing vs. Genetics  (pages 165-181)
José M. Barreiro, Juan Pazos
This chapter first presents the interrelations between computing and genetics, which both are based on information and, particularly... Sample PDF
Computing vs. Genetics
Chapter 11
Iara Moema Oberg Vilela
This chapter discusses guidelines and models of Mind from Cognitive Sciences in order to generate an integrated architecture for an artificial mind... Sample PDF
Artificial Mind for Virtual Characters
Chapter 12
Zhijun Yang, Felipe M.G. França
As an engine of almost all life phenomena, the motor information generated by the central nervous system (CNS) plays a critical role in the... Sample PDF
A General Rhythmic Pattern Generation Architecture for Legged Locomotion
Chapter 13
Marcos Gestal, José Manuel Vázquez Naya, Norberto Ezquerra
Traditionally, the Evolutionary Computation (EC) techniques, and more specifically the Genetic Algorithms (GAs), have proved to be efficient when... Sample PDF
Genetic Algorithms and Multimodal Search
Chapter 14
Jesús M. Miró, Alfonso Rodríguez-Patón
Synthetic biology and biomolecular computation are disciplines that fuse when it comes to designing and building information processing devices. In... Sample PDF
Biomolecular Computing Devices in Synthetic Biology
Chapter 15
Alejandro Rodríguez, Alexander Grushin, James A. Reggia
Drawing inspiration from social interactions in nature, swarm intelligence has presented a promising approach to the design of complex systems... Sample PDF
Guiding Self-Organization in Systems of Cooperative Mobile Agents
Chapter 16
Agostino Forestiero, Carlo Mastroianni, Fausto Pupo, Giandomenico Spezzano
This chapter proposes a bio-inspired approach for the construction of a self-organizing Grid information system. A dissemination protocol exploits... Sample PDF
Evaluating a Bio-Inspired Approach for the Design of a Grid Information System: The SO-Grid Portal
Chapter 17
Steven M. Corns, Daniel A. Ashlock, Kenneth Mark Bryden
This chapter presents Graph Based Evolutionary Algorithms. Graph Based Evolutionary Algorithms are a generic enhancement and diversity management... Sample PDF
Graph Based Evolutionary Algorithms
Chapter 18
Daniela Danciu
Neural networks—both natural and artificial, are characterized by two kinds of dynamics. The first one is concerned with what we would call... Sample PDF
Dynamics of Neural Networks as Nonlinear Systems with Several Equilibria
Chapter 19
Jianhua Yang, Evor L. Hines, Ian Guymer, Daciana D. Iliescu, Mark S. Leeson, Gregory P. King, XuQuin Li
In this chapter a novel method, the Genetic Neural Mathematical Method (GNMM), for the prediction of longitudinal dispersion coefficient is... Sample PDF
A Genetic Algorithm-Artificial Neural Network Method for the Prediction of Longitudinal Dispersion Coefficient in Rivers
Chapter 20
Malcolm J. Beynon, Kirsty Park
This chapter employs the fuzzy decision tree classification technique in a series of biological based application problems. With its employment in a... Sample PDF
The Exposition of Fuzzy Decision Trees and Their Application in Biology
About the Contributors