Abstract
Artificial Neural Networks have proven, along the last four decades, to be an important tool for modelling of the functional structures of the nervous system, as well as for the modelling of non-linear and adaptive systems in general, both biological and non biological (Haykin, 1999). They also became a powerful biologically inspired general computing framework, particularly important for solving non-linear problems with reduced formalization and structure. At the same time, methods from the area of complex systems and non-linear dynamics have shown to be useful in the understanding of phenomena in brain activity and nervous system activity in general (Freeman, 1992; Kelso, 1995). Joining these two areas, the development of artificial neural networks employing rich dynamics is a growing subject in both arenas, theory and practice. In particular, model neurons with rich bifurcation and chaotic dynamics have been developed in recent decades, for the modelling of complex phenomena in biology as well as for the application in neuro-like computing. Some models that deserve attention in this context are those developed by Kazuyuki Aihara (1990), Nagumo and Sato (1972), Walter Freeman (1992), K. Kaneko (2001), and Nabil Farhat (1994), among others. The following topics develop the subject of Chaotic Neural Networks, presenting several of the important models of this class and briefly discussing associated tools of analysis and typical target applications.
TopIntroduction
Artificial Neural Networks have proven, along the last four decades, to be an important tool for modelling of the functional structures of the nervous system, as well as for the modelling of non-linear and adaptive systems in general, both biological and non biological (Haykin, 1999). They also became a powerful biologically inspired general computing framework, particularly important for solving non-linear problems with reduced formalization and structure. At the same time, methods from the area of complex systems and non-linear dynamics have shown to be useful in the understanding of phenomena in brain activity and nervous system activity in general (Freeman, 1992; Kelso, 1995). Joining these two areas, the development of artificial neural networks employing rich dynamics is a growing subject in both arenas, theory and practice. In particular, model neurons with rich bifurcation and chaotic dynamics have been developed in recent decades, for the modelling of complex phenomena in biology as well as for the application in neuro-like computing. Some models that deserve attention in this context are those developed by Kazuyuki Aihara (1990), Nagumo and Sato (1972), Walter Freeman (1992), K. Kaneko (2001), and Nabil Farhat (1994), among others. The following topics develop the subject of Chaotic Neural Networks, presenting several of the important models of this class and briefly discussing associated tools of analysis and typical target applications.
TopBackground
Artificial Neural Networks (ANNs) is one of the important frameworks for biologically inspired computing. A central characteristic in this paradigm is the desire to bring to computing models some of the interesting properties of the nervous system such as adaptation, robustness, non-linearity, and the learning through examples.
When we focus on biology (real neural networks), we see that the signals generated in real neurons are used in different ways by the nervous system to code information, according to the context and the functionality (Freeman, 1992). Because of that, in ANNs we have distinct model neurons, such as models with graded activity based on frequency coding, models with binary outputs, and spiking models (or pulsed models), among others, each one giving emphasis to different aspects of neural coding and neural processing. Under this scenario, the role of neurodynamics is one of the target aspects in neural modelling and neuro-inspired computing; some model neurons include aspects of neurodynamics, which are mathematically represented through differential equations in continuous time, or difference equations in discrete time. As described in the following topic, dynamic phenomena happen at several levels in neural activity and neural assembly activity (in internal neural structures, in simple networks of interacting neurons, and in large populations of neurons). The model neurons particularly important for our discussion are those that emphasize the relationship between neurocomputing and non-linear dynamical systems with bifurcation and rich dynamic behaviour, including chaotic dynamics.
TopNeurocomputing And The Role Of Rich Dynamics
The presence of dynamics in neural functionality happens even at the more detailed cellular level: the well known Hodgkin and Huxley model for the generation and propagation of action potentials in the active membrane of real neurons is an example; time dependent processes related to synaptic activity and the post synaptic signals is another example. Dynamics also appears when we consider the oscillatory behaviour in real neurons under consistent stimulation. Additionally, when we consider neural assemblies, we also observe the emergence of important global dynamic behaviour for the production of complex functions.
Key Terms in this Chapter
Bifurcation and Diverse Dynamics: The concept of bifurcation, present in the context of non-linear dynamic systems and theory of chaos, refers to the transition between two dynamic modalities qualitatively distinct; both of them are exhibited by the same dynamic system, and the transition (bifurcation) is promoted by the change in value of a relevant numeric parameter of such system. Such parameter is named “bifurcation parameter”, and in highly non-linear dynamic systems, its change can produce a large number of bifurcations between distinct dynamic modalities, with self-similarity and fractal structure. In many of these systems, we have a cascade of numberless bifurcations, culminating with the production of chaotic dynamics.
Artificial Neurons / Model Neurons: Mathematical description of the biological neuron, in what respects representation and processing of information. These models are the processing elements that compose an artificial neural network. In the context of chaotic neural networks, these models include the representation of aspects of complex neurodynamics.
Stability: The study of repellers and attractors is done through stability analysis, which quantifies how infinitesimal perturbations in a given trajectory performed by the system are either attenuated or amplified with time.
Chaotic Model Neurons: Model neurons that incorporate aspects of complex dynamics observed either in the isolated biological neuron or in assemblies of several biological neurons. Some of the models with complex dynamics, mentioned in the main text of this article, are the Aihara’s model neuron, the Bifurcation Neuron proposed by Nabil Farhat, RPEs networks, Kaneko’s CMLs, and Walter Freeman’s K Sets.
Attractors, Repellers and Limit Cycles: These three concepts are related to the concept of dynamic modality and they regard the long-term behaviour of a dynamical system. Attractors are trajectories of the system state variable that emerge in the long-term, with relative independence with respect to the exact values of the initial conditions. These long-term trajectories can be either a point in the state space (a static asymptotic behaviour), named fixed-point, a cyclic pattern (named limit cycle), or even a chaotic trajectory. Repellers correspond, qualitatively speaking, to the opposite behaviour of attractors: given a fixed-point or a cyclic trajectory of a dynamic system, they are called repeller-type trajectories if small perturbations can make the system evolve to trajectories that are far from the original one.
Spatio-Temporal Collective Patterns: The observed dynamic configurations of the collective state variable in a multi neuron arrangement (network). The temporal aspect comes from the fact that in chaotic neural networks the model neurons’ states evolve in time. The spatial aspect comes from the fact that the neurons that compose the network can be viewed as sites of a discrete (grid-like) spatial structure.
Chaotic Dynamics: Dynamics with specific features indicating complex behaviour, only produced in highly non-linear systems. These indicative features, formalized by the discipline of “Theory of Chaos”, are high sensibility to initial conditions, non-periodic behaviour, and production of a large number of different trajectories in the state space, according to the change of some meaningful parameter of the dynamical system (see bifurcation and diverse dynamics ahead). For some of the tools related to chaotic dynamics, see the related topic in the main text: Non-linear dynamics tools.