System and Circuit Design for Biologically-Inspired Intelligent Learning

System and Circuit Design for Biologically-Inspired Intelligent Learning

Turgay Temel (Bahcesehir University, Turkey)
Indexed In: SCOPUS
Release Date: October, 2010|Copyright: © 2011 |Pages: 412|DOI: 10.4018/978-1-60960-018-1
ISBN13: 9781609600181|ISBN10: 1609600185|EISBN13: 9781609600204|ISBN13 Softcover: 9781609600198
Hardcover:
Available
$180.00
TOTAL SAVINGS: $180.00
Benefits
  • Free shipping on orders $395+
  • Printed-On-Demand (POD)
  • Usually ships one day from order
  • 20% discount on 5+ titles*
E-Book:
(Multi-User License)
Available
$180.00
TOTAL SAVINGS: $180.00
Benefits
  • Multi-user license (no added fee)
  • Immediate access after purchase
  • No DRM
  • ePub with PDF download
  • 20% discount on 5+ titles*
Hardcover +
E-Book:
(Multi-User License)
Available
$215.00
TOTAL SAVINGS: $215.00
Benefits
  • Free shipping on orders $395+
  • Printed-On-Demand (POD)
  • Usually ships one day from order
  • Multi-user license (no added fee)
  • Immediate access after purchase
  • No DRM
  • ePub with PDF download
  • 20% discount on 5+ titles*
OnDemand:
(Individual Chapters)
Available
$37.50
TOTAL SAVINGS: $37.50
Benefits
  • Purchase individual chapters from this book
  • Immediate PDF download after purchase or access through your personal library
  • 20% discount on 5+ titles*

Description

Despite significant research and studies in biologically-inspired circuit design, the capability of the biological creatures still excels that of artificially human-made systems inspired in terms of adaptivity, sensitivity, and spectral characteristics.

System and Circuit Design for Biologically-Inspired Intelligent Learning aims to bridge that gap in the belief that compact organization and arrangement of the circuit and system design will be a major factor in attaining the pursued benefits of biological systems. Including research from abstract fields such as psychology to more concrete topics including circuit design, this text is designed for even fresh reader to make a smooth transition from principles to system/circuit architectures which simulate learning intelligence.

Topics Covered

The many academic areas covered in this publication include, but are not limited to:

  • Analog Circuit Design with Fuzzy Inference
  • Biologically-inspired Learning and Intelligent System Modeling
  • DCGA Optimization Techniques
  • Implantable Neural Decoders
  • Neural Network Circuits
  • Neuromodeling and Natural Optimization
  • Parallel Hardware for Artificial Neural Networks
  • Recurrent Neural Networks
  • Representation of Neuro-information and Knowledge
  • Self-organizing Fuzzy Neural Network Training

Reviews and Testimonials

This is an excellent book. The many figures and formulas help clarify the text. ... If you are interested in system and circuit design in order to simulate intelligent learning, this is the book to have.

– Gary B Kaniuk, Psy.D., Cermak Health Services, Doody's Book Review

This book on system and circuit design for biologically-inspired intelligent learning is intended to give readers the background needed to make a successful transition from principles to system circuit architectures that simulate learning intelligence. It includes research from many diverse and abstract fields, as well as fundamental topics and current studies. Subjects addressed include: analog circuit design with fuzzy interface, biologically-inspired learning and intelligent system modeling, DCGA optimization techniques, implantable neural decoders, neural network circuits, recurrent neural networks, and representation of neuro-information and knowledge. ...

– Sci Tech Book News, BookNews.com

Biologically-inspired design is mainly concerned with thorough understanding of the behavioral and functional association of the system of interest. In the belief that compact organization and arrangement of the circuit and system design will be a major factor in attaining the pursued benefits of biological systems, the book is intended to contain fundamental topics as well as up-to-date studies for even fresh readers to make a smooth transition to system design from principles. ... We hope that the book will be a useful resource to those who wish to be familiar with intelligent systems and their implementation.

– Turgay Temel, Bahcesehir University, Turkey

Table of Contents and List of Contributors

Search this Book:
Reset

Preface

Most living biological organisms have developed particular capabilities such that they find viable and successful solutions to challenges and their needs. The development of those capabilities has been known in the form of experiences repeated long enough to reproduce, i.e. trial and error. They simply replicate experimental outcomes in the form of codes into genes and pass the associated information to the next generation. The process of learning the most successfully fitting outcome and replicating it can be investigated within the scope of various scientific fields. Moreover, such a process occurs in the scopes ranging from nano, e.g. bacteria, viruses and other cell-based structures to macro, e.g. social structure, (Brabazon, & O’Neill, 2006). 

The learning capability and relevant process that biological organisms with a certain degree of intelligence have has inspired humans to devise new methodologies, techniques and systems (Samsonovich, & Muller, 2008), which are attributed to by the phrase biologically-inspired systems. Some other terms which have been used commonly in place as substitutions are, bionics (Kodogiannis, Lygouras, Tarcynzki, & Chowdrey, 2008; Vincent, 2001) and biomimetics, (Cohen, 2006). Although it would be almost impossible to count otherwise examples which would be observed, a comprehensive list of biological mechanisms and inspired system design examples with relevant application and literature details are provided in (Cohen, 2006):

  • A very common example of biological inspiration can be given from flying creatures which benefit from their excellent aerodynamics. The body shape of these creatures allows them to fly with superior maneuvering capability. The aircraft systems have been benefiting from these biological systems.
  • A well-known and yet to develop further scientific example is of bats, which do not see well and are almost blind. As known, bats can fly in darkness without colliding into any obstacle with the help of their amazing hearing system. Their ear shape differs for differing species. The shape of the bat pinnae causes adaptable directivity patterns which interact with the ear itself. They can direct their ears all directions for localizing the sound sources and recognizing their prey in mixed environments. With their developed skills in navigating and hunting capabilities, they have been a major inspiration of biologically-inspired systems, particularly robots, which emulate landmark recognition with navigation and classification of echo sources.
  • Rats exploit their own whiskers to avoid collision and find food. Whiskers are highly sensitive to touching of environmental objects. Hence, emulating whiskers in rodents can be an inspiration to develop as sensory mechanisms to be used with collision-avoidance robotic systems. 
  • Elephants can feel and recognize the acoustic wave emitted by another elephant when it rocks its feet on the ground at long distances, even several kilometers. The hyper-low frequency sound emitted by a whale and transmitted over long distances in the ocean can be recognized by other whales. The mechanism which governs the low-frequency sound wave transmission along the earth and its surface, which has been inspired by these animals, has been used in accelerometers for detection of earthquakes. They have also been used by the sonar systems in submarines. 
  • Spiders produce silky fiber web which is even tougher than those materials used in bullet-proof vests and stronger than the steel. The fiber web is so sticky that it functions as trap for catching preys. Nano-technology is promising possibilities for making enduring, continuous fibers with use of an electrospinning technique. Velcro which was inspired by seeds that stick to an animal’s fur has allowed development in various fields’ textile and electric-wires strapping. 
  • Besides animals, plants also suggest a model for biological inspiration. For example the sunflower directs itself to follow the sun throughout the day for maximum sunlight exposure, which has been an inspiration in developing a cheap solar energy production mechanism with optimality. 

Despite a great volume of research and studies in academia and other disciplines with highly successful adaptation into daily life and available technologies, the capability of the biological creatures still excedes the artificially human-made systems that they inspired in terms of adaptivity, sensitivity, and spectral characteristics, (Kodogiannis, Lygouras, Tarcynzki, & Chowdrey, 2008). However, scientific approaches which resort on biologically-developed mechanisms, have made it possible to comprehend the phenomena of interest in biological inspiration and associated principles to devise novel systems and improve their capability, particularly in electronics, or to be more precise, silicon microstructures. 

The response of a biological organism to a particular stimulus is mainly determined by the cells which make up associated subsystem to process it. Therefore, it is of prime importance to understand the mechanism and structural functionality demonstrated by the cell structure of associated biological subsystem so it would be possible to realize associated self-assembly silicon circuits, (Saeedi, Kim, & Parviz, 2008). Recently, a considerable effort has exploited micro-/nano- electromechanical systems (MEMS/NEMS) toward realization of cell-based structures, (Tahir, & Aubert, 2010). 

It should be stated that in the context of biological inspiration, main design objective generally aims at emulating capabilities of the biological origin in decision making developed on natural feedback for each individual experience. While available design approaches aim at attaining much more diversity in applicable biological fields, those disciplines which prevail intelligence and adaptivity are gaining favor. The biological systems, particularly human, with the capabilities of perception, learning and reasoning with development of reasoning and mind relay vast opportunities for achieving such overwhelming and natural albeit complex structures. How human brain as the main processing unit in the nervous system achieves grasping discernible features of external stimuli was postulated long ago, (Nielsen, 2006). It has been well-publicized that such a system is composed of highly complex information processing units operating in immense parallelism with capability of adapting itself with experience. With this capability of brain, human is able to discriminate a familiar person and associated features in crowd; recognize a blurred image as the original; read writings unseen before etc. Amazing capabilities of nervous system and the brain have been a major inspiration to develop intelligently operating architectures, such as circuits and systems, (Derakshani, & Schuckers, 2002).  

Concerning cell-based design, it has been known that major issue for building silicon architectures is to a neural model which transforms the external stimuli into coded quantity, transmitted from sensory neurons to nervous system, mainly the brain, and then to the motor system which actuates the motor commands as a response. Codified patterns between these systems are in the form of spiking impulses, which can be considered as digital words, (Dayan, & Abbot, 2001). However, slow transmission of biological spikes which carry information to be processed by brain makes it possible to exploit the faster conventional electronics to emulate the connectivity of biological networks. It is known that almost 80% of brain energy is consumed for the transmission of impulses along nerve fibers. In low-power artificial implementation of biological counterparts, it is advantageous to adopt analogue design methodologies rather than digital approach, (Dugger, 2003). Analog design readily facilitates major arithmetic operations such as addition, subtraction, exponentiation and integration. Having realization with a pre-specified threshold has allowed to build sparse-coded decision making based on so-called integrate-and-fire (IF) neurons. Design approach with IF-neuron model established the discipline called neuromorphic engineering, (Mead, 1989; Eliasmith, & Anderson, 2003). In most neuromorphic circuit designs, active elements, i.e. the transistors, operate in sub threshold region where they simulate the cell membranes of real neurons while voltage- and time-dependent current flows of real ion channels are emulated with external circuitries. 

Another major objective of the circuit design with biological inspiration is to introduce implantable replacements for those who have lost in sensing, such as silicon retina for the blind, (Boahen, 2005), and cochlea for the deaf, (Sit, & Sarpeshkar, 2008). From an implementation point of view, most artificial systems differ in terms of the design aspects of the sensing stage and relevant interface which transform the stimuli into electrical spike patterns. Mostly, the source identification and associated activity units are implemented as embedded software to fulfill an artificial neural network (ANN) algorithm, (Zhang, Sitte, & Rueckert, 2005). Despite considerable studies on other sensory systems, (Cho, et al, 2005; Rulcker, Stenberg, Winquist, & Lundstrom, 2000), outcomes are not mature enough to yield system design with microcircuit realization. Two well-known sensory systems, i.e. vision and hearing, are the two which have been progressed for biologically-inspired microcircuit implementation: 

Some well-known major circuit realizations of retina for emulating biological vision system are proposed in (Zaghloul, & Boahen, 2006; Santos, Gotarredona, & Barranco, 2007). They resort on the visual sensory system model where a group of photoreceptors reflect electrical patterns which represent light intensity difference in the sector of the visual field. The patterns are then conveyed to over a million of ganglion cells as coded spikes depending on the relative change in light intensity. In a silicon version, the current between different locations of the silicon chip is controlled by the voltage produced by photoreceptors and the ion channels of optical ganglion cell neurons are excited or inhibited and controlled by active feedback components, e.g. transistors. A digitally programmed interface is usually employed for converting the analog signal so obtained into digital form and then processing it conveniently as well as providing necessary events addressing and controlling signals, (Stocker, 2006).

Since the prominent study in (Lyon, & Mead, 1988) which reveals analog realization of cochlea, there have been a large number of attempts to replicate auditory system depending on the developed models, (Kuszta, 1998; Hamilton, 2008). Generally, it is accepted that the overall cochlea response can be modeled as a filter bank operation which allocates the incoming sound filtered by the outer ear into a number of channels, e.g. gamma tone filter bank model, (Patterson, 1994). Each channel exhibits spectral response mainly determined by spatial features across the cochlea. The output of the first stage is delivered to spike generation. The source of the sound is identified with use of the coded spike ensembles.

Biologically-inspired design is mainly concerned with thorough understanding of the behavioral and functional association of the system of interest. In the belief that compact organization and arrangement of the circuit and system design will be a major factor in attaining the pursued benefits of biological systems, the book is intended to contain fundamental topics as well as up-to-date studies for even fresh readers to make a smooth transition to system design from principles. Bearing this in mind, the chapters and their scope are organized in a hierarchical manner from abstract fields such as psychology to more concrete topics including circuit design as follows:

Chapter 1 is concerned with evolution of skill and preference development processes in time. It focuses on the impact of literature and experience on learning with competence and relevance to development of models in learning. It has been extended to adaptive learning. 

Chapter 2 discusses the correlative interactions between learning components. The author presents detailed neuro-informative background concerning how dynamics in learning is achieved with cognition. Symbolic and expressive embodiment with cognitive information processing toward learning is discussed. Formation of knowledge with underlying reasoning consisting in overall dynamics of neural architecture is shown to be the main factor for grounded generalization. Temporal and local connections form an integrated mechanism with grounding which leads symbols to be exploited as embedding representations globally processed dynamically. The chapter investigates productivity in learning resulting from interactions with environment and experience.

Chapter 3 attempts to reveal prominent associative models in learning namely conditional and visual learning as well as propositional model. These learning models are also investigated with relationship to understanding and reasoning with neural architectures for association, which is the major source for learning paradigms.

Chapter 4 introduces a thorough overview of artificial neural systems and learning.  Relevant bio-physiological aspects are explained in a hierarchy, i.e. functional transition from neural cells to brain. The main objective of the chapter is to get the reader familiar with the principles of artificial mechanism inspired biologically which pursues learning for decision making given an appropriately arranged input. Major learning algorithms which can be come up with biological organisms are specified in terms of generalization capability so that they can be embedded into a system design. The chapter also presents an application of a well-known learning algorithm to emulate biological olfaction. 

Chapter 5 describes a new method toward optimality-oriented stability for deterministic and stochastic neural networks. The approach proposed in the chapter attains a solution of Hamilton-Jacobi equation for nonlinear systems, which is involved in recurrent neural networks by using Lyapunov technique, inverse optimality and differential game theory. The chapter also exemplifies the proposed method.

Chapter 6 focuses on noisy modeling recurrent neural networks for global robustness. The algorithm is developed with use of Hamilton-Jacobi-Isaac equation as well as Lyapunov technique, differential minimax theory and the inverse optimality similar to the previous chapter where each input vector is considered a player against the noise-built player vector. The algorithm is shown to achieve global robustness given some predefined cost functions.

Chapter 7 is mainly concerned with building an evolving adaptive neural network which grants biologically-plausible coded spike populations to represent taste recognition system. Taste is a relatively neglected sensation with respect to vision and hearing. For this reason, it is expected that the study has importance to those who desire to study in this research field. 

Chapter 8 presents some scenarios to augment the learning performance and computational efficiency performance of well-known self-organizing fuzzy neural network (SOFNN). Modified network architecture is partly traced while progress of training through the use of a record which keeps the firing strength of the neurons involved in training stage. A new improved-performance SOFNN is also proposed with new feedback connections at a particular layer. Owing to feedback connections which can be added/removed, it is shown that new recurrent network is able to learn temporal behavior of the input. 

Chapter 9 deals with approximate reasoning and inference based on fuzzy logic and their circuit realization. The fuzzy logic has been a principle expert system tool and exploited in various disciplines such as control and machine learning. Based on the axiomatic norms and rules, fuzzy logic yields considerably simple expressions which can be straightforwardly realized in circuits. It should be noted that the norms being used in fuzzy logic provide a mechanism valid for most classification algorithms, fuzzy logic design is an elegant solution to most biologically-inspired system and circuit design. The chapter finalizes its argument by introducing a fuzzy classifier circuit based on concurrent use of complementary norm circuits recently brought into literature.

Chapter 10 proposes an ANN-based biomimetic algorithm for decoding neural cell ensemble signals. It also proposes a computationally efficient, low-power CMOS analog circuit which can be used in real-time neuromorphic signal processing. The decoding algorithm implements a continuous-time ANN utilizing adaptive linear filter bank represented by kernels to simulate synaptic dynamics such as local field potentials and spike rates where the filters transform neural signal inputs into control-parameter outputs. As the realization of the algorithm, the proposed circuit is able to learn with active elements operating in sub threshold region. The experimental results verify that the proposed architecture operates as predicted for the applications considered. 

Chapter 11 presents a digital FPGA-based coprocessor implementation of a recurrent neural network to emulate unsupervised learning with a large number of neurons involved. The architecture is able to operate process-and-store based to simulate a connectivity sparse matrix to represent association given an input database, which employs and is conditioned on state- and activity vectors. 

Chapter 12 is concerned with implementation and performance issues of programmable analogue ANNs for sensor conditioning in embedded systems. The chapter also investigates the effect of weight selection on performance for a given task such as training by using perturbation algorithm and associated tuning. 

Chapter 13 illustrates a digital implementation of ANN architecture with dedicated units operating in fixed-point mathematics. For a fast operation, the overall architecture brings in simplicity and use of mere combinational circuitry. The proposed hardware indicates relevant software which provides the sequence of hardware computation, inputs, weights and biases. 

Chapter 14 exploits diversity-controlled (DC) GA for rapid development and optimization of frequency-response masking (FRM) digital filters which incorporates BIBO-stable bilinear lossless discrete-integrator (LDI) IIR filter. Use of DCGA facilitates simultaneous optimization of the magnitude-frequency as well of the group-delay frequency response of the desired FRM digital filters. 

Chapter 15 applies ANN paradigm to obtain the electromagnetic model of nonlinear devices and circuits while it optimizes them by using genetic algorithm (GA) with particle swarm optimization (PSO). The ANN being used is of single hidden layer multilayer perception (MLP) trained with back propagation algorithm. The chapter also covers applications of the proposed scheme.

We hope that the book will be a useful resource to those who wish to be familiar with intelligent systems and their implementation. 


REFERENCES
Boahen, K. (2005). Neuromorphic Microchips- Compact Electronics Based on Neural Wiring Could Yield Implantable Silicon Retinas to restore Vision, as well as Robotic Eyes and other Smart Sensors. Scientific America Perspectives, 292(5), 56-63.

Brabazon, A., & O’Neill, M. (2006). Biologically-inspired Algorithms for Financial Modeling, Berlin, Germany: Springer.

Cho, S. M., et al. (2005). Monolithic Electronic Nose System Fabricated by Post CMOS Micromachining. Proceedings of IEEE on Sensors, 420-423.

Cohen, Y. B. (2006). Introduction to Biomimetics: The Wealth of Inventions in Nature as an Inspiration for Human Innovation. In Y. B. Cohen (Ed.). Biomimetics: Biologically-inspired Technologies, (pp. 1-40), Boca Raton, FL: Taylor & Francis.

Dayan, P., & Abbot, L.F.  (2001). Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: MIT Press.

Dugger, J. (2003). Adaptive Analog VLSI Signal Processing and Neural Networks. PhD Dissertation.

Eliasmith, C., & Anderson, C. H. (2003). Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems. Cambridge, MA: MIT Press.

Derakshani, R., & Schuckers, S. A. C. (2002). Biologically Inspired Evolutionary Temporal Neural Circuits. In Proceedings of IEEE International Joint Conference on Neural Networks, (IJCNN), 1357-1361.

Hamilton, T. J. (2008). Analogue VLSI Implementations of Two Dimensional, Nonlinear, Active Cochlea Models, PhD Thesis.

Kodogiannis, V. S., Lygouras, J. N., Tarcynzki, A., & Chowdrey, H. S. (2008). Artificial Odor Discrimination System Using Electronic Nose and Neural Networks for the Identification of Urinary Tract Infection. IEEE Trans. Information Technology in Biomedicine, 12(6), 707-713. 

Kuszta, B. (1998). Silicon Cochlea. In Proceedings of IEEE WESCON, 282-285.
Lyon, R. F. & Mead, C. (1988). An Analog Electronic Cochlea. IEEE Transactions on Acoustics, Speech, and Signal Processing, 36, 1119-1134.

Mead, C. (1989). Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley.

Nielsen, R. H. (2006). Mechanization of Cognition. In Y. B. Cohen (Ed.). Biomimetics: Biologically-inspired Technologies, (pp. 58-125), Boca Raton, FL: Taylor & Francis.

Patterson, R. D. (1994). The Sound of a Sinusoid: Spectral Models. Journal of American Society of Acoustics, 96(3), 1409-1418.

Saeedi, E., Kim, S., & Parviz, B. A. (2008). Self-assembled Crystalline Semiconductor Optoelectronics on Glass and Plastic. Journal of Micromechanics and Microengineering, 18, 1-7.

Samsonovich, A. V., & S. T. Muller. (2008). Toward a Growing Computational Replica of the Human Mind. Proceedings of AAAI Symposium on Biologically-inspired Cognitive Architectures, 1-4.

Santos, J. C., Gotarredona, T. S.-, & Barranco, B. L. (2007).  A Spatial Contrast Retina with on-chip Clibration for Neuromorphic Spike-based AER Vision Systems. IEEE Transactions on Circuits Syst. I, Regular Papers, 54(7), 1444–1458.

Sit, J. J. & Sarpeshkar, R. (2008). A Cochlear-Implant Processor for Encoding Music and Lowering Stimulation Power. IEEE Pervasive Computing, 7(1), 40-48.

Stocker, A. A. (2006). Analog VLSI Circuits for the Perception of Visual Motion. New York: John-Wiley & Sons.

Tahir, F. A., & Aubert, H. (2010). Equivalent Electrical Circuit for Designing MEMS-controlled Reflect Array Phase Shifters. In Proceedings of Progress in Electromagnetics Research, (PIER), 100, 1-12.

Vincent, J.F.V. (2001). Stealing Ideas From Nature. (2001). In S. Pellegrino (Ed.), Deployable Structures. (pp. 51-58), Berlin, Germany: Springer-Verlag. 

Zhang, L., Sitte, J., & Rueckert, U. (2005). Local Cluster Neural Network Chip for Control. In Proceedings of the International Conference on Adaptive and Natural Computing Algorithms, 116-119.

Zaghloul, K. A., & Boahen, K. (2006). A Silicon Retina that Reproduces Signals in the Optic Nerve, Journal of Neural Engineering, 3(4), 123-135.

Author(s)/Editor(s) Biography

Turgay Temel, received his PhD in 2002 in the area of hybrid-mode circuit design based on multi-valued logic and higher radices. In the field, he just completed a state-funded project which aims at using analog-wise information processing with digital concepts as a microarchitecture. The project was the first in this field in literature to his knowledge. His research interest diversifies, which spans to machine learning, neuroscience, biomedical engineering, sensor networks, biologically-inspired system and circuit design and high-performance circuit design. Recently, he has been focusing on reconfigurable analog systems with the hope that major units of intelligence can be emulated in an evolvable manner. The sensor development for biological information processing and bio-sensor systems have been at the core of his current scope. He has also developed unsupervised learning, i.e. clustering, techniques and currently working on single-step methods for high-performance pattern recognition.

Indices