This section focuses on the pattern of connections between the units and the propagation of data. As for this pattern of connections, the main distinction we can make is between feedforward neural network and recurrent neural network. Below you first learn about the feedforward and then recurrent neural network. This section explains the fundamental of a feedforward neural network.
1.1 Definition
By considering the human brain to be an ‘ultimate’ neural network, then it is desirable to build a device, which imitates the brain's functions. However, we must settle for a simpler design because of the limits in our technology. The obvious method is to design an electronic device, and then using RLC networks to connect each neuron to many other neurons to imitate the dendrites, axons, and synapses. It is very difficult to ‘teach’ the network to do anything useful and this type of electronic model is still rather complex to implement. Further constraints are needed to make the design more manageable. First, we change the connectivity between the neurons so that they are in distinct layers, such that each neuron in one layer is connected to every neuron in the next layer. Further, we define that signals flow only in one direction across the network, and we simplify the neuron and synapse design to behave as analog comparators being driven by the other neurons through simple resistors. We now have a feedforward neural network model that may actually be practical to build and use (Krose & Smagt, 1996).
A “feedforward” neural network has an input layer that is connected to a hidden layer. This hidden layer can then be connected to another hidden layer or directly to the output layer. There can be any number of hidden layers so long as at least one hidden layer is provided. In common use, most neural networks will have only one hidden layer. It is very rare for a neural network to have more than two hidden layers. We will now examine, in detail, and the structure of a “feed forward neural network”.
There are some layers or subgroups of processing elements in each layered feedforward neural network. A layer of processing elements makes independent computations on data that it receives and passes the results to another layer. The next layer may in turn make its independent computations and pass on the results to yet another layer. Finally, the output of the neural network is a subgroup of one or more processing elements. The computation of each processing element is based upon a weighted sum of its inputs. The first layer is the input layer and the last the output layer. The layers that are placed between the first and the last layers are the hidden layers. The processing elements are similar to the neurons in a human brain, and hence, they are referred to as cells, neuromimes, or artificial neurons. Synapses between neurons are the connections between neurons by the edges of a directed graph. A threshold function is sometimes used to qualify the output of a neuron in the output layer. Even though our subject matter deals with artificial neurons, we will simply refer to them as neurons. (Hassoun, 1995).