Hierarchical Function Approximation with a Neural Network Model

Hierarchical Function Approximation with a Neural Network Model

Luis F. de Mingo (Universidad Politécnica de Madrid, Spain), Nuria Gómez (Universidad Politécnica de Madrid, Spain), Fernando Arroyo (Universidad Politécnica de Madrid, Spain) and Juan Castellanos (Universidad Politénica de Madrid, Spain)
Copyright: © 2012 |Pages: 14
DOI: 10.4018/978-1-4666-0261-8.ch027


This article presents a neural network model that permits to build a conceptual hierarchy to approximate functions over a given interval. Bio-inspired axo-axonic connections are used. In these connections the signal weight between two neurons is computed by the output of other neuron. Such arquitecture can generate polynomial expressions with lineal activation functions. This network can approximate any pattern set with a polynomial equation. This neural system classifies an input pattern as an element belonging to a category that the system has, until an exhaustive classification is obtained. The proposed model is not a hierarchy of neural networks, it establishes relationships among all the different neural networks in order to propagate the activation. Each neural network is in charge of the input pattern recognition to any prototyped category, and also in charge of transmitting the activation to other neural networks to be able to continue with the approximation.
Chapter Preview


Concepts and categories are being objects to study in practical Psychology and in Artificial Intelligence (AI) years ago. It deals with the research of how the knowledge and meaning are represented in the memory (Tulving, 1972; Wang 2003, 2008a, 2008b). Why are concepts important? Concepts or conceptual categories are stable representations, stored in the memory, that permit to treat different samples as members of a same class; without them we will be slaves of particular (Wallace, Silverstein, Bluff, & Pipingas, 1994).

From a psychologist point of view, when dealing with objects, situations and actions as members of conceptual categories, the perceived reality is mentally being divided into different groups. A structure is imposed to the world, dividing mentally the reality (Quillian, 1968; Tulving, 1972). Categories seem to be ruled by the cognitive economy principle, this is, to extract essential things from the known world and to represent them in the most economical way in order to minimize the cognitive effort.

According to Collins and Quillian (Collins & Quillian, 1969; Quillian, 1968), the importance of a concept is not the concept by itself; it is determined by the set of relationships that are established with other different concepts. These relationships can be of two types: Subset and Property. The former type expresses the specialization of some concepts into others; it produces the categorization, allowing to mentally structuring the perceived reality.

What could be added from a connectionist point of view to the concept representation in memory? (Arroyo, Castellanos, Luengo, & Mingo, 1997). Why could not it be thought that the concepts are supported by some small sets of neural assemblies? These ones form a mix of specialized neural networks concerning the samples recognition task and they determined if a sample belongs or is classified into a concept.

Concepts are not isolated into the human cognitive system. They are immersed into a hierarchical structure that facilitates, among others, classification tasks to the human cognitive system. This conceptual hierarchy expresses a binary relationship of inclusion defined by the following criterions:

  • Inclusion criterion: Each hierarchy node determines a domain included into domain of its father node. Each hierarchy node determines a domain that includes every domain of its son nodes.

  • Generalization and Specialization criterion: Every node in the hierarchy, has differentiating properties that make it different from its father node, if it exists, and from the others son nodes of its father, if they exist.

This Inclusion relation makes possible to keep the maximal economy criterion in the conceptual hierarchy. The defined inclusion relation makes also possible to represent a conceptual hierarchy as an acyclic directed graph in which nodes represent concepts and edges the inclusion relation among them. Edges connecting nodes establishes that the father nodes are situated in the upper hierarchical level of its son nodes. Obviously, concepts are generated in a natural manner and it will not be constrained by these formal restrictions. However, many of the classifications learnt during our school years are built using these rules. In this article, we will focus our work in this kind of hierarchies, which are learnt with a teacher or in a supervised manner.

Next sections deal with the representation of any conceptual hierarchy using a set of neural networks, permitting the complete or exhaustive classification of an input. Exhaustive classification means that an input belongs to each one of the supported concepts by the system and to their specializations. The hierarchical architecture is controlled using neural network outputs, that is, depending on the output a different neural network is activated.


Problem Description

Let D domain of any samples. Let H a conceptual hierarchy to the elements of the domain D. Let T the tree of the conceptual hierarchy H. Each node in the tree T represents a concept in hierarchy H, and an arrow between two nodes implies that the end node is a specialization of the start node.

Complete Chapter List

Search this Book: