Machine Learning as a Commonsense Reasoning Process

Introduction

One of the most important tasks in database technology is to combine the following activities: data mining or inferring knowledge from data and query processing or reasoning on acquired knowledge. The solution of this task requires a logical language with unified syntax and semantics for integrating deductive (using knowledge) and inductive (acquiring knowledge) reasoning.

In this paper, we propose a unified model of commonsense reasoning. We also demonstrate that a large class of inductive machine learning (ML) algorithms can be transformed into the commonsense reasoning processes based on well-known deduction and induction logical rules. The concept of a good classification (diagnostic) test (Naidenova & Polegaeva, 1986) is the basis of our approach to combining deductive and inductive reasoning.

The unique role of the good test’s concept is explained by the equivalence of the following relationships (Cosmadakis et al., 1986):

• •

  Functional/implicative dependencies between attributes/values of attributes;

• •

  Partition dependencies between classifications generated by attributes (attributes’ values) on a set of objects descriptions.

The task of inferring good diagnostic tests is formulated as the search for the best approximations of a given classification (a partitioning) on a given set of objects’ examples. It is this task that some well known ML problems can be reduced to (Naidenova, 1996): finding keys and functional dependencies in database relations, finding implicative dependencies and association rules, inferring logical rules (if-then rules, rough sets, and “ripple down” rules) and decision tree from examples, learning by discovering concept hierarchies, and some others.

The analysis of ML algorithms in the framework of good tests inferring and their decomposition to subtasks and elementary operations made it possible to see that they are the processes of interconnected deductive and inductive commonsense reasoning.

Background

There is not an exact definition of commonsense reasoning. This area of research covers a wide range of topics: default reasoning (Sakama, 2005), active agent’s reasoning (Thomason, 2007) and some others, for instance, qualitative reasoning, everyday thought about physical systems, spatial reasoning (Mueller, 2006).

Traditionally, commonsense reasoning is considered only as deduction (using knowledge). Induction of new knowledge from observations is considered in the framework of ML problems. That’s why many efforts are made in order to combine inductive and deductive reasoning. Two basic ways for this goal exist: 1) to aggregate into a whole system some well-known models of ML and deductive reasoning (Lavrac & Flash, 2000); 2) to enlarge the logic programming language to support both types of inference in a single formalism (Aragão, & Fernandes, 2004), (Sakama, 2005), (Galitsky et al., 2005), (Lisi, 2006). These approaches are very promising. However, the theoretical basis of these works is first-order predicate calculus with predicate as the main element of knowledge description while the main element of commonsense reasoning is concept.

We propose a model of commonsense reasoning based on processes of classification. Knowledge in this model is a system of coordinated links: objects ↔ classes of objects, classes of objects ↔ properties, objects ↔ properties. For instance, “all squares are rhombs”, “square is a rhomb”, “all the angles of rectangle are right”, “square is a rhomb all the angles of which is right”, “if the sun is in the sky and not raining, then the weather is good”, “conifers are pine-tree, fir-tree, cedar”. These connections have causal nature and can be formally expressed with the aid of implications. By commonsense reasoning we understand constructing and using the coordinated classification connections between objects, properties and classes. This understanding goes back to the work of Jean Piaget & Bärvel Inhelder (1959).