Machine Learning

Introduction

Machine learning was a founding branch of artificial intelligence in the late 1950s. However, it has become very attractive, especially recently, thanks to the Expert Systems (ES), Knowledge Discovery from Databases (KDD), Data Mining (DM), and their developments and applications. This article covers the following topics:

• •

  Basic definitions of machine learning.

• •

  Formal definition of empirical concept learning from data.

• •

  Understanding learning as search or as approximation.

• •

  Description of basic forms of selected learning algorithms.

Background

Human learning is one of the most important characteristics of human intelligence. In a similar way, machine learning (ML) is one of the most significant fields of artificial intelligence that attracts the interest of researchers for decades. Some researchers even argue that learning is so essential for intelligence, there can be no artificial intelligence without learning abilities.

There are numerous definitions of machine learning, most of them relating learning to the ability to improve a performance of a system over time:

Learning deals with “agents that can improve their behavior trough diligent study of their own experiences.” (Russell, 2009, p. 649)

We can distinguish two basic types of learning: concept learning and skill acquisition. Concept learning consists of inferring and assimilating new material composed of concepts, general laws, procedures, etc. The knowledge acquired should be expressed in a formalized description that allows us to solve a problem, explain a situation, predict behavior, etc. Refinement of skills through practice consists of gradually correcting deviations between observed and desired behavior through repeated practice. So an example of concept learning can be a learning process that results in describing the concept “dog” (as a hairy animal with four legs that barks) while an example of skill refinement can be learning to drive a car or play a piano.

Based on the feedback available during learning, we can distinguish different forms of learning. Reinforcement learning, where the teacher observes the learning system (typically a robot) and gives his feedback to the system in the form of rewards (when learning correctly) or punishments (when learning incorrectly), is the form of learning used for skill refinement. The other forms of learning referred here are used for concept learning:

• •

  Supervised Learning: Where pre-classified examples are available for learning and the task is to build a (classification or prediction) model that will work on unseen examples,

• •

  Unsupervised Learning: Where there is no feedback at all (this is typical for clustering and segmentation tasks)

• •

  Semi-Supervised Learning: Falls between supervised and unsupervised learning; it uses a small number of pre-classified examples and a large number of examples without known class,

• •

  Active Learning: Is a form of supervised learning, where the algorithm can query the teacher for class membership of unclassified examples,

• •

  Apprenticeship Learning: Where the system uses indirect hints derived from the teacher´s behavior, is also a form of supervised learning.

We can also distinguish between analytic and empirical learning. In analytic learning the system formulates a generalization by observing only a single example (or absent any examples at all), and by exploiting an extensive background knowledge about the given domain. In empirical learning, the generalization is based on a huge amount of data (examples) and only very limited (if any) background knowledge.