Creating and Analyzing Induced Decision Trees From Online Learning Data

Introduction

At their most simple form, a decision tree is a branching structure that starts with an originating root node at the top. From that node, there are two or more branches that emanate from that node, and the branches show respective possible paths that may be taken from that starting node. In general, the paths are one-directional, from the root node through the respective branches towards the leaves (Figure 01).

Defined by function, decision trees are branching structures that are used in various ways. Manually drawn decision trees are read top-down, and they may be understood as models of decisions or decision paths that people may take related to a particular in-world phenomenon. At each decision juncture, there is at least a choice between two or more options, and the choices have repercussions in terms of follow-on choices and ultimate outcomes. Such decision-based sequences help capture dependencies. They are reductive, sparse, simple, and parsimonious, so as to enhance human thinking about a complex context and to increase the transferability of such decision trees to other contexts. Visually, such manually drawn decision trees are 2D diagrams consisting of top-down or left-right flows, with objects and arrowed lines (like nodes and links, vertices and edges). Such manually drawn trees may be created for a variety of reasons:

• 1.

  To describe actual or theorized decision-making processes (and the implications of those decisions).

• 2.

  To highlight critical points in human decision-making (and decision junctures which may be highly important).

• 3.

  To introduce strategy in human decision-making (such as in drawing complete game trees to capture all possible moves in a context and all possible outcomes—with varying probabilities of outcomes; informed by game theory) (Dixit, Skeath, & Reiley, 1999, 2004, 2009, pp. 47 - 57).

• 4.

  To capture data patterns, and others.

The underlying data supporting the drawing of decision trees may be conceptual or theoretical. For example, a “what-if” decision tree could be created for simplified disaster scenarios and various choices individuals may face and possible likely outcomes for respective decisions. Or a theorized model may be express as a decision tree to describe what is conceptually predicted based on particular decision-making around events that have not yet occurred in the real but might. Decision trees may be based on empirical data that map to real-world events and real-world probabilities. Fully defined decision trees are often used in game theory to define strategy and optimal outcomes and probable outcomes as decisions are made. Another form of decision tree may be induced from data using any of a range of decision tree algorithms.

Figure 1.

Some basic parts of a shallow binary decision tree

This lesser-known form of decision tree involves those induced from data through machine learning (data mining, pattern identification). The first regression tree algorithm was created in 1963 (Loh, 2014), and in the intervening fifty-some years, decision trees (or “classification trees” or “regression trees”) have gone through many changes and improvements (Loh, 2014). In their classification application, decision trees may be used to identify relevant attributes that may predict labeled categories; in their regression application, decision trees may be used to identify partitions (or sets) based on continuous number outcomes. The induction of decision trees comes from the idea of “inductive learning,” or extracting understandings from evidence based on observed data patterns and interpreting those patterns. In induced decision trees, while patterns are surfaced, people play a critical role in interpreting and understanding the data.