Monocular Vision System that Learns with Approximation Spaces

Abstract

This paper introduces a monocular vision system that learns with approximation spaces to control the pan and tilt operations of a digital camera that is tracking a moving target. This monocular vision system has been designed to facilitate inspection by a line-crawling robot that moves along an electric power transmission line. The principal problem considered in this chapter is how to use various forms of reinforcement learning to control movements of a digital camera. Prior work on the solution to this problem was done by Chris Gaskett using neural Q-learning starting in 1998 and more recently by Gaskett in 2002. However, recent experiments have revealed that both classical targets tracking as well as other forms of reinforcement learning control outperform Q-learning. This chapter considers various forms of the Actor Critic (AC) method to solve the camera movement control problem. Both the conventional AC method as well as a modified AC method that has a built-in run-and-twiddle (RT) control strategy mechanism is considered in this article. The RT mechanism introduced by Oliver Selfridge in 1981 is an action control strategy, where an organism continues what it has been doing while things are improving (increasing action reward) and twiddles (changes its action strategy) when past actions yield diminishing rewards. In this work, RT is governed by measurements (by a critic) of the degree of overlap between past behaviour patterns and a behavior pattern template representing a standard are carried out within the framework provided by approximation spaces introduced by Zdzislaw Pawlak during the early 1980s. This paper considers how to guide reinforcement learning based on knowledge of acceptable behavior patterns. The contribution of this article is an introduction to actor critic learning methods that benefit from approximation spaces in controlling camera movements during target tracking.