Abstract for tham_apwamc93

Presented at the IEEE Asia-Pacific Workshop on Advances in Motion Control 26-27 July 1993, Singapore

REINFORCEMENT LEARNING METHODS FOR MULTI-LINKED MANIPULATOR OBSTACLE AVOIDANCE AND CONTROL

Chen K. Tham & Richard W. Prager

March 1993

This paper treats the multi-linked manipulator obstacle avoidance and control task as the interaction between a learning agent and an unknown environment. The role of the agent is to generate actions that maximises the reward that it receives from the environment, i.e. when the goal of reaching the destination without collision with obstacles is achieved. We demonstrate how two learning algorithms common in reinforcement learning literature: Adaptive Heuristic Critic (AHC) (Barto et al, 1983) and Q-Learning (Watkins, 1989), can be used to solve the task successfully in two different ways: firstly, through the generation of position commands to a PD controller which produces the torque commands to drive the manipulator, and secondly, through the direct generation of torque commands, removing the need for a PD controller. During the process, the inverse kinematics problem for multi-linked manipulators is automatically solved. Fast function approximation is achieved through the use of an array of Cerebellar Model Arithmetic Computers (CMAC) (Albus, 1975). The generation of both discrete and continuous actions are investigated and the performance of the algorithms in terms of learning rates, efficiency of solutions, and memory and computation requirements are evaluated.

Keywords: Reinforcement Learning; Machine Learning; Adaptive Systems; Neural Control


(ftp:) tham_apwamc93.ps.Z (http:) tham_apwamc93.ps.Z
PDF (automatically generated from original PostScript document - may be badly aliased on screen):
  (ftp:) tham_apwamc93.pdf | (http:) tham_apwamc93.pdf

If you have difficulty viewing files that end '.gz', which are gzip compressed, then you may be able to find tools to uncompress them at the gzip web site.

If you have difficulty viewing files that are in PostScript, (ending '.ps' or '.ps.gz'), then you may be able to find tools to view them at the gsview web site.

We have attempted to provide automatically generated PDF copies of documents for which only PostScript versions have previously been available. These are clearly marked in the database - due to the nature of the automatic conversion process, they are likely to be badly aliased when viewed at default resolution on screen by acroread.