REINFORCEMENT LEARNING METHODS FOR MULTI-LINKED MANIPULATOR OBSTACLE AVOIDANCE AND CONTROL
Chen K. Tham & Richard W. Prager
March 1993
This paper treats the multi-linked manipulator obstacle avoidance and control task as the interaction between a learning agent and an unknown environment. The role of the agent is to generate actions that maximises the reward that it receives from the environment, i.e. when the goal of reaching the destination without collision with obstacles is achieved. We demonstrate how two learning algorithms common in reinforcement learning literature: Adaptive Heuristic Critic (AHC) (Barto et al, 1983) and Q-Learning (Watkins, 1989), can be used to solve the task successfully in two different ways: firstly, through the generation of position commands to a PD controller which produces the torque commands to drive the manipulator, and secondly, through the direct generation of torque commands, removing the need for a PD controller. During the process, the inverse kinematics problem for multi-linked manipulators is automatically solved. Fast function approximation is achieved through the use of an array of Cerebellar Model Arithmetic Computers (CMAC) (Albus, 1975). The generation of both discrete and continuous actions are investigated and the performance of the algorithms in terms of learning rates, efficiency of solutions, and memory and computation requirements are evaluated.
Keywords: Reinforcement Learning; Machine Learning; Adaptive Systems; Neural Control
If you have difficulty viewing files that end '.gz',
which are gzip compressed, then you may be able to find
tools to uncompress them at the gzip
web site.
If you have difficulty viewing files that are in PostScript, (ending
'.ps' or '.ps.gz'), then you may be able to
find tools to view them at
the gsview
web site.
We have attempted to provide automatically generated PDF copies of documents for which only PostScript versions have previously been available. These are clearly marked in the database - due to the nature of the automatic conversion process, they are likely to be badly aliased when viewed at default resolution on screen by acroread.