Section 1. Introduction

Early models of procedural learning assumed actors were isolated, model-based thinkers. More recently, learning techniques have become more sophisticated as this assumption has been replaced with less restrictive ones. To date, however, there has been no thorough investigation of multiple, heterogeneous, situated agents who learn from the pragmatics of their domain rather than from a model. In this paper, we develop techniques that allow agents to improve their performance in a dynamic environment by using past run-time behavior to learn procedures to better coordinate their actions. These techniques are based upon a structure called collective memory.

This paper begins with general overviews of collective memory and MOVERS-WORLD, our test-bed domain. The heart of the paper is an exposition on how cooperative procedures are added to collective memory. The techniques which agents use to learn cooperative procedures from their run-time behavior are outlined and applied to a simple example. Reasoning from execution traces is a key aspect of the system because it enables agents to learn plans beyond the scope of the scratch planner. Finally, results are presented that clearly show the effectiveness of collective memory in improving community performance.