Knowledge Growth in an

Artificial Animal

by Stewart Wilson

Patrick Tufts

Computer Science Department Brandeis University

http://www.cs.brandeis.edu/˜ zippy/papers/

February 22, 1995 Introduction Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion Intelligent behavior is to be repeatedly successful in satisfying one’s psychological needs in diverse, observably different situations on the basis of past experience.

–van Heerden

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion Model simple animals before complex.

Four characteristics of simple animals:

1. Many sense signals, only some of which are significant at any time

2. Animal acts in and effects environment

3. Some signals (food, pain) have special status for animal.

4. Animal acts to optimize rate of occurrence of special signals.

Introduction Definition of Intelligence

The Animat Model Implementation

Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion Environment 58 x 18 field with wraparound Components of a Classifier System:

• condition/action pairs (classifiers)

• set of inputs

• set of actions

• message system

• fitness/reward function

• discovery functions:

– crossover

– mutation

• reinforcement method: the bucket brigade Sensory Channels Starting at 12 o’clock and going clockwise, sense vector is:

bbbbTFFb This is further broken down to a binary representation.

2 bits per slot are sufficient to represent the 3 possible objects.

F: 11 T: 01 b: 00

So vector bbbbTFFb becomes:

00 00 00 00 01 11 11 00 Association Rules A sense vector and an associated action.

    00 00 11 00 #1 00 00 ##  /  move-east

Actions Move in any of 8 directions: north, northwest, west, ...

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm

Adaptation Algorithm

Experiments

Results Conclusion Performance Algorithm

• match set

• reinforcement algorithm • message list Animat execution cycle:

1. Create match set

2. Tax rules in match set

3. Pass tax on to rules that matched in last time step

4. Choose action from match set

5. Pass reward to acting rule(s)

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion Adaptation Algorithm genetic algorithm

• crossover

• mutation

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion An environment is a Markov decision process if:

1. different states have different features

2. a particular action in a given state always has same result

Woods 7 is a Non-Markovian environment

Agent needs to use internal state to know what to do

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments Results Conclusion Parameter Values arrived at by experiment

• 400 classifiers

• tax of 20% for activation

Introduction Definition of Intelligence

The Animat Model

Implementation Performance Algorithm Adaptation Algorithm

Experiments

Results Conclusion What’s good about classifier systems?

• They can generalize over states (unlike Qlearning)

• Learn about environment incrementally • require less memory and peak computation

than batch-learning methods

• Rules are symbolic, possible to point at knowledge in system What’s bad about classifier systems?

• Message list must be modified to be useful.

• Bucket brigade doesn’t work for chains of more than five rules.

Current work • Q-learning reinforcement • flexible representations • Restricted message lists • Higher level features • Expected outcomes