Tuning up the Novelty Engine

These results prompted a change in parameters in our novelty engine; we decided to reduce f, the size of the fixed part of the training set. Now the bulk of the fitness comes from the coevolving population itself. We keep the one all-time best as the single fixed training example.

The novelty engine has changed configurations twice:

• Initially (robots 1-739), f=15, MAXGEN=0,SEED=false.
  The population is reset every time the front end finishes a generation, and the best 15 against humans are used as trainers.
• Between robots 740 and 2539, f=15, MAXGEN=100, SEED=false.
  The pace of the front end was faster than expected, so the background population was being reset too often; by setting MAXGEN=100 we forced them to continue coevolving for at least 100 generations.
• After robot 2540, f=1, MAXGEN=500, SEED=true.
  With the results of our control, we realized that it takes up to 500 generations for a population to reach a performance plateau; f was reduced to 1 because pure coevolution was shown to outperform evolution against fixed trainers. Instead, the SEED option was incorporated to provide a means of feedback from the experienced foreground into the novelty engine.

The graph of rookie robots' strengths (fig. 3.9b) shows how the first change introduced a slight improvement, and the second an important improvement in new robots' qualities.