Human-Machine Coevolution

The word coevolution is used in computer science literature to describe relative fitness situations, where an individual's value depends upon the population itself. But the present work is the first example, to the best of our knowledge, of coevolution between an agent and an animal species. Agents are selected, through the fitness rules coded into the system, based only on their interaction with humans.

With Tron we are proposing a new paradigm for evolutionary computation: creating niches where agents and humans interact, leading to the evolution of the agent species. There are two main difficulties introduced when one attempts this type of coevolution against real people:

• Interactions with humans are a sparse resource.
• Opponents are random and known tournament techniques for coevolution become infeasible.

The first problem is common to all applications that wish to learn from a real, or even simulated, environment: interactions are slow and costly. We address this problem by nesting an extra loop of coevolution: while the system is waiting for human opponents, it runs many generations of agent-agent coevolution.

The second problem led us to develop a new evaluation strategy, based on the paired comparisons statistics. With it we were able to successfully select the best strategies, pushing the system to the level of a top 3% human.

The differences between self evaluation and human evaluation, studied in section 3.7, indicate, on the one hand, that evolving Tron agents by playing each other was not sufficient, as the top agents are usually not so special against people. But on the other, some of them are good, so expertise against other robots and expertise against people are not independent variables either.

We think that this is the general case: evolutionary computation is useful in domains that are not entirely unlearnable; at the same time, there is no substitute for the real experience: simulation can never be perfect.

We have also been able to show here, how most humans -- at least those who stay for a while -- learn from their interaction with the system; some of them quite significantly. Even though the system was not designed as a training environment for people, but rather simply as an artificial opponent, the implications for human education are exciting: evolutionary techniques provide us with a tool for building adaptive environments, capable of challenging humans with increased efficiency by interacting with a large group of people.