Analysis of Results

In an effort to track the Red Queen (see section 3.6), without having to play games outside those involved in the coevolutionary situation, we can think of each player as a relative reference. In Tron, each agent has a fixed strategy and thus constitutes a marker that gives a small amount of evaluation information. A single human, as defined by their login name and password, should also be relatively stable, in the short term at least. The paired comparisons model described above is a powerful tool that uses the information of all the interwoven relationships of a matrix of games (fig. 3.6) at the same time. Every player, with his/her/its wins and loses, contributes useful bits of information to evaluate all the rest.

There are degenerate situations where the present model would give no answer. If one has knowledge of games between players A and B for example, and also between C and D, but nor A nor B have ever played C or D, there is no connectivity, and consequently no solution to equation 3.8. In the Tron case, connectivity is maintained throughout the experiment by the multitude of players who come and go, coexisting for a while with other players who are also staying for a limited time. The whole matrix is connected, and the global solution propagates those relative references throughout the data set.

An increasing WR (section 3.4.1) is not a whole proof that our system was evolving towards better agents. It could be the case, for example, that humans became increasingly sloppy, losing more and more of their games while agents stayed more or less the same.

Applying the paired comparisons model gave us more reliable information. We computed the RS for every computer and human player^3.5. For the first time we were able to compare agents and humans with each other: fig. 3.1 lists the 15 best (a) and worst (b) players. Each human and robot is labelled with a unique ID number: humans were numbered consecutively by their first appearance, and robots have id numbers all greater than 10,000 (the first 3 digits encode the generation number).

The top players table (fig. 3.1a) has 6 humans at the top, the best agent so far being seventh. The best player is a human, far better than all others: according to eq. 3.7, an estimated 87% chance of beating the second best!. This person must be a genius.^3.6

 

Table 3.1: Best players and worst players lists. Only players with 100 games or more are been considered (244 humans, 391 robots). ID numbers greater than 10000 correspond to artificial agents.

(a) Best Players Strength Player ID 1. 3.55 887 2. 1.60 1964 3. 1.26 388 4. 1.14 155 5. 1.07 1636 6. 1.05 2961 7. 0.89 3010008 8. 0.89 3100001 9. 0.84 1754 10. 0.81 2770006 11. 0.81 3130004 12. 0.76 2980001 13. 0.70 1860 14. 0.66 2910002 15. 0.62 3130003

(b) Worst Players Strength Player ID 1. -4.64 2407 2. -3.98 2068 3. -3.95 3982 4. -3.88 32 5. -3.75 1986 6. -3.73 33 7. -3.69 3491 8. -3.41 2146 9. -3.39 2711 10. -3.36 3140 11. -3.31 1702 12. -3.31 1922 13. -3.30 2865 14. -3.27 2697 15. -3.22 2441

The difference between the top group of human players (RS around 1.1) and the top agent players (RS's around 0.7) is about 60%. Seven out of the best 15 players are agents. The best agent, R. 301008, is estimated to be better than 97.5% of the human population.

The worst players table (fig. 3.1b) is composed of all humans. This does not indicate that all agents are good but rather, that most bad agents are eliminated before reaching 100 games.