An intuitive performance measure is the win rate (WR), which is the fraction
of games that the artificial players win;
The average win rate over the total number of games played is 0.55, meaning that 55% of all games completed resulted in agent victories.
The WR has been changing over time (fig. 3.7), in an oscillating
fashion. This noisy behavior is a natural phenomenon in a coevolutionary environment,
and occurs here more noticeably since one of the evolving populations consists
of random human players. Each of the 4037 persons sampled here has a different
level of expertise and has played a different number of games (another variable
factor is the speed of the game on the user's machine, which may have a slower
pace when the Java environment is too slow3.4).
There is a visible trend toward improvement. Whereas at the beginning of our experiment, the Tron system won about 30% of its games, by the end of the period it wins about 80% of its games. This is a strong indication of the improvement of the system's performance.
An increasing WR could be hiding other phenomena besides improvement in the quality of game of the Tron system. Humans could be playing worse now than before, for example, for whatever the reasons; or novices could be dominating the human population and playing the bulk of games. In the next section we describe a statistical technique that yields a much finer measure, avoiding the pitfalls of WR.