next up previous
Next: Statistical Relative Strength (RS) Up: Results Previous: Results

  
Win Rate (WR)

An intuitive performance measure is the win rate (WR), which is the fraction of games that the artificial players win;

 \begin{displaymath}
\text {WR}=\frac{\text {games\, won}}{\text {games\, played}}
\end{displaymath} (3.4)

The average win rate over the total number of games played is 0.55, meaning that 55% of all games completed resulted in agent victories.

The WR has been changing over time (fig. 3.7), in an oscillating fashion. This noisy behavior is a natural phenomenon in a coevolutionary environment, and occurs here more noticeably since one of the evolving populations consists of random human players. Each of the 4037 persons sampled here has a different level of expertise and has played a different number of games (another variable factor is the speed of the game on the user's machine, which may have a slower pace when the Java environment is too slow3.4).

  
Figure 3.7: Evolution of the win rate:WR sampled in groups of 1000 games.

\resizebox*{0.7\textwidth}{!}{\includegraphics{newfit/ts2f09.eps}}


There is a visible trend toward improvement. Whereas at the beginning of our experiment, the Tron system won about 30% of its games, by the end of the period it wins about 80% of its games. This is a strong indication of the improvement of the system's performance.

An increasing WR could be hiding other phenomena besides improvement in the quality of game of the Tron system. Humans could be playing worse now than before, for example, for whatever the reasons; or novices could be dominating the human population and playing the bulk of games. In the next section we describe a statistical technique that yields a much finer measure, avoiding the pitfalls of WR.


next up previous
Next: Statistical Relative Strength (RS) Up: Results Previous: Results
Pablo Funes
2001-05-08