Experiments in Parameter Learning Using Temporal Differences

Abstract

This paper deals with the problem of automatically learning evaluation-function parameters. In particular, we describe some experiments in which our chess program KNIGHTCAP learnt the parameters of its evaluation function using a combination of temporal difference learning and on-line play on FICS (Free Internet Chess Server) and ICC (Internet Chess Club). The main success reported is that KNIGHTCAP went from a blitz rating of 1650 to a blitz rating of 2150 in just three days and 308 games. We provide details of our learning algorithm, details of KNIGHTCAP, and some reasons for KNIGHTCAP’S rapid improvement in playing strength.