Modeling Behavior Cycles as a Value System for Developmental Robots

Abstract

The behavior of natural systems is governed by rhythmic behavior cycles at the biological, cognitive, and social levels. These cycles permit natural organisms to adapt their behavior to their environment for survival, behavioral efficiency, or evolutionary advantage. This article proposes a model of behavior cycles as the basis for motivated reinforcement learning in developmental robots. Motivated reinforcement learning is a machine learning technique that incorporates a value system with a trial-and-error learning component. Motivated reinforcement learning is a promising model for developmental robotics because it provides a way for artificial agents to build and adapt their skill-sets autonomously over time. However, new models and metrics are needed to scale existing motivated reinforcement learning algorithms to the complex, real-world environments inhabited by robots. This article presents two such models and an experimental evaluation on four Lego Mindstorms NXT robots. Results show that the robots can evolve measurable, structured behavior cycles adapted to their individual physical forms.