This paper addresses the assumption, implicit in many robot and animat models of learning, that learning and unlearning are a pair of symmetrical processes. Unlearning mechanisms supposedly erase or remove existing learning because it is no longer relevant. Whether learning and unlearning result from the operation of symmetrical and antagonistic processes is an issue which has had a long but uneven history in animal and human psychology. This history is briefly recapitulated here. In particular, there is a contrast in the significance of the antagonistic processes model in the area of motivation compared to associative learning which has theoretical significance. For example, animat modelers frequently adopt a generic strengthening and weakening mechanism for all forms of learning and motivation representations without any consideration for its biological and psychological validity. In order to evaluate and question this, we examine the unlearning concept in a number of artefactual models drawn from a range of robotic and artificial life perspectives, and discuss their validity in terms of contemporary models of animal learning and motivation. Finally, we outline an alternative view of learning/unlearning, based on a recent contingency model of causality learning in humans, which does not rely on antagonistic processes and may have applications for artefacts.
Adams, C.D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. The Quarterly Journal of Experimental Psychology, 34B, pp.77-98.
2.
Adams, C.D. and Dickinson, A. (1981). Instrumental responding following reinforcement devaluation. Quarterly Journal of Experimental Psychology , 33B, pp.109-22.
3.
Amsel, A. (1958). The role of frustrative non reward in non continuous reward situations. Psychological Bulletin, 55, pp.102-119.
4.
Baeyens, F., Eelen, P. and Crombez, G. (1995). Pavlovian associations are forever: On classical conditioning and extinction. Journal of Psychophysiology, 9, pp.127-141.
5.
Balkenius, C. (1996). Generalization in instrumental learning. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior . Cambridge, MA: The MIT Press/ Bradford Books.
6.
Balkenius, C. (1995). Natural Intelligence in Artificial Creatures. Sweden: Lund University.
7.
Balkenius, C. (1994). Natural Intelligence for Autonomous Agents. LUCS 29. Sweden: Lund University.
8.
Barnes, J.M., and Underwood, B.J. (1959). Fate of first-list associations in transfer theory . Journal of Experimental Psychology, 58, pp. 97-105.
9.
Blumberg, B., Todd, P. and Maes, P. (1996). No bad dogs: ethological lessons for learning in hamsterdam. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/ Bradford Books.
10.
Bouton, M.E. (1993). Context, time and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114 (1), pp.80-99.
11.
Bouton, M.E. (1991). Context and retrieval in extinction and in other examples of interference in simple associative learning. In L. Dachowski and C.F. Flaherty (Eds.), Current topics in animal learning: Brain, emotion, and cognition. Hillsdale, NJ: LEA .
12.
Bouton, M.E. and Swartzentruber, D. (1989). Slow reacquisition following extinction: Context, encoding, and retrieval mechanisms. Journal of Experimental Psychology: Animal Behavior Processes, 15, pp.43-53.
13.
Brown, P.L. and Jenkins, H.M. (1968). Auto-shaping of the pigeon's key peck. Journal of the Experimental Analysis of Behavior, 11, pp.1-8.
14.
Cecconi, F. and Parisi, D. (1993). Neural networks with motivational units. In J.-A. Meyer, H.L. Roitblat and S.W. Wilson (Eds.), From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books .
15.
Chatlosh, D.L., Neunaber, D.J., and Wasserman, E.A. (1985). Response-outcome contingency: behavioral and judgmental effects of appetitive and aversive outcomes with college students. Learning and Motivation, 16, pp. 1-34.
16.
Chang, C. and Gaudiano, P. (1998). Application of biological learning theories to mobile robot avoidance and approach behaviors. Journal of Complex Systems, 1, pp.79-114.
17.
Cheng, P.W. and Holyoak, K.J. (1995). Adaptive systems as intuitive statisticians: Causality, contingency, and prediction. In J.-A. Meyer and H. Roitblat (Eds.), Comparative approaches to cognition. Cambridge, MA: The MIT Press.
18.
Colwill, R.M. (1994). Associative representations of instrumental contingencies . The Psychology of Learning and Motivation, 31, pp. 1-72.
19.
Colwill, R.M. (1993). Signalling the omission of a response-contingent outcome reduces discriminative control. Animal Learning and Behavior, 21, pp.337-345.
20.
Colwill, R.M. and Rescorla, R.A. (1990). Evidence for the hierarchical structure of instrumental learning. Animal Learning and Behavior, 18 (1), pp.71-82.
21.
Colwill, R.M. and Rescorla, R.A. (1986). Associative structures in instrumental learning . In G.H. Bower (Ed.) The Psychology of Learning and Motivation, 20, pp.55-104.
22.
Davey, G. (1989). Ecological learning theoryLondon: Routledge.
23.
Dickinson, A. (1994). Instrumental conditioning. In N. J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press .
24.
Dickinson, A. and Mulatero, C.W. (1989). Reinforcer specificity of the suppression of instrumental performance on a non-contingent schedule. Behavioural Processes , 19, pp.167-180.
25.
Dickinson, A. and Shanks, D.R. (1995). Instrumental action and causal representation . In D. Sperber, D. Premack and A.J. Premack (Eds.), Causal Cognition: A multidisciplinary debate. Oxford: Clarendon Press.
26.
Domjan, M. (1993). The Principles of Learning and Behavior (3rd Edition). CA: Brooks/Cole .
27.
Donahoe, J.W., Burgos, J.E., and Palmer, D.C. (1993). A selectionist approach to reinforcement. Journal of the Experimental Analysis of Behavior, 60, pp. 17-40.
28.
Gabora, L.M. (1993). Should I stay or should I go: Co-ordinating biological needs with continuously-updated assessments of the environment. In J.-A. Meyer, H.L. Roitblat and S.W. Wilson (Eds.), From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
29.
Galbicka, G. (1994). Shaping in the 21st century: moving percentile schedules into applied settings. Journal of Applied Behavior Analysis, 27, pp.739-760.
30.
Gaudiano, P., Zalama, E., Chang, C. and Coronado, J.L. (1996). A model of operant conditioning for adaptive obstacle avoidance. In P Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/ Bradford Books.
31.
Groves, P.M. and Thompson, R.F. (1970). Habituation: a dual-process theory. Psychological Review, 77, pp.419-450.
32.
Guha, D., Dutta, S.N. and Pradhan, S.N. (1974). Conditioning of gastric secretion by epinephrine in rats. Proceedings of the Society for Experimental Biology and Medicine, 147, pp.817-919.
33.
Halperin, J.R.P. (1995). Cognition and emotion in animals and machines . In H.L. Roitblat and J.-A. Meyer (Eds.), Comparative Approaches to Cognitive Science. Cambridge, MA: The MIT Press.
34.
Halperin, J.R.P. (1991). Machine Motivation. In J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats, Proceedings of the First International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books .
35.
Hammond, L.J. (1980). The effects of contingencies upon appetitive conditioning of free-operant behavior. Journal of the Experimental Analysis of Behavior, 34, pp.297-304.
36.
Hebb, D.O. (1949). The Organization of Behavior. NY: Wiley.
37.
Herrnstein, R.J. (1990). Levels of stimulus control: A functional approach . Cognition, 37, pp.133-166.
38.
Holland, P.C. (1983). "Occasion-setting" in conditional discriminations. In M.L. Commons, R.J. Herrnstein and A.R. Wagner (Eds.), Quantitative analyses of behavior: Discrimination processes (Vol. 4)NY: Ballinger.
39.
Hull, C.L. (1951). Essentials of behaviorNew Haven, CT: Yale University Press.
40.
Kamin, L.J. (1969). Predictability, surprise, attention, and conditioning . In B.A. Campbell and R.M. Church (Eds.), Punishment and aversive behavior. NY: Appleton-Century-Crofts .
41.
Lawrence, D.H. (1952). The transfer of a discrimination along a continuum . Journal of Comparative and Physiological Psychology, 45, pp.511-16.
42.
Lieberman, D.A., McIntosh, D.C. and Thomas, G.V. (1979). Learning when reward is delayed: A marking hypothesis . Journal of Experimental Psychology: Animal Behavior Processes , 5, pp.224-242.
43.
Lorenz, K. (1950). The comparative method in studying innate behavior patterns. Symp. Soc. Exp. Biol., 4, pp. 221-268.
44.
Mackintosh, N.J. (1983). Conditioning and associative learningOxford: Oxford University Press.
45.
Mackintosh, N.J. (1974). The Psychology of Animal LearningLondon: Academic Press.
46.
Mackintosh, N.J. and Dickinson, A. (1979). Instrumental (Type II) conditioning. In A. Dickinson and R.A. Boakes (Eds.), Mechanisms of learning and motivation. Hillsdale, NJ: Erlbaum.
Maes, P. (1991). Learning behavior networks from experience. In F.J. Varela and P Bourgine (Eds.), Towards a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life. Cambridge, MA: The MIT Press.
49.
McFarland, D.J. and Sibly, R.M. (1975). The behavioural final common path. Philosophical Transactions of the Royal Society, 270, B.907, pp. 265-293.
50.
Midgley, M., Lea, S.E.G. and Kirby, R. (1989). Algorithmic shaping and misbehavior in the acquisition of token deposit by rats. Journal of the Experimental Analysis of Behavior, 52, pp.27-40.
51.
Pearce, J.M. (1994). Discrimination and categorisation. In N.J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press .
52.
Rescorla, M.E., and Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. Black and W. Prokasy (Eds.), Classical Conditioning II, Current Research and Theory. NY: Appleton-Century-Crofts.
53.
Roitblat, H.L. (1994). Mechanism and process in animal behavior: models of animals, animals as models. In D. Cliff , D, P Husbands, J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/ Bradford Books.
54.
Saksida, L.M., Raymond, S.M., Touretzky, D.S. (1997). Shaping robot behavior using principles from instrumental conditioning. Robotics and Autonomous Systems, 22, pp.231-249.
55.
Salomon, R. (1998). Improving the DAC architecture by using proprioceptive sensors. In R. Pfeifer, B. Blumberg, J-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats 5, Proceedings of the Fifth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
56.
Savage, T. (2000a). Artificial motives: a review of motivation in artificial creatures. Connection Science, 12, pp.1-67.
57.
Savage, T. (2000b). Shaping: a multi-contingency model. Submitted to Connection Science.
58.
Schmajuk, N. (1997). Animal learning and cognition: a neural network approachCambridge: Cambridge University Press.
59.
Seely, R., Ramsey, D. and Woods, S. (1997). Regulation of food intake: interactions between learning and physiology. In M.E. Bouton and M.S. Fanselow (Eds.), Learning, Motivation and Cognition . American Psychological Association.
60.
Shanks, D.R. (1995). The Psychology of Associative LearningCambridge: Cambridge University Press.
61.
Shanks, D.R. (1994). Human Associative Learning. In N.J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press .
62.
Shanks, D.R. and Dickinson, A. (1991). Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Memory and Cognition , 19, pp.353-360.
63.
Shanks, D.R. and Dickinson, A. (1990). Contingency awareness in evaluative conditioning: A comment on Baeyens, Eelen and van den Berg. Cognition and Emotion, 49 (1), pp.19-30.
64.
Siegal, S. (1989). Pharmacological conditioning and drug effects. In A.J. Goudie and M.W. Emmett-Oglesby (Eds.), Psychoactive drugs: Tolerance and sensitization . Clifton, NJ:Humana Press.
65.
Skinner, B.F. (1938) The Behavior of OrganismsNew York: Appleton-Century-Crofts.
66.
Solomon, R.L. and Corbit, J.D. (1974). An opponent-process theory of motivation: I. The temporal dynamics of affect. Psychological Review, 81, pp.119-145.
67.
Spence, K.W. (1956). Behavior theory and conditioningNew Haven, CT: Yale University Press.
68.
Spier, E. and McFarland, D.J. (1996). A finer-grained motivational model of behaviour sequencing. In P Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior . Cambridge, MA: The MIT Press/ Bradford Books.
69.
Spier, E. and McFarland, D.J. (1998). Learning to do without cognition. In R. Pfeifer, B. Blumberg, J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats 5, Proceedings of the Fifth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/ Bradford Books.
70.
Stewart, J., and Eikelbloom, R. (1987). Conditioned drug effects. In L.L. Iversen and S.H. Snyder (Eds.), Handbook of Psychopharmacology, 19, pp.1-57.
71.
Tait, R. and Saladin, M.E. (1986). Concurrent development of excitatory and inhibitory associations during backward conditioning. Animal Learning and Behavior, 14, pp.133-137.
72.
Toates, F. (1986). Motivational systemsCambridge : Cambridge University Press.
Tinbergen, N. (1951). The study of instinctLondon : Oxford University Press.
75.
Toates, F. (1995). Animal motivation and cognition, In H. Roitblat and J.-A. Meyer (Eds.), Comparative approaches to cognitive science. Cambridge, MA: The MIT Press.
76.
Uhl, C. and Garcia, E. (1969). Comparison of omission and extinction in response elimination in rats. Journal of Comparative and Physiological Psychology, 69, pp.554-562.
77.
Wagner, A.R. and Brandon, S.E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In S.B. Klein and R.R. Mower (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Erlbaum.
78.
Wasserman, E.A., Elek, S.M.Chalosh, D.L. and Baker, A.G. (1993). Rating causal relations: The role of probability in judgements of response-outcome contingency. Journal of Experimental Psychology: Learning, Memory and Cognition, 19, pp.174-88.
79.
Williams, D.R., and Williams, H. (1969). Auto-maintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, pp.511-520.
