A new neuronal model of classical conditioning is proposed. The model, called the delay-producing connections (or DPC) model, is an extension of Klopf's (1988) drive-reinforcement model and Sutton and Barto's (1981) model. The DPC model makes two contributions: lt represents the trace of each conditioned stimulus (CS) by a differential equation; and it replaces each CS in the activation rule by its CS trace, which is assumed to be equal to the CS eligibility. The DPC model reproduces the usual shape of a conditioned response, the curve of efficacy of conditioning as a function of the interstimulus interval (ISI), the dependence of the optimal ISI on CS duration, the extinction of a conditioned response [even for long-lasting CSs as opposed to Klopf's (1988) model], and several other properties of classical conditioning.
Anderson, J. A., & Rosenfeld, E. (Eds.). (1988). Neurocomputing: Foundations of research. Cambridge, MA: MIT
2.
Baker, A.G., & Mackintosh, N.J. (1977). Excitatory and inhibitory conditioning following uncorrelated presentations of the CS and US. Animal Learning and Behavior, 5, 315-319.
3.
Barnes, G.W. (1956). Conditioned stimulus intensity and temporal factors in spaced-trial classical conditioning. Journal of Experimental Psychology, 51, 192-198.
4.
Blanchard, R.J., & Blanchard, D.C. (1969). Passive and active reactions to fear-eliciting stimuli. Journal of Comparative and Physiological Psychology, 68, 129-135.
5.
Blazis, D.E.J., Desmond, J.E., Moore, J.W., & Berthier, N.E. (1986). Simulation of the classically conditioned nictitating membrane response by a neuron-like adaptive element: A real-time variant of the Sutton-Barto model. In Program of the Eighth Annual Conference of the Cognitive Science SocietyHillsdale, NJ: Lawrence Erlbaum Associates.
6.
Bliss, T.V.P., & Lomo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetised rabbit following stimulation of the perforant path. Journal of Physiology ( London), 232, 331-356.
7.
Buonomano, D.V., & Byrne, J.H. (1990). Long-term synaptic changes produced by a cellular analog of classical conditioning in Aplysia. Science, 249, 420-423.
8.
Burkhardt, P.E., & Ayres, J.J.B. (1978). CS and US duration effects in one-trial simultaneous fear conditioning as assessed by conditioned suppression of licking in rats . Animal Learning and Behavior, 6, 225-230.
9.
Bush, R.R., & Mosteller, F. (1955). Stochastic models for learning. New York: Wiley.
10.
Buss, A.H. (1967). Stimulus generalization and the matching principle . Psychological Review, 74, 40-50.
Byrne, J.H., Zwartjes, R., Homayouni, R., Critz, S.D., & Eskin, A. (1993). Roles of second messenger pathways in neuronal plasticity and in learning and memory. Insights gained from Aplysia. Advances in Second Messenger and Phosphoprotein Research, 27, 47-108.
13.
Carew, T.J., Hawkins, R.D., & Kandel, E.R. (1983). Differential classical conditioning of a defensive withdrawal reflex in Aplysia californica. Science, 219, 397-400.
14.
Czaplicki, J.A., Borreback, D.E., & Wilcoxon, H.C. (1976). Stimulus generalization of an illness-induced aversion to different intensities of colored water in Japanese quail. Animal Learning and Behavior, 4, 45-48.
15.
Davies, P. (1974a). Conditioned visual afterimages I. British Journal of Psychology, 65, 191-204.
16.
Davies, P. (1974b). Conditioned visual afterimages II. British Journal of Psychology, 65, 377-393.
17.
Desmond, J.E. (1990). Temporal adaptive responses in neural models: The stimulus trace. In M. Gabriel & J. W. Moore (Eds.), Learning and computational neuroscience. Cambridge, MA: MIT.
18.
Desmond, J.E., & Moore, J.W. (1988). Adaptive timing in neural networks: The conditioned response. Biological Cybernetics, 58, 405-415.
19.
Desmond, J.E., & Moore, J.W. (1991). Altering the synchrony of stimulus trace processes: Tests of neural-network model. Biological Cybernetics, 65, 161-169.
20.
Dickinson, A. (1980). Contemporary animal learning theory. Cambridge, England: Cambridge University Press.
21.
Dickinson, A., Hall, G., & Mackintosh, N.J. (1976). Surprise and the attenuation of blocking. Journal of Experimental Psychology: Animal Behavior Processes, 2, 313-322.
22.
Doyere, V., Burette, F., Redini-Del Negro, C., & Laroche, S. (1993). Long-term potentiation of hippocampal afferents and efferents to prefrontal cortex: Implications for associative learning . Neuropsychologia, 31, 1031-1053.
23.
Feldman, J. A. (Ed.). (1985). Special issue on connectionist models and their applications. Cognitive Science, 9, 1-169.
24.
Flaherty, C.F. (1985). Animal learning and cognition. New York: Knopf.
25.
Frey, P.W. (1969). Within- and between-session CS intensity performance effects in rabbit eyelid conditioning. Psychonomic Science , 17, 1-2.
26.
Frey, P.W., & Sears, R.J. (1978). Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. Psychological Review, 85, 321-348.
27.
Garcia, R., Vouimba, R.M., & Jaffard, R. (1993). Spatial discrimination learning induces LTP-like changes in the lateral septum of mice. Neuroreport, 5, 329-332.
28.
Gluck, M.A., & Granger, R. (1993). Computational models of the neural bases of learning and memory. Annual Review of Neuroscience, 16, 667-706.
29.
Gluck, M.A., & Thompson, R.F. (1987). Modeling the neural substrates of associative learning and memory: A computational approach. Psychological Review, 94, 176—191.
30.
Gormezano, I., Kehow, E.J., & Marshall, B.S. (1983). Twenty years of classical conditioning research with the rabbit. Progress in Psychobiology and Physiological Psychology, 10, 197-275.
31.
Gray, J.A. (1965). Stimulus intensity dynamism. Psychological Bulletin, 63, 180-196.
Guevara, M.R., Glass, L., Mackey, M.C., & Shrier, A. (1983). Chaos in neurobiology. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 790-798.
35.
Halperin, J.R.P. (1991). Machine motivation. In J.-A. Meyer & S. W Wilson (Eds.), From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT.
36.
Hawkins, R.D., Carew, T.J., & Kandel, E.R. (1986). Effects of interstimulus interval and contingency on classical conditioning of the Aplysia siphon withdrawal reflex. Journal of Neuroscience, 6, 1695-1701.
37.
Hebb, D.O. (1949). Organization of behavior. New York: Wiley.
38.
Hertz, J., Krogh, A., & Palmer, R.G. (1991). Introduction to the theory of neural computation . New York: Addison-Wesley.
39.
Hodgkin, A.L., & Huxley, A.F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. Journal of Physiology, 117, 500-544.
40.
Hull, C.L. (1939). The problem of stimulus equivalence in behavior theory. Psychological Review, 46, 9-30.
41.
Hull, C.L. (1943). Principles of behavior. Englewood Cliffs, NJ:Prentice Hall.
Hull, C.L. (1952). A behavior system: An introduction to behavior theory concerning the individual organism. New Haven: Yale University.
44.
Jones, L., & Bellingham, W.P. (1981). Taste aversion conditioning and barpressing: A quality-quantity interaction. Australian Journal of Psychology, 33, 61-71.
45.
Kamin, L.J. (1968). "Attention-like" processes in classical conditioning . In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation . Miami: University of Miami.
46.
Kehoe, E.J. (1989). Connectionist models of conditioning: A tutorial . Journal of the Experimental Analysis of Behavior, 52, 427—440.
47.
Kehoe, E.J. (1992). More than meets the eye: Animal learning and knowledge induction. Paper presented at the Thirteenth Machine Intelligence Conference, Ross Priory, Scotland.
48.
Klein, S.B. (1991). Learning (2nd ed.). New York: McGraw-Hill.
49.
Klein, S.B., McGee-Davis, T., Cohen, L., & Weston, D. (1984). Relative influence of cue predictiveness and salience on flavor aversion learning. Learning and Motivation, 15, 188—202.
50.
Klopf, A.H. (1972). Brain function and adaptive systems: A heterostatic theory. (Air Force Research Laboratories Research Rep. No. 133 [AFCRL-72-0164]) . Bedford, MA: Air Force Cambridge Research Laboratories.
51.
Klopf, A.H. (1982). The hedonistic neuron: A theory of memory, learning, and intelligence. Washington, DC: Hemisphere Publishing.
52.
Klopf, A.H. (1986). Drive-reinforcement model of single neuron function: An alternative to the Hebbian neuronal model. In J. Denker (Ed.), AIP Conference Proceedings, 151: Neural Networksfor Computing. New York, NY: American Institute of Physics.
53.
Klopf, A.H. (1988). A neuronal model of classical conditioning. Psychobiology, 16, 85-125.
54.
Klopf, A.H. (1992). Modeling nervous system function with a hierarchical network of control systems that learn. In J.-A. Meyer, H. L. Roitblat, & S. W Wilson (Eds.), From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Honolulu, Hawaii.
55.
Klopf, A.H., Morgan, J.S., & Weaver, S.E. (1993). A hierarchical network of control systems that learn: Modeling nervous system function during classical and instrumental conditioning. Adaptive Behavior, 1, 263-319.
56.
Lacouture, Y., & Marley, A.A.J. (1991). A connectionist model of choice and reaction time in absolute identification. Connection Science, 3, 401-433.
57.
Larson, J., & Lynch, G. (1986). Induction synaptic potentiation in hippocampus by patterned stimulation involves two events. Science, 232, 985-988.
58.
Larson, J., Xiao, P., & Lynch, G. (1993). Reversal of LTP by theta frequency stimulation . Brain Research, 600, 97-102.
59.
Lubow, R.E., & Moore, A.V. (1959). Latent inhibition: The effect of nonreinforced preexposure of the conditioned stimulus. Journal of Comparative and Physiological Psychology, 52, 415-419.
60.
MacGregor, R.J. (1987). Neural and brain modeling. San Diego: Academic.
61.
Mackintosh, N.J. (1975). A theory of attention: Variations in the association of stimuli with reinforcement. Psychological Review, 82, 276—298.
62.
Mackintosh, N.J., Bygrave, D.J., & Picton, D.M.B. (1977). Locus of the effect of a surprising reinforcer in the attenuation of blocking. Quarterly Journal of Experimental Psychology, 29, 327-336.
63.
Malaka, R., & Hammer, M. (1996). Real-time models of classical conditioning. Proceedings of the International Conference on Neural Networks (ICNN'96), Washington, 2, 768—773.
64.
Manning, A.A., & Schneiderman, N. (1970). Stimulus generalization in rabbits conditioned to different intensities of caudate nucleus stimulation . Journal of Comparative and Physiological Psychology, 72, 374—383.
65.
Mignault, A. (1993). Mathematical models of classical conditioning: A critical review and extensions. Unpublished MSc thesis, Department of Psychology, McGill University, Montreal .
66.
Moore, J.W., Desmond, J.E., Berthier, N.E., Blazis, D.E.J., Sutton, R.S., & Barto, A.G. (1986). Simulation of the classically conditioned nictitating membrane response by a neuron-like adaptive element: Response topography, neuronal firing, and interstimulus intervals. Behavioural Brain Research, 21, 143-154.
67.
Mowrer, O.H. (1950). Learning theory and personality dynamics. New York: Ronald Press.
68.
Nowlis, G.H. (1974). Conditioned stimulus intensity and acquired alimentary aversions in the rat. Journal of Comparative and Physiological Psychology, 86, 1173-1184.
Pearce, J.M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532-552.
71.
Prokasy, W.E., Jr., Grant, D.A., & Meyers, N.A. (1958). Eyelid conditioning as a function of unconditioned stimulus intensity and intertrial interval. journal of Experimental Psychology, 55, 242-246.
72.
Prokasy, W.F., Jr., Hall, J.F., & Fawcett, J.T. (1962). Adaptation, sensitization, forward and backward conditioning, and pseudo-conditioning of the GSR. Psychological Reports, 10, 103-106.
73.
Reeke, G.N., & Sporns, O. (1993). Behaviorally based modeling and computational approaches to neuroscience. Annual Review of Neuroscience , 16, 597-623.
74.
Rescorla, R.A., & Wagner, A.R. (1972). A theory of Pavlovian conditioning. Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W F. Prokasy (Eds.), Classical conditioning II: Current research and theory. New York: Appleton-Century-Crofts .
75.
Rizley, R.C., & Rescorla, R.A. (1972). Associations in second-order conditioning and sensory preconditioning. Journal of Comparative and Physiological Psychology, 81, 1-11.
76.
Rosenblatt, F. (1962). Principles of neurodynamics. New York: Spartan Books.
77.
Rumelhart, D.E., McClelland, J.L., & the PDP Research Group (1986 ). Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 1. Foundations. Cambridge, MA: MIT Press/Bradford Books.
78.
Scavio, J.M., & Gormezano, I. (1974). CS intensity effects on rabbit nictitating membrane conditioning, extinction and generalization. Pavlovian Journal of Biological Science, 9, 25-34.
79.
Schmajuk, N.A., & Moore, J.W. (1989). Effects of hippocampal manipulations on the classically conditioned nictitating membrane response: Simulations by an attentional-associative model. Behavioural Brain Research, 32, 173-189.
80.
Schneiderman, N. (1966). Interstimulus interval function of the nictitating membrane response of the rabbit under delay versus trace conditioning. Journal of Comparative and Physiological Psychology, 62, 397-402.
81.
Shultz, T.R., & Schmidt, W.C. (1991). A cascade-correlation model of balance scale phenomena. In Proceedings of the Thirteenth Annual Conference of the Cognitive Science SocietyHillsdale, NJ: Lawrence Erlbaum Associates.
82.
Stutz, R., Hastings, L., Rossi, R.R., & Maroli, A.N. (1978). Intensity generalization with positively reinforcing intracranial stimulation. Physiology and Behavior, 21, 531—535.
83.
Sutton, R.S., & Barto, A.G. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88, 135-170.
84.
Sutton, R.S., & Barto, A.G. (1987). A temporal-difference model of classical conditioning. In Proceedings of the Ninth Conference of the Cognitive Science SocietyHillsdale, NJ: Lawrence Erlbaum Associates .
85.
Sutton, R.S., & Barto, A.G. (1990). Time-derivative models of Pavlovian reinforcement . In M. Gabriel & J. W. Moore (Eds.), Learning and computational neuroscience: Foundations of adaptive networks. Cambridge, MA: MIT.
86.
Thompson, R.F., Clark, G.A., Donegan, N.H., Lavond, D.G., Madden, J., IV, Mamounas, L.A., Mauk, M.D., & McCormick, D.A. (1984). Neuronal substrates of basic associative learning. In L. Squire & N. Butters (Eds.), Neuropsychology of memory. New York: Guilford.
87.
Turkkan, J.S. (1989). Classical conditioning: The new hegemonyBehavioral and Brain Sciences, 12, 121-179.
88.
Uttley, A.M. (1975). The informon in classical conditioning. Journal of Theoretical Biology, 49, 355-376.
89.
Wagner, A.R. (1976). Expectancies and the priming of STM. In S. H. Hulse, H. Fowler, & W K. Honig (Eds.), Cognitive processes in animal behavior. Hillsdale, NJ: Lawrence Erlbaum Associates.
90.
Wagner, A.R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. Spear & G. Miller (Eds.), Information processing in animals: Memory mechanisms. Hillsdale, NJ: Lawrence Erlbaum Associates.
91.
Wagner, A.R., & Brandon, S.E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Lawrence Erlbaum Associates.
92.
Zimmer-Hart, C.L., & Rescorla, R.A. (1974). Extinction of Pavlovian conditioned inhibition. Journal of Comparative and Physiological Psychology, 86, 837-845.
