Action selection (AS) is defined as the process where an action is selected among a number of alternatives. This definition, however, does not sufficiently describe what an action is. What is the unit of selection in the first place? We maintain that the artificial intelligence (AI) accounts of AS typically mix and merge two AS situations that indeed are qualitatively different. Most of the accounts actually deal only with one type of AS but purport to cover both types of AS. We propose three dimensions along which the commonalities and the differences between various AS accounts can be analyzed, and use these for a preliminary conceptualization of what we call a two-system action selection account. In particular, we identify two qualitatively different AS situations whose architectures, we suggest, can be designed inspired by neuroscience models of the basal ganglia (BG) and the cerebellum, respectively.
Almang, J. ( 2007). Intentionality and intersubjectivity. Goteborg: Acta Universitatis Gothoburgensis.
2.
Andersen, R.A., & Buneo, C.A. ( 2002). Intentional maps in posterior parietal cortex. Annual Review of Neuroscience, 25, 189-220.
3.
Arbib, M. ( 1992). Schema theory. In S. Shapiro (Ed.), Encyclopedia of artificial intelligence (Volume 2, 2nd ed., pp. 1427-1443). Wiley, New York.
4.
Arkin, R.C. ( 1998). Behaviour-based robotics. Cambridge, MA: MIT Press.
5.
Atkeson, C.G., & Santamaria, J. (1997). A comparison of direct and model-based reinforcement learning. International Conference on Robotics and Automation (pp. 11-29). Albuquerque, New Mexico, USA, IEEE press.
6.
Baldassarre, G. ( 2002). A modular neural-network model of the basal ganglia’s role in learning and selecting motor behaviours. Journal of Cognitive Systems Research, 3, 5-13.
7.
Barto, A. ( 1976). Adaptive critics and the basal ganglia. In C. J. Houk, L. J. Davis, & G. D. Beiser (Eds.), Models of information processing in the basal ganglia. MIT Press, Cambridge.
8.
Blumberg, B., Todd, P., & Maes, P. ( 1996). No bad dogs: Ethological lessons for learning in hamsterdam . P. Maes, M.J. Mataric, J.-A. Meyer, J. Pollack, and S.W. Wilson (Eds.) In Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. (pp. 295-304). Cambridge, MA: MIT Press.
9.
Brooks, R. ( 1991). Intelligence without representation. Artificial Intelligence, 47, 139-159.
10.
Bryson, J. ( 2000a). Cross-paradigm analysis of autonomous agent architecture . Journal of Experimental and Theoretical Artificial Intelligence , 12(2), 165-189.
11.
Bryson, J.Meyer, J. , Berthoz, A., Floreano, D., Roitblat, H., and Wilson, S., (Eds.) (2000b). Hierarchy and sequence vs. full parallelism in action selection. In The 6th International Conference on the Simulation of Adaptive Behavior (SAB2000) (pp. 147-156). Paris, France. MIT Press, Cambridge, MA.
12.
Bryson, J. ( 2001). Intelligence by design: Principles of modularity and coordination for engineering complex adaptive agents. MPhil Thesis: University of Edinburgh.
13.
Buneo, C.A., & Andersen, R.A. (2006). The posterior parietal cortex: Sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia, 44, 2594-2606.
14.
Chandrasekaran, B. ( 1986). Generic tasks involved in knowledge-based reasoning: High-level building blocks for expert system design. IEEE expert, 1(3), 23-29.
15.
Cisek, P., & Kalaska, J. ( 2005). Neural correlates of reaching decisions in dorsal premotor cortex: Specification of multiple direction choices and final selection of action. Neuron, 45, 801-814.
16.
Cooper, R., & Shallice, T. ( 2000). Cognitive neuropsychology. The Journal of Neuroscience, 17(4), 297-338.
17.
Diedrichsen, J., Hashambhoy, Y., Rane, T., & Shadmehr, R. ( 2005). Neural correlates of reach errors. The Journal of Neuroscience, 25(43), 9919-9931.
18.
Doya, K. ( 1999). What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Networks, 12(7-8), 961-974.
19.
Doya, K., Samejima, K., Katagiri, K., & Kawato, M. ( 2000). Multiple model-based reinforcement learning. Neural Computation, 14(6), 1347-1369.
20.
Genovesio, A., Brasted, P.J., & Wise, S.P. ( 2006). Representation of future and previous spatial goals by separate neural populations in prefrontal cortex. Journal of Neuroscience, 26(27), 7305-7316.
21.
Grafton, S., & Hamilton, A. ( 2007). Evidence for a distributed hierarchy of action representation in the brain. Human Movement Science, 26, 590-616.
22.
Graybiel, A., & Saka, E. ( 2003). The basal ganglia and the control of action. In M. S. Gazzaniga (Ed.), The new cognitive neurosciences (pp. 495-510). Cambridge, MA: MIT Press.
23.
Gurney, K., Humphries, R.W., Prescott, T.J., & Redgrave, P. ( 2004). Testing computational hypotheses of brain systems function: a case study with the basal ganglia. Network: Computation in Neural Systems, 15, 263-290.
24.
Hamilton, A.F. d. C., & Grafton, S.T. (2006). Goal representation in human anterior intraparietal sulcus. Journal of Neuroscience , 26, 1133-1137.
25.
Hamilton, A.F. d. C., & Grafton, S.T. (2008). Action outcomes are represented in human inferior frontoparietal cortex. Cerebral Cortex, 18(5), 1160-1168.
26.
Haruno, M., Wolpert, D., & Kawato, M. ( 2001). Mosaic model for sensorimotor learning and control. Neural Computation, 13(10), 2201-2220.
27.
Hawkins, J., & Blakeslee, S. ( 2004). On intelligence. New York: Times books.
28.
Hendler, J., Tate, A., & Drummond, M. ( 1990). AI planning: systems and techniques. AI Magazine, 11(2), 61-77.
29.
Hoshi, E., & Tanji, J. ( 2000). Integration of target and bodypart information in the premotor cortex when planning action. Nature, 408, 466-470.
30.
Hoshi, E., & Tanji, J. ( 2002). Contrasting neuronal activity in the dorsal and ventral premotor areas during preparation to reach. Journal of Neurophysiology , 87, 1123- 1128.
31.
Hoshi, E., Tremblay, L., Féger, L., Carras, P.L., & Strick, P.L. ( 2005). The cerebellum communicates with the basal ganglia. Nature Neuroscience, 8, 1491-1493.
32.
Humphries, M.D., Gurney, K., & Prescott, T.J. (2007). Is there a brainstem substrate for action selection? Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1627-1639.
33.
Ito, M., Sakurai, M., & Tongroach, P. ( 1982). Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar purkinje cells. Journal of Physiology, 324, 113-134.
34.
Jacobs, R., Jordan, M.I., Nowlan, S., & Hinton, G. ( 1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
35.
Jaeger, H., & Haas, H. ( 2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, (304), 78-80.
36.
Jeannerod, M. ( 1997). The cognitive neuroscience of action. Oxford: Blackwell.
37.
Jueptner, M., & Weiller, C. ( 1996). A review of differences between basal ganglia and cerebellar control of movements as revealed by functional imaging studies. Brain, 16(127), 1437-1449.
38.
Kawato, M. ( 1990). Feedback-error-learning neural network for supervised motor learning. In R. Eckmiller (Ed.), Advanced neural computers (pp. 365-472). Elsevier, Amsterdam, The Netherlands.
39.
Khamassi, M., Lachèze, L., Girard, B., Berthoz, A., & Guillot, A. ( 2005). Actor-critic models of reinforcement learning in the basal ganglia: From natural to artificial rats. Adaptive Behavior, Special Issue Towards Artificial Rodents, 13(2), 131-148.
40.
Krichmar, J.L. ( 2008). The neuromodulatory system: A framework for survival and adaptive behaviour in a challenging world. Adaptive Behavior , 16(6), 385-399.
41.
Maes, P. ( 1990). Situated agents can have goals. In P. Maes (Ed.) Designing autonomous agents (pp. 49-70). Cambridge, MA: MIT Press.
42.
Markman, A.B., & Dietrich, E. ( 2000). In defense of representation. Cognitive Psychology, 40, 138-171.
43.
Marr, D. ( 1982). Vision: A computational approach. San Francisco: Freeman.
44.
Meyer, J.-A., Guillot, A., Girard, B., Khamassi, M., Pirim, P., & Berthoz, A. ( 2005). The psikharpax project: Towards building an artificial rat. Robotics and Autonomous Systems, 50(4), 211-221.
45.
Minsky, M. ( 1986). The society of mind. New York: Simon and Schuster.
46.
Mitchell, T. ( 1997). Machine learning. McGraw Hill, New York.
47.
Muller, J., & Pischel, M. ( 1993). The agent architecture inter-rap: Concept and application . Technical Report RR-93-26, DFKI Saarbrucken.
48.
Pacherie, E. ( 2000a). The content of intentions. Mind and Language, 15(4), 400-432.
49.
Pacherie, E. ( 2000b). Conscious experience and concept forming abilities. Acta Analytica, 26, 45-52.
50.
Plaut, D.C., & Botvinick, M.M. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111, 395-429.
51.
Pockett, S. ( 2006). The neuroscience of movement. In S. Pockett, W. P. Banks, & S. Gallagher (Eds.), Does consciousness cause behavior? (pp. 10-24). Cambridge, MA: MIT press.
52.
Prescott, T.J., Bryson, J., & Seth, A.K. ( 2007). Introduction. Modelling natural action selection. Philosophical Transactions of the Royal Society B: Biology, 362(1485), 1521- 1529.
53.
Prescott, T.J., Montes Gonzales, F.M., Gurney, K., Humphries, M.D., & Redgrave, P. ( 2006). A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Networks, 19, 31-61.
54.
Rosenblatt, N., & Gorce, P. ( 1997). The distributed architecture for mobile navigation. Journal of Experimental and Theoretical Artificial Intelligence, 9(2/3), 339-360.
55.
Searle, J. ( 1983). Intentionality: An essay in the philosophy of mind. New York: Cambridge University Press.
56.
Serrien, D.J., Ivry, R.B., & Swinnen, S.P. ( 2007). The missing link between action and cognition. Progress in Neuro-biology, 82(2), 95-107.
57.
Sheeran, P., Webb, T.L., & Gollwitzer, P.M. (2005). The interplay between goal intentions and implementation intentions. Personality and Social Psychology Bulletin, 31(1), 87-98.
58.
Szaniawski, K. ( 1980). Philosophy of decision making. Acta Psychologica , 45, 327-341.
59.
Tani, J., & Yamamato, J. ( 2002). On the dynamics of robot exploration learning. Cognitive Systems Research, 3(3), 459-470.
60.
Tidemann, A., & Öztürk, P. (2008). Learning dance movements by imitation: A multiple model approach. In 31st Annual German Conference on AI (pp. 380-388). Berlin: Springer.
61.
Tyrrel, T. ( 1993). Computational mechanisms for action selection. Ph.D. thesis: University of Edinburgh.
62.
Whiteson, S., Taylor, M., & Stone, P. ( 2007). Empirical studies in action selection for reinforcement learning. Adaptive Behavior, 15(1), 33-50.
63.
Wolpert, D., & Kawato, M. ( 1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11, 1317-1329.
64.
Wolpert, D., Miall, R., & Kawato, M. ( 1998). Internal models in cerebellum. Trends in Cognitive Science, 2(9), 337- 348.
