Both animals and humans use meta-rules in their daily life, in order to adapt their behavioral strategies to changing environmental situations. Typically, the term meta-rule encompasses those rules that are applied to rules themselves. In cognitive science, conventional approaches for designing meta-rules follow human hardwired architectures. In contrast to previous approaches, this study employs evolutionary processes to explore neuronal mechanisms accounting for meta-level rule switching. In particular, we performed a series of experiments with a simulated robot that has to learn to switch between different behavioral rules in order to accomplish given tasks. Continuous time recurrent neural networks (CTRNN) controllers with either a fully connected or a bottleneck architecture were examined. The results showed that different rules are represented by separate self-organized attractors, while rule switching is enabled by the transitions among attractors. Furthermore, the results showed that neural network division into a lower sensorimotor level and a higher cognitive level enhances the performance of the robot in the given tasks. Additionally, meta-cognitive rule processing is significantly supported by the embodiment of the controller and the lower level sensorimotor properties of environmental interaction.
Anderson, M., & Oates, T. (2007). A review of recent research in metareasoning and metalearning. AI Magazine, 28, 7- 16.
2.
Arie, H., Namikawa, J., Ogata, T., Tani, J., & Sugano, S. (2006). Reinforcement learning algorithm with CTRNN in continuous action space. In Proceedings of the International Conference on Neural Information Processing, ICONIP-2006. (pp. 387-396).
3.
Bateson, G. (1972). Steps to an ecology of mind: Collected essays in anthropology, psychiatry, evolution, and epistemology. Chandler Publishing Company.
4.
Beer, R. (1995). On the dynamics of small continuous-time recurrent neural networks. Adaptive Behavior, 3, 471-511.
5.
Beer, R., & Gallagher, J. (1992). Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior, 1, 92- 122.
6.
Bessiere, C., Freuder, E., & Regin, J.-C. (1999). Using constraint metaknowledge to reduce arc consistency computation. Artificial Intelligence, 107, 125-148.
7.
Bogacz, R., & Giraud-Carrier, C. (1998). Learning meta-rules of selection in expert systems. In Proceedings of the 4th World Congress on Expert Systems (pp. 576-581).
8.
Bown, O., & Lexer, S. (2006). Continuous-time recurrent neural networks for generative and interactive musical performance. In Proceedings of EvoWorkshops 2006 (pp. 652-663).
9.
Bryson, J. (2001). Intelligence by design: principles of modularity and coordination for engineering complex adaptive agents. Ph.D. Thesis, Massachusetts Institute of Technology.
10.
Cary, M., & Reder, L. (2002). Metacognition in strategy selection: giving consciousness too much credit. In M. Izaute, P. Chambres, & P. J. Marescaux (Eds.), Metacognition: Process, function and use (pp. 63-78).
11.
Cazenave, T. (2003). Metarules to improve tactical go knowledge. Information Science, 154, 173-188.
12.
Clancey, W., & Bock, C. (1985). Representing control knowledge as abstract task and metarules. Technical Report, STAN-CS-87-1168, Department of Computer Science, Stanford University, Stanford, CA.
13.
Conitzer, V. (2008). Metareasoning as a formal computational problem . In AAAI-08 Workshop: Metareasoning : Thinking About Thinking.
14.
Cox, M. (2005). Metacognition in computation: a selected research review. Artificial Intelligence, 169, 104-141.
15.
Dastani, M., Governatori, G., Rotolo, A., Song, I., & Torre, L. van der. (2007). Contextual deliberation of cognitive agents in defeasible logic. In Proceedings of Autonomous Agents and Multi-Agent Systems (AAMAS- 2007).
16.
Di Paolo, E., & Harvey, I. (2003). Decisions and noise: The scope of evolutionary synthesis and dynamical analysis. Adaptive Behavior, 11, 284-288.
17.
Dunlosky, J.T., & Hertzog, C. (1998). Training programs to improve learning in later adulthood: Helping older adults educate themselves. In J. Hacker, J. Dunlosky, & A. C. Graesser (Eds), Metacognition in educational theory and practice (pp. 249-275). Mahwah, NJ : Lawrence Erlbaum Associates.
18.
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence . Current Directions in Psychological Science, 12, 83-87.
19.
Fernandez-Duque, D., Baird, J.A., & Posner, M.I. (2000). Executive attention and metacognitive regulation . Consciousness and Cognition, 9, 288-307.
20.
Flavell, H. (1971). First discussant's comments: What is memory development the development of? Human Development, 14, 272-278.
21.
Flavell, H. (1976). Metacognitive aspects of problem solving. In The nature of intelligence (pp. 231-235).
22.
Garner, B. (2000). Meta-knowledge acquisition strategies in asynchronous learning frameworks. In Proceedings of the International Workshop on Advanced Learning Technologies (IWALT-00) (pp. 247-248).
23.
Giraud-Carrier, C., Vilalta, R., & Brazdil, P. (2004). Introduction to the special issue on meta-learning . Machine Learning, 54, 187-193.
24.
Gordon, A., & Hobbs, J. (2003). Coverage and competency in formal theories: A commonsense theory of memory. In Proceedings of AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning.
25.
Gordon, A., Hobbs, J., & Cox, M. (2008). Anthropomorphic self-models for metareasoning agents. In AAAI-08 Workshop: Metareasoning: Thinking About Thinking.
26.
Greve, K., Stickle, T., Love, J., Bianchini, K., & Stanford, M. (2005). Latent structure of the Wisconsin Card Sorting Test: A confirmatory factor analytic study. Archives of Clinical Neuropsychology, 20, 355-364.
27.
Hu, X., & Edwards, D. (2006). Context-dependent structure control for adaptive behavior selection. In Proceedings of the Workshop on Bio-inspired Cooperative and Adaptive Behaviors in Robots, 9th International Conference on the Simulation of Adaptive Behavior, (SAB-06).
28.
Iizuka, H., & Paolo, E.A.D. (2007). Toward spinozist robotics: exploring the minimal dynamics of behavioral preference. Adaptive Behavior, 15, 359-376.
29.
Ikegami, T., & Iizuka, H. (2003). Joint attention and dynamics repertoire in coupled dynamical recognizers. In Proceedings of the 2nd International Symposium on Imitation in Animals and Artifacts (AISB-03). (pp. 125-130).
30.
Joel, D., Weiner, I., & Feldon, J. (1997). Electrolytic lesions of the medial prefrontal cortex in rats disrupt performance on an analog of the Wisconsin Card Sorting Test, but do not disrupt latent inhibition: Implications for animal models of schizophrenia. Behavioural Brain Research, 85, 187- 201.
Kaplan, G., Sengör, N., Gürvit, H., Genç, I., & Güzelis, C. (2006). A composite neural network model for perseveration and distractibility in the wisconsin card sorting test. Neural Networks, 19, 375-387.
33.
Kelso, J. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press.
34.
Klinkenberg, R. (2005). Meta-learning, model selection, and example selection in machine learning domains with concept drift. In Proceedings of Lernen, Wissensentdeckung und Adaptivitat (LWA-05) (pp. 164-171).
35.
Maniadakis, M., & Tani, J. (2008). Dynamical systems account for meta-level cognition . In Proceedings of the 10th International Conference on the Simulation of Adaptive Behavior (SAB-08).
36.
Maniadakis, M., & Trahanias, P. (2006). Hierarchical cooperative coevolution facilitates the redesign of agent-based systems. In Proceedings of the 9th International Conference on the Simulation of Adaptive Behavior, (SAB-06) (pp. 582-593).
37.
Mansouri, F., & Tanaka, K. (2003). Wisconsin Card Sorting Test with macaque monkeys . International Congress Series, 1250, 105-118.
38.
Meeden, L.A. (1996). An incremental approach to developing intelligent neural network controllers for robots. IEEE Transactions on Systems, Man, and Cybernetics, 26, 474- 485.
39.
Milner, B. (1963). Effect of different brain lesions on card sorting . Archives of Neurology, 9, 90-100.
40.
Nelson, T., & Narens, L. (1990). Metamemory: A theoretical framework and new findings . In The Psychology of Learning and Motivation , 125-173.
41.
Nishimoto, R., & Tani, J. (2004). Learning to generate combinatorial action sequences utilizing the initial sensitivity of deterministic dynamical systems. Neural Networks, 17, 925-933.
42.
Nolfi, S., & Floreano, D. (2000). Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. Cambridge, MA: MIT Press/Bradford Books.
43.
Paine, R., & Tani, J. (2005). How hierarchical control self-organizes in artificial adaptive systems. Adaptive Behavior, 13, 211-225.
44.
Proust, J. (2003). Does metacognition necessarily involve metarepresentation ? Behavior and Brain Sciences, 26, 352- 355.
45.
Ramesh, B. (1997). Towards a meta-model for representing organizational memory. In Proceedings of the 30th Hawaii International Conference on System Sciences (HICSS-97) (pp. 320-329).
46.
Reder, L., & Schunn, C. (1996). Metacognition does not imply awareness: strategy choice is governed by implicit learning and memory. In L. Reder (Ed.), Implicit Memory and Metacognition (pp. 45-78). Mahwah, NJ: Erlbaum.
47.
Russell, S. (1991). Principles of metareasoning. Artificial Intelligence, 49, 361-395.
48.
So, R., & Sonenberg, L. (2004). Situation awareness in intelligent agents: Foundations for a theory of proactive agent behavior. In Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT'04).
49.
Stemme, A., Deco, G., & Busch, A. (2007). The neuronal dynamics underlying cognitive flexibility in set shifting tasks. Journal of Computational Neuroscience , 23, 313-331.
50.
Sun, R., Zhang, X., & Mathews, R. (2006). Modeling metacognition in a cognitive architecture . Cognitive Systems Research, 7, 327-338.
51.
Swanson, H.L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306-314.
52.
Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: a robot experiment. IEEE Transactions on Systems, Man, and Cybernetics Part A, 33, 481-488.
53.
Tuci, E., Quinn, M., & Harvey, I. (2002). An evolutionary approach to the study of learning behavior using a robot-based model. Adaptive Behavior, 10, 201-221.
54.
Tuci, E., Trianni, V., & Dorigo, M. (2004). Feeling of time through sensory-motor coordination. Connection Science Journal, Special Issue on Evolutionary Robotics, 16, 301- 324.
55.
Van Gelder, T. (1998). The dynamical hypothesis is cognitive science . Behavioral and Brain Sciences, 21, 615-628.
56.
Varela, F.J. (1979). Principles of biological autonomy. McGraw-Hill/Appleton & Lange.
57.
Weischedel, R.M., & Sondheimer, N.K. (1983). Meta-rules as a basis for processing ill-formed input. American Journal of Computational Linguistics, 9, 161-177.
58.
Yamauchi, B.M., & Beer, R.D. (1996). Spatial learning for navigation in dynamic environment . IEEE Transactions on Systems, Man, and Cybernetics Part A, 26.
59.
Yzerbyt, V., Lories, G., & Dardenne, B. (1998). Metacognition: Cognitive and social dimensions . London: Sage Publications Ltd .
60.
Ziemke, T. (1996). Towards adaptive behaviour system integration using connectionist infinite state automata. In Maes et al. (Eds.) From Animals to Animats 4 (pp. 145- 154). Cambridge, MA: MIT Press.
61.
Ziemke, T. (1999). Remembering how to behave: Recurrent neural networks for adaptive robot behavior. In l. Medsker & l. Jain (Eds.), Recurrent neural networks: Design and applications (pp. 359-389).
62.
Ziemke, T., & Thieme, M. (2002). Neuromodulation of reactive sensorimotor mappings as short-term memory mechanism in delayed response tasks. Adaptive Behavior, 10, 185- 199.
