Joint Range of Motion (JROM) development has been shown to facilitate learning motor control in human beings. This developmental strategy has been applied in robotics to improve learning performance with different outcomes: sometimes it is favourable, others irrelevant, and others, even detrimental. The reasons that underpin this variability in the results are still not well understood. In this paper, we seek to better understand the principles underlying the application of JROM based morphological development to make its use more straightforward. To this end, empirical studies were carried out over two representative use cases: quadruped and bipedal robot morphologies learning to walk. Different parameters of the application of JROM development (morphological configuration, JROM developmental strategy, etc.) have been evaluated to elucidate their effects over learning. The results show that there are significant connections between the reduction of the motor space induced by JROM and the way the exploration and exploitation of the solution space is carried out by the learning algorithm, and the performance achieved. Through these connections, we have identified a set of conditions that must be satisfied for JROM development to be effective as a tool for learning improvement.
TommasinoPCaligioreDMirolliM, et al.A reinforcement learning architecture that transfers knowledge between skills when solving multiple tasks. IEEE Trans Cognit Dev Syst2019; 11: 292–317.
2.
BecerraJARomeroABellasF, et al.Motivational engine and long-term memory coupling within a cognitive architecture for lifelong open-ended learning. Neurocomputing2021; 452: 341–354.
3.
XieZJinY. An extended reinforcement learning framework to model cognitive development with enactive pattern representation. IEEE Trans Cognit Dev Syst2018; 10: 738–750.
4.
RomeroAMedenBBellasF, et al.Using perceptual classes to dream policies in open-ended learning robotics. Integr Comput Aided Eng2023; 30: 205–222.
5.
PrietoARomeroABellasF, et al.Introducing separable utility regions in a motivational engine for cognitive developmental robotics. Integr Comput Aided Eng2019; 26: 3–20.
6.
RomeroABellasFPrietoA, et al.Developmental learning of value functions in a motivational system for cognitive robotics. In: 2020 International joint conference on neural networks (IJCNN), 2020, pp.1–8. IEEE.
7.
NadizarGMedvetEMirasK. On the schedule for morphological development of evolved modular soft robots. In: Medvet E, Pappa G and Xue B (eds) Genetic Programming. Cham: Springer International Publishing, 2022, pp.146–161.
8.
Max LungarellaRPGiorgioMettaSandiniG. Developmental robotics: a survey. Connect Sci2003; 15: 151–190.
9.
MüllerVCHoffmannM. What is morphological computation? on how the body contributes to cognition and control. Artif Life2017; 23: 1–24.
10.
FreybergSHauserH. The morphological paradigm in robotics. Stud Hist Philos Sci2023; 100: 1–11.
11.
Naya-VarelaMFaíñaADuroRJ. Morphological development in robotic learning: a survey. IEEE Trans Cognit Dev Syst2021; 13: 750–768.
12.
ZhuJRongCIidaF, et al.Bootstrapping virtual bipedal walkers with robotics scaffolded learning. Front Robot AI2021; 8
13.
Naya-VarelaMFainaAMalloA, et al.A study of growth based morphological development in neural network controlled walkers. Neurocomputing2022; 500: 279–294.
14.
SavastanoPNolfiS. A robotic model of reaching and grasping development. IEEE Trans Auton Ment Dev2013; 5: 326–336.
15.
GómezGEggenberger HotzP. Investigations on the robustness of an evolved learning mechanism for a robot arm. In: Proc. of the 8th int. conf. on intelligent autonomous systems (IAS-8), 2004, pp.818–827.
16.
BenureauFCYTaniJ. Morphological development at the evolutionary timescale: robotic developmental evolution. Artif Life2022; 28: 3–21.
17.
LungarellaMBerthouzeL. On the interplay between morphological neural environmental dynamics: a robotic case study. Adapt Behav2002; 10: 223–241.
18.
LiuXRongHNeriF, et al.Deep deterministic policy gradient with constraints for gait optimisation of biped robots. Integr Comput Aided Eng2024; 31: 139–156.
19.
BernsteinN. The Coordination and Regulation of Movements. London: Pergamon Press, 1967.
20.
GrayR. Changes in movement coordination associated with skill acquisition in baseball batting: freezing/freeing degrees of freedom and functional variability. Front Psychol2020; 11.
21.
BongaardtRMeijerOG. Bernstein’s theory of movement behavior: historical development and contemporary relevance. J Motor Behav2000; 32: 57–71.
22.
BerthierNECliftonRKMcCallDD, et al.Proximodistal structure of early reaching in human infants. Exp Brain Res1999; 127: 259–269.
23.
ArutyunyanGAGurfinkelVSMirskiiML. Investigation of aiming at a target. Biophysics1968; 13: 642–645.
24.
McDonaldPVVan EmmerikREANewellKM. The effects of practice on limb kinematics in a throwing task. J Mot Behav1989; 21: 245–264.
25.
VereijkenBEmmerikREAWhitingHTA, et al.Free(z)ing degrees of freedom in skill acquisition. J Mot Behav1992; 24: 133–142.
26.
HaehlVVardaxisVUlrichB. Learning to cruise: bernstein’s theory applied to skill acquisition during infancy. Hum Move Sci2000; 19: 685–715.
27.
Ramírez-ContlaSCangelosiAMaroccoD. Developing motor skills for reaching by progressively unlocking degrees of freedom on the icub humanoid robot. In: Proceedings of the post-graduate conference on robotics and development of cognition, 2012.
28.
BerthouzeLLungarellaM. Motor skill acquisition under environmental perturbations: on the necessity of alternate freezing and freeing of degrees of freedom. Adapt Behav2004; 12: 47–64.
29.
BongardJ. Morphological change in machines accelerates the evolution of robust behavior. Proc Natl Acad Sci2011; 108: 1234–1239.
30.
BaranesAOudeyerPY. The interaction of maturational constraints and intrinsic motivations in active motor development. In: IEEE international conference on development and learning (ICDL), 2011, pp.1–8.
31.
LapeyreMLyOOudeyerPY. Maturational constraints for motor learning in high-dimensions: The case of biped walking. In: 2011 11th IEEE-RAS international conference on humanoid robots, 2011, pp.707–714.
32.
WuR, et al.A developmental evolutionary learning framework for robotic Chinese stroke writing. IEEE Trans Cognit Dev Syst2022; 14: 1155–1169.
33.
BraudRGiagkosAShawP, et al.Robot multimodal object perception recognition: synthetic maturation of sensorimotor learning in embodied systems. IEEE Trans Cognit Dev Syst2020; 13: 416–428.
34.
GiagkosALewkowiczDShawP, et al.Perception of localized features during robotic sensorimotor development. IEEE Trans Cognit Dev Syst2017; 9: 127–140.
Campos-AlfaroFJaraCRomeroA, et al.Learning adaptable utility models for morphological diversity. In: International work-conference on the interplay between natural and artificial computation, 2024, pp.105–115. Springer.
37.
RomeroABellasFBecerraJA, et al.Motivation as a tool for designing lifelong learning robots. Integr Comput Aided Eng2020; 27: 353–372.
38.
IvanchenkoVJacobsRA. A developmental approach aids motor learning. Neural Comput2003; 15: 2051–2065.
39.
GómezGLungarellaMEggenberger HotzP, et al.Simulating development in a real robot: on the concurrent increase of sensory, motor, and neural complexity. 2004.
40.
ChoiKChoeYParkH. Reinforcement learning may demystify the limited human motor learning efficacy due to visual-proprioceptive mismatch. Int J Neural Syst2024; 34: 2450037.
41.
Naya-VarelaMFainaADuroRJ. Engineering morphological development in a robotic bipedal walking problem: an empirical study. Neurocomputing2023; 527: 83–99.
42.
LiuXRongHNeriF, et al.Entropy-weighted numerical gradient optimization spiking neural system for biped robot control. Int J Neural Syst2024; 34: 2450030.
43.
LiuXZhangGMastoiMS, et al.A human-simulated fuzzy membrane approach for the joint controller of walking biped robots. Integr Comput Aided Eng2023; 30: 105–120.
44.
Naya-VarelaMFainaADuroRJ. Learning bipedal walking through morphological development. In: Hybrid artificial intelligent systems: 16th international conference, HAIS 2021, Bilbao, Spain, September 22–24, 2021, Proceedings 16, 2021. pp.184–195. Springer.
