Abstract
This article offers new insights on the learning control approach developed by [Hu et al. IEEE/ASME Trans. Mechatronics, 19(1): 191–200, 2014]. Theoretical insights are further proposed to unveil why the contraction-type iterative learning control (ILC) schemes are suitable and effective in compensating for hysteresis, widely existing in biorobotic locomotion. Under such circumstances, iteration-based second-order dynamics is adopted to describe the biorobotic systems acted upon by one unknown Preisach hysteresis term. The memory clearing operator is mathematically proven to enable feasibility of contraction-type ILC methods, regardless of whether the initial state is accurately set or not. The simulation examples confirm that the developed iteration-based controller combined with a preceded operator effectively reduce tracking errors caused by the hysteresis nonlinearity. Furthermore, the new insights on theoretical feasibility are definitively corroborated in accordance with the previously published experimental results.
1. Introduction
1.1. Overview of biorobotic systems
Biorobotics, or biologically inspired robotics, integrates the interests of both biologists and roboticists. In the past few years, an abundance of scientific collaboration and prototype development has been undertaken in this field [1–3]. Indeed, biorobotics is increasingly contributing to scientific understanding of, and inspiration from, biological principles. Bio-inspirations are used to develop robotic systems that satisfy the practical requirements of stability, manoeuvrability, autonomy and endurance. In the opposite direction, biorobots have also served as important scientific tools in the investigation of animal locomotion, when used as physical models. Biorobots are employed in the testing of hypotheses since they are qualified for repeatable and parameterized experiments [1].
When biorobots are employed in investigation of animal locomotion, the question arises as to whether and to what extent the developed robotic systems can replicate animal movement and behaviours. In fact, inconsistency between the desired and actual performance is comprehensive in bio-inspired robotic locomotion. The robotic undulating fin is a typical case.
1.2. Control problem of reproducing animal locomotion
Since 2001, bio-inspired undulating fin models [4–11] have been developed to investigate the undulation principles of knifefish. The aforementioned inconsistency does exist and has prevented robotic fish models from replicating the locomotion or behaviours of their biological counterparts. An experimental-numerical approach was employed to reveal that the tracking inconsistency is jointly caused by hysteresis nonlinearity from hydrodynamics and the elastic effect of the membrane [12, 13]. Significantly, some bio-inspired joints or components are driven by smart materials that possess an inherent feature of hysteresis nonlinearity. The smart materials employed in biorobotics include shape memory alloy (SMA), ionic polymer metal composite (IPMC) and piezo-based actuators [14–18].
Iterative learning control (ILC) schemes have been designed and used for robotic arms and other dynamic systems performing repetitive tasks [19–21]. Such iteration-based control methods are model-free and have no critical dependence on system dynamics. Hu et al. [12, 22, 23] thus employed ILC into biorobotic fins due to the repeatability of undulations. A hybrid control scheme of anticipant ILC, associated with memory clearing, was proposed and implemented to erase tracking errors. Experimental results showed that the learning controller combined with a memory clearing operator effectively enabled the robotic undulating fin to replicate the fin-ray undulations of live fish.
1.3. Organization of this paper
This paper is a further study of research [12, 22, 23] that experimentally demonstrates the ILC scheme's mimicking of fin-ray undulating gaits. Particularly, this study concentrates on theoretical insights regarding the use of the proposed ILC scheme to compensate for hysteresis of bio-inspired fin-ray undulations.
The remainder of the paper is organized as follows. Section 2 presents an iteration-based state-space dynamics, modelling the biomimetic systems acted upon by one unknown hysteresis term. The classic Preisach model is adopted to describe the unknown hysteresis nonlinearity. In Section 3, the memory clearing operator is mathematically proven to enable the feasibility of ILC methods, even when the initial states are not identically set at the beginning of each iteration. The simulation examples are conducted in Section 4. Consistency between the simulation and corresponding experiment confirms theoretical feasibility and generality of the Preisach hysteresis compensation ILC scheme. Finally, Section 5 offers some concluding remarks.
2. Biorobotic Systems with Preisach Hysteresis
2.1. Dynamics model
As for the theoretical study of the hysteresis-compensating control scheme, two degree of freedom (2DOF) bio-inspired joints or components are centred due to their basic foundation and generality for biorobotic prototypes. Hereafter, we consider the iteration-based second-order biorobotic dynamic systems as described by
where t denotes the time and the nonnegative subscript k denotes the operation or iteration number; θ(t) denotes the biorobotic joint angle; y(t) the system output; u(t) the control input containing the torques or forces to be applied at the biorobotic joint; M(θ) the inertia matrix; C(θ, θ̇) a coefficient resulting from Coriolis and centrifugal forces; G(θ) the dynamic term resulting from gravitational forces and h(t) the hysteresis nonlinearity in the biorobotic joints.
2.2. Preisach-type hysteresis
Hysteresis is a form of nonlinearity with memory. In published hysteresis models [24–27], the Preisach model straddles the boundary between physics and mathematics. In recent decades, it has been well studied, widely accepted and successfully applied.
According to the Preisach-type hysteresis [24], the mentioned term h(t) in (1) can be expressed as
where
where
3. Theoretical Insights
This study penetrates into theoretical insights on the basis of experimental validation of the learning control approach developed by Hu et al. in 2014 [22]. Specifically, it concentrates on the Lipschitz condition of contraction-type ILC schemes that are mostly practical within robotic systems of repetitive tasks.
3.1. ILC scheme with a memory-clearing operator
To our best knowledge, the Lipschitz condition is necessary for the contraction-type ILC methods [19–21]. Nonetheless, the Preisach hysteresis in dynamic systems (1)–(2) is demonstrated not to hold the Lipschitz condition by using a counterexample [22]. Under such circumstances, the memory clearing operator was developed and fused into ILC algorithms. The iteration-based approach means rectifying trajectories to clear hysteresis effects. The hysteresis memory clearing operator is given as
where {θ d (t), t∈[0, T]} denotes the desired locomotive trajectory of biorobotic prototypes, {θ d * (t), t∈[0, T * ]} the rectified biorobotic desired trajectory, sgn(•) the sign function, T r >0 the preceded rectifying interval and T * =T+T r .
In terms of the counterexample [22], a parabolic rectification is defined as
where
Using the anticipant scheme [12, 21–23] as a representative of contraction-type ILC methods, the iterative learning control law is stated in the form
where L denotes the learning matrix
and
Note that the desired input, θ d (t), t∈[0, T], is preceded by a rectifying section, θ d * (t), t∈[0, T r ]. This rectifying section is a parabola, which is confirmed with the following constraints: (i) attaching the prescribed minimum of α or β only once, and (ii) connecting the hysteresis initial memory point θ d (0−) and the desired input starting point θ d (0) smoothly.
3.2. Terminology and formulation
Continuity and repeatability have been confirmed in dynamic systems with Preisach hysteresis in [28]. We must now further consider the satisfaction of the Lipschitz condition. From the counterexample [22], we can see that the initial hysteresis state has a remarkable effect on ILC methods. Thereafter, a specific hysteresis state should be determined as the starting state reference during iterations.
is defined as Minimum Hysteresis Initial Memory State.
is defined as the Minimum Hysteresis Output.
Once the minimum hysteresis initial memory state is confirmed before the first iteration, the desired output corresponds to one and only one hysteresis initial state. That is, a desired output, y d (t), being continuous for t∈Γ is achievable with a unique input, u d (t), for t∈Γ and a unique minimum initial state, hd,0-. This desired input, u d (t), is bounded and so is the corresponding state sequence, θ d (t), t∈Γ.
3.3. Main theorem
To state and prove the convergence of the ILC, the following assumptions and properties are stated:
That is
where, k u and k o are both positive constants.
Under these assumptions and the previously presented analysis, this study focuses on whether hysteretic systems acted upon by the updated ILC law (8) satisfy the Lipschitz condition.
In light of Lemma 1, since μ(α, β) is bounded, the hysteresis outputs with θ d (t) and θ i (t), t∈Γ, satisfy
where k u and k o are both positive constants.
According to the initial states, two cases will be respectively considered in the following procedure.
Defining Ω d and Ω i as the Minimum Hysteresis Initial Positive Preisach Plane, we have
In view of the intuitive geometrical interpretation of the Preisach hysteresis model, shown in Figure 1, using θ i (0)=θ d (0) yields
Using (16) and (20), and setting the positive constant k v = k u , we achieve
Since μ(α, β) is bounded over the Preisach plane, the maximum is defined as
According to (12), we have
Combining (21), (22) and |θ i (0) - θ d (0)|<δ implies that Ω d ≠Ω i . Thereafter, define
as shown in Figure 1. In addition, S Ω is defined as the area of Ω.
According to the norm definition
we obtain
Substituting (26) into (24) produces
Combining (16) and (27) generates
Similarly, define a positive constant k v , as
Substituting (29) into (28), we obtain
Thus, the above equation is satisfied in both cases, θ d (0)= θ i (0) or 0<|θ d (0)-θ i (0)|<δ.

The Preisach planes with several minimum initial memory states for the proof of Theorem 1
4. Examples and Simulations
4.1. Tracking performance of the developed ILC scheme associated with the memory-clearing operator
In this simulation, the second-order dynamic system (1)–(2) is considered. The dynamic system parameters are set as M(θ)=1 and C(θ, θ̇)=G(θ)=0, and the Preisach density is given as
The operation cycle is T=6s and the desired trajectory is given as
The rectifying interval is T r =2s, the sampling interval is set as Δ=20ms and the learning matrix is set as
and
Two iterative learning procedures are carried out to validate the memory clearing operator's performance in eliminating the hysteresis effect. In the first procedure, the conventional ILC without memory clearing is applied to the dynamic system (1)–(2). To validate the effectiveness of the conventional anticipant ILC in hysteretic systems, the initial memory state is set as a random real number within a specified bound. The dashed line of Figure 2 exhibits the corresponding robot management system (RMS) errors against iterations. Naturally, the tracking accuracy is enhanced with iterative learning. After only a few iterations, RMS error decreases quickly into a limited region, compared with the original tracking error achieved by the independent feedback control.
This clearly illustrates the convergence of the tracking errors in the hysteretic system and, thus, ILC can effectively compensate for the hysteresis nonlinearity in dynamic systems. However, a specified steady RMS error cannot be further reduced following several iterations, as the dashed line shows in Figure 2. This steady RMS error is related to the hysteresis initial state, which will be uncovered in the coming procedure.
As for this example, the memory clearing operator, O mc , is introduced into the anticipant ILC [21, 22]. In terms of practical applications, the Minimum Hysteresis Initial Memory States are unlikely to be identical during iterations, so the starting point for the hysteresis loops is iteratively reset with some errors within a specified boundary. The solid line of Figure 2 shows the corresponding RMS errors against iterations in the second procedure.
Comparison between the conventional and developed ILC schemes is presented in Figure 2. The steady RMS error is located in a more limited region when the memory-clearing operator is applied. Specifically, the A-type ILC is preceded by the memory clearing operator, and controls the steady error to noise levels. It is clear that the tracking accuracy is further improved by the memory operator when compared with the ILC method alone. Simulations validate the veracity of Theorem 1, and also the conventional ILC methods’ applicability to other biorobotic systems with Preisach hysteresis nonlinearity.

Biorobotic tracking performance against learning iterations. The dashed line denotes the tracking RMS errors for the conventional ILC, while the solid line denotes the tracking performance for the developed ILC associated with the memory clearing operator, O mc .
4.2. Corresponding simulation for experiments involving biorobotic undulating fins
Biorobotic undulating fins have been developed to improve understanding of the swimming locomotion and adaptive behaviours of Gymnarchus niloticus, eel and knifefish [4–12]. Experimental results have confirmed that the proposed ILC scheme associated with memory clearing can effectively conquer the hysteresis nonlinearity [12].
A corresponding comparison simulation is conducted to test the feasibility and generality of the proposed theoretical insights of this study. The desired trajectories and controller parameters are kept consistent between the simulation and the previously assigned experiments [12], thereby ensuring that the results from the two approaches are comparable. Three sections are designed for performance comparison, corresponding to pre-, on- and post-learning. Specifically, the learning controller is not enabled for the pre-learning section and, therefore, the original parameters are retained; the controller is enabled for the on-learning section to tune the control parameters online; the learning controller is disabled in the post-learning section to retain the learned parameters. The trial values for the three sequential sections are five, 10 and 10, respectively (see Figure 3). That is to say, in total, there are 25 trials for each simulation or experiment. Comparison between the first and third sections concentrates on the improvement made by the learning controller, while the second on-learning section focuses on the updating procedure it enacts.
As shown in Figure 3, the new insights on theoretical feasibility are definitively in accordance with the previously published experimental results. The RMS tracking errors of the two approaches share quite consistent tendencies within all three sections. Certainly, simulations present smoother learning convergence than experiments, due to environmental disturbance and measurement noise of engineering practice in the latter. Both of the improved tracking RMS errors drift around, approximately, one degree due to their use of the iteratively tuned parameters at the 15th iteration. Such consistency confirms that the presented theoretical insights are useful for broadening the application scope of the novel ILC scheme. That is to say, the developed ILC scheme combined with memory clearing is not only experimentally feasible for the case of a biorobotic undulating fin, but also theoretically widens its generality to other biorobotic systems associated with Preisach hysteresis.

Performance tracking along with iterations from both theoretical simulations and corresponding experiments. For experimental results, refer to [12]. Three sections of pre-, on and post- learning are sequentially presented.
5. Concluding Remarks
In this paper, two issues were considered for theoretical investigation based on previously accomplished experimental and numerical results. Firstly, it was mathematically proven that the memory clearing operator enables the hysteretic term satisfying the Lipschitz condition. This is necessary for the feasibility of the contraction mapping-based ILC scheme. Secondly, the proposed ILC scheme associated with memory clearing was further explored from the point of view of general application. Consistency was confirmed between the corresponding simulations and experiments and these theoretical insights demonstrated the scheme's generality, from bio-inspired undulating fins to various biorobotic prototypes associated with hysteresis.
Specifically, the present work extends and consolidates the applicable generality from fish-like locomotion to bio-inspired robotic locomotion, such as climbing, walking, running, flying, swimming and multi-modal behaviours. The present work can contribute to the control design of smart materials [29] that are used as biorobotic actuators, due to their close relationship to challenging the compensation of hysteresis nonlinearity.
Footnotes
6. Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC) under Grants 61005077 and 61273347. This article is a revised and expanded version of a paper entitled ”Applying Iterative Learning Control into Preisach-Type Hysteresis Compensation for Bio-Inspired Locomotion”, presented at the 2015 International Conference on Climbing and Walking Robots (CLAWAR 2015), Hangzhou, China, September 6–9, 2015.
