Sage Journals: Discover world-class research

Abstract

This study investigated the stability of virtual reality (VR) haptic feedback from the perspective of energy. A fuzzy controller was designed on the basis of passivity theory to mitigate the shock and divergence in a virtual component caused by the negative damping coefficient and the high spring coefficient. Compared with a standard passivity controller, a fuzzy-passivity controller is more effective in shortening the time required to stabilize a system with high initial energy and devices that consume energy slowly. First, MATLAB/Simulink was employed for stability simulation and analysis. The fuzzy-passivity controller provided more favorable stability convergence than did the standard passivity controller. The simulation results align with expectations. Second, the completed VR system was applied as an experiment platform for verifying the feasibility of the fuzzy-passivity controller. To achieve one degree of freedom in the experiments, a robotic arm was exclusively designed to replace manual force. Finally, compared with standard passivity control systems, the results indicated that the fuzzy-passivity controller enables improved control over the stability of VR haptic feedback.

Keywords

Haptic feedback fuzzy-passivity control virtual reality (VR)

Introduction

Advances in information, visual display, and haptic feedback technologies have generated VR applications for medical surgery simulations.^1–4 The VR surgery training systems enable users to train using simulations in a safe and realistic environment. Relevant studies have proposed such applications; for example, the Department of Oto-Rhino-Laryngology in Helios Hospital Krefeld, Germany, used VR to provide real-time simulation of endoscopic sinus surgery.⁵ The Department of Otorhinolaryngology in Seoul National University Hospital has also applied augmented reality for medical applications,⁶ and other scholars have achieved promising results in applications including remote surgery⁷ and medical robots.^8–11 In VR technology, the stability of haptic feedback is vital because the computers that create virtual environments use a digital discrete-time system in which system information in generated through data samples. However, reality is a continuous-time system. The interaction between the two-time systems inevitably causes stability problems during haptic simulations, specifically data sampling, integration, or quantification errors.

Researchers have explored the haptic feedback stability of VR in medical applications.^12–14 In particular, Hannaford and Ryu¹⁴ employed energy reference as a foundation to propose a control method for haptic interfaces on the basis of time-domain passivity. The control design achieved stability between the passivity observer and controller. However, sampling frequencies are often too low, making them insufficient for achieving dynamic responses in VR,¹⁵ which results in energy estimation errors. Therefore, Ryu proposed a new passivity observer to resolve the energy estimation errors caused by changes in sampling speed.^16–18 Adams¹²developed a virtual coupling method based on a two-port network; this method employs the dynamic mathematical model of the system to infer the optimal coupling parameters of the haptic device and the VR environment. Although less secure in comparison with passivity theorem, this method only requires data on the energy passivity of the VR environment to determine the coupling parameters optimal for achieving system stability. Colgate and Schenkel¹⁹ performed frequency domain analysis on a closed-loop system and applied passivity theorem to determine the system’s stability range according to VR environment parameters. By adopting the concept of linear circuit networks, Colgate and Brown²⁰ hypothesized that both the operating end and the VR environment achieved energy passivity and inferred the virtual coupling parameters to ensure a stable interaction between the system and the VR environment. Standard passivity controllers can provide compensation to ensure stability through oscillation when the system total output is lower than zero. However, more time is required for such controllers to stabilize systems with large initial energy and which contain devices that slowly consume energy.

Most control systems for complex nonlinear systems²¹ are based on nonlinear control theorems and smart control. In recent years, feedback linearization has always been an active research field, especially the related results in system control are also the focus of attention, such that sliding mode control.^22–25 Feedback linearization requires extremely accurate measurements of system parameters to eliminate the effects of system nonlinearity in order to achieve the desired results. Conversely, certain smart control systems (e.g. those based on fuzzy control theory^26,27) can accurately achieve target control without information on the actual system parameters and model. Many works use non linear hybrid control structure, combining fuzzy and passivity based control mode. The fuzzy logic controller is adopted by the Passivity Controller to design a desired torque dynamic to guarantee a fast convergence, stability, and robustness against parameter variations of the closed-loop, such as for flexible joint robot,^28,29 the adaptive-fuzzy passivitybased control of power systems,³⁰ the position Control of an Induction Machine,³¹ and the PMSG-based tidal turbine.³² These were different areas of research with this study investigated the stability of VR haptic feedback from the perspective of energy. Because, for real-time haptic VR applications, haptic rendering requires both fidelity and stability.

The main purposes of this thesis are to investigate a stable haptic simulation development scheme for modules on the basis of deformable simulation technology, haptic feedback technology, and information technology and to alleviate the oscillation and divergence effects caused by negative damping coefficients and high spring constants, the stability of haptic feedback is considered. To increase stability response speed, passivity theorem and the stability parameters obtained from closed-loop system analysis were combined, and system energy prediction and the two-way energy compensation functions of fuzzy theory were employed to design a fuzzy-passivity controller. The proposed controller can optimize system stability by changing its parameters according to changes in virtual environment parameters. In the design stage, MATLAB/SIMULINK software and PC-based VR system are used to verify the rapidly stable control of this scheme, respectively. In addition, a haptic-based training platform is developed to regard as proving the experiment platform of this fuzzy-passivity controller of research. Besides, to reach the consistency of the experiment, we designed the exclusive mechanical robot to replace the way of manpower operation. Finally, compare with basic passivity, rapidly stable control is achieved under the fuzzy-passivity control.

In the remainder of this paper, Section 2 presents the stability analysis method for the VR environment, and Section 3 describes the system framework and the fuzzy-passivity controller design. Section 4 outlines the experiment simulation and overall testing process, and Section 5 presents the conclusion and future research directions.

Stability analysis of VR environments

Passivity theorem

Passivity theorem is an analysis method based on the fundamental energy concepts of nonlinear systems. The original construct of the theorem originated from digital circuit concepts. We combined passivity theorem with network concepts to explore the relationship between system input and output energy. If system total energy is greater than zero, the system must consume energy before achieving stability; if the system total energy is zero, oscillation will occur. We referenced the passivity-based control concepts on haptic feedback proposed by Hannaford and Ryu¹⁴ to define passivity control for continuous-time systems in single-port networks and discrete-time systems in multiple-port networks. The definitions of energy consumption in these systems are as follows:

Definition 1. In continuous-time systems, initial energy is represented as $E (0)$ . If and only if the single-port network is considered a passivity system, then

\int_{0}^{t} f (τ) v (τ) d τ + E (0) \geq 0, \forall t \geq 0

(1)

where $f$ denotes energy $(N)$ .

Definition 2. In discrete-time systems, initial energy is represented as $E (0)$ . According to the general law of energy conservation proposed by Joule, energy exists in various convertible forms but does not disappear. For any given sampling time ( $Δ T$ ), if and only if the multiple-port network is considered a passivity system, then

Δ T \sum_{k = 0}^{n} f (k) v (k) + E (0) \geq 0, \forall n \geq 0

(2)

Additionally, Colgate and Schenkel combined passivity theorem with system frequency domain analysis properties and inferred a third relation between virtual environments and haptic devices¹⁹ as follows:

Definition 3. Data sampling systems in VR consume energy. Therefore, the relation between the device and VR environment is represented as

b > \frac{T}{2} \frac{1}{1 - \cos wT} Re {(1 - e^{- jwT}) E (e^{jwT})}, 0 \leq w \leq w_{N}

(3)

where $b$ denotes the device damping coefficient, $w$ denotes the system resonance frequency, $T$ denotes the sampling time, and $w_{N} = π / T$ denotes the Nyquist frequency.

Routh–Hurwitz stability analysis

The passivity control methods described in Section 2.1 can provide compensation in time-domain haptic environments through timely energy estimations without information on the haptic device properties. However, these methods do not provide a control analysis of the overall system and do not explain in detail the relationship between the haptic device and the virtual environment parameters. Therefore, we paired the virtual environment system transfer function by Gil et al.³² with the Routh–Hurwitz stability analysis to determine the relation between the haptic device parameters and the virtual environment parameters. An impedance-passivity controller was added to the system proposed by Gil to reanalyze stability and redesign the controller. In Figure 1, $G (s)$ is the mathematical model of the haptic device, $E (z)$ is the virtual environment, $C (z)$ is the controller parameter, $G_{z} (s)$ is the zero-order hold, $f_{h} (t)$ is the external force end, and X[k] $x (t)$ is the post-sampling displacement output of the haptic device.

Figure 1.

Haptic feedback system framework of the standard passivity controller.

On the basis of the aforementioned system framework, the system transfer function is derived as follows:

Let

\begin{matrix} G (s) = \frac{1}{m s^{2} + bs}, E (z) = K + B \frac{1 - z^{- 1}}{T} \\ C (z) = α \frac{1 - z^{- 1}}{T}, G_{z} (s) = \frac{1 - e^{- Ts}}{s} \end{matrix}

(4)

According to Figure 1, the relation between input and output is

{\begin{matrix} X (z) = Z {G (s) \cdot F_{h} (z)} - F_{pc} (z) \\ F_{pc} (z) = Z {G_{z} (s) \cdot G (s)} \cdot (E (z) + C (z)) \cdot X (z) \end{matrix}

(5)

After computing, the following equation is derived:

X (z) = \frac{Z {G (s) \cdot F_{h} (z)}}{1 + Z {G_{z} (s) \cdot G (s)} \cdot (E (z) + C (z))}

(6)

Let $F_{h}$ be a force constant. The aforementioned equation is represented as

\frac{X (z)}{F_{h} (z)} = \frac{Z {G (s)}}{1 + Z {G_{z} (s) \cdot G (s)} \cdot (E (z) + C (z))}

(7)

The characteristic equation is

1 + Z {G_{z} (s) \cdot G (s)} \cdot (E (z) + C (z))

(8)

In equation (8),

Z {G_{z} (s) \cdot G (s)}

(9)

= Z {\frac{1 - e^{Ts}}{s} \cdot \frac{1}{m s^{2} + bs}}

(10)

= (1 - z^{- 1}) z {\frac{1}{s^{2} (ms + b)}}

(11)

= (1 - z^{- 1}) z {\frac{1}{b s^{2}} + (- \frac{m}{b^{2}}) \frac{1}{s} + {(\frac{m}{b})}^{2} \frac{1}{ms + b}}

(12)

\begin{matrix} = (1 - z^{- 1}) \frac{1}{b^{2}} \\ [\frac{bTz (z - e^{- bT / m}) - mz (z - 1) (z - e^{- bT / m}) + mz {(z - 1)}^{2}}{{(z - 1)}^{2} (z - e^{- bT / m})}] \end{matrix}

(13)

= \frac{1}{b^{2}} \frac{(bT + ma - m) z + (m - bTa - ma)}{(z - a) (z - 1)}

(14)

where $a = e^{- bT / m}$

To determine the stability relation between the virtual environment parameters, controller parameters, and haptic device parameters, the position of the characteristic root on the z-plane must be decided. Let equation (8) be zero and substitute equations (4) and (14) into equation (8):

\frac{\begin{matrix} (- abT α z - 2 zam α + abTB + abT α + amKT z^{2} + 2 zmB + b T^{2} K z^{2} \\ + bTB z^{2} + bT α z^{2} + b^{2} Taz - b^{2} Ta z^{2} - mKT z^{2} + amB z^{2} + am α z^{2} \\ - 2 amBz - bTBz - amKTz - ab T^{2} Kz - abTBz - bT α z + mKTz \\ + amB - mB z^{2} - b^{2} T z^{2} - m α z^{2} + am α + 2 m α z + b^{2} T z^{3} - m α - mB) \end{matrix}}{z (z - 1) (- z + a) b^{2} T} = 0

(15)

The equation is compiled as

\begin{matrix} = > b^{2} T z^{3} \\ + {(- a - 1) T b^{2} + (a - 1) m α + (K T^{2} + T α + TB) b \\ + [(a - 1) KT + (a - 1) B] m} z^{2} \\ + {b^{2} Ta + [(- a - 1) BT - aK T^{2} + (- a - 1) T α] b \\ + [(- a + 1) KT + 2 (1 - a) B] m + 2 (1 - a) m α} z \\ + (a - 1) Bm + (a - 1) m α + (aT α + aBT) b = 0 \end{matrix}

(16)

To perform stability analysis on equation (16), bilinear transformation must be performed on the equation:

Let z = \frac{1 + w}{1 - w}, X_{1} = T b^{2}

\begin{matrix} X_{2} = (- a - 1) T b^{2} + (a - 1) m α + (K T^{2} + T α + TB) b \\ + [(a - 1) KT + (a - 1) B] m \end{matrix}

\begin{matrix} X_{3} = b^{2} Ta + [(- a - 1) BT - aK T^{2} + (- a - 1) T α] b \\ + [(- a + 1) KT + 2 (1 - a) B] m + 2 (1 - a) m α \end{matrix}

X_{4} = (a - 1) Bm + (a - 1) m α + (aT α + aBT) b be the constant .

Then $X_{1}, X_{2} and X_{3}$ be the $z^{3}, z^{2}, z^{1}$ coefficient, respectively. $X_{3}$ be be the constant.

After substituting into equation (16) and multiplying $(1 - w^{3})$ on both sides,

\begin{matrix} X_{1} - X_{2} + X_{3} - X_{4}) w^{3} + (3 X_{1} - X_{2} - X_{3} + 3 X_{4}) w^{2} \\ + (3 X_{1} + X_{2} - X_{3} - 3 X_{4}) w + (X_{1} + X_{2} + X_{3} + X_{4}) = 0 \end{matrix}

(17)

From the Routh–Hurwitz stability criterion, we derive four inequalities to determine the stability range of K and B as follows:

Let

\begin{matrix} Y_{1} = X_{1} - X_{2} + X_{3} - X_{4}, Y_{2} = 3 X_{1} + X_{2} - X_{3} - 3 X_{4} \\ Y_{3} = 3 X_{1} - X_{2} - X_{3} + 3 X_{4}, Y_{4} = X_{1} + X_{2} + X_{3} + X_{4} \end{matrix}

(18)

Substitute into equation (17) and the Routh array derived from this equation is

\begin{matrix} \begin{matrix} w^{3} \\ w^{2} \\ w^{1} \\ 1 \end{matrix} & | \begin{matrix} Y_{1} & Y_{2} \\ Y_{3} & Y_{4} \\ \frac{Y_{3} Y_{2} - Y_{1} Y_{4}}{Y_{3}} \\ Y_{4} \end{matrix} \end{matrix}

(19)

The system is stable if the first line of the stability criterion is greater, and the sign does not change after substitution. By using the criterion, the stability range of K, $B$ , and $α$ can be determined. The four inequalities are computed as

\begin{matrix} (1 + a) [bT (2 b - TK) - 2 bT (B + α)] \\ + m (1 - a) [2 TK + 4 (B + α)] > 0 \end{matrix}

(20)

4 b^{2} T + (a - 1) T^{2} bK + 4 BabT + 4 (a - 1) m (B + α) > 0

(21)

\frac{{\begin{matrix} (1 - a) T b^{4} + [aK T^{2} + (- a^{2} + 1) BT] b^{3} \\ + {[(- 1 + a) KT - {(1 - a)}^{2} (B + α)] m \\ + aKB T^{2} + a (1 - a) {(B + α)}^{2} T} b^{2} \\ + [(a^{2} - 1) K (B + α) T - {(1 - a)}^{2} {(B + α)}^{2}] mb \\ + {(a - 1)}^{2} m^{2} K (B + α) \end{matrix}}}{4 b^{2} T + ((- 1 + a) T^{2} K + 4 BaT) b + 4 (- 1 + a) m (B + α)} > 0

(22)

Kb T^{2} (1 - a) > 0

(23)

When using the Routh–Hurwitz stability criterion for analysis, the system is stable if the sign of the first line in the Routh table does not change. Accordingly, equations (20)–(23) can be all set as zero. However, because the device damping coefficient ( $b$ ) and sampling time ( $T$ ) in equation (23) are both positive and because $a$ is a positive number lower than 1, $(1 - a)$ is a positive number. $K$ denotes the spring constant and cannot be negative; thus, equation (23) must be positive. Therefore, when using the Routh–Hurwitz stability criterion to explore the stability relation, the four aforementioned inequalities must be greater than zero. The stability range of the four equations were drawn using MATLAB as follows:

In Figure 2(a) to (c) represent equations (20)–(22), respectively. Equation (23) is not presented because according to the equation, $K > 0$ is mandatory to ensure system stability; therefore equation (23) does not require presentation. Figure 2(d) depicts the intersection of the four equations. Figure 3 depicts a magnified image of Figure 2(d), indicating the limitations of the haptic system parameters and the controller parameters. Under certain circumstances, if the virtual environment damping coefficient is changed, the parameters may change from positive to negative or demonstrate negative damping characteristics. Therefore, the stability range of the virtual environment given a negative damping coefficient was also considered and is presented in Figure 4.

Figure 2.

Stability analysis after addition of the controller parameters: (a) to (c) represent the stability range of the equations (20)–(22), respectively and (d) depicts the intersection of the four equations(20)-(23).

Figure 3.

Routh–Hurwitz stability range of the system with the controller (B is positive).

Figure 4.

Routh–Hurwitz stability range of the system with the controller (B is positive or negative).

The maximum moment force output of the haptic device used (Phantom Omni) is 4 N. Figure 3 depicts the stability range of the virtual environment parameters. However, the maximum power output must also be considered when responding to the haptic feedback generated by the virtual environment to prevent the force generated by the controller parameters and the virtual environment parameters from exceeding the hardware limitations of the haptic device. Accordingly, a more suitable stability range of the virtual environment parameters and the controller parameters was selected and is presented in Figure 5, which presents the magnified image of the small parameter stability range displayed in Figure 4. Specifically, if the parameter is very small, the stability range would resemble a flat surface. The area between the surface and the three axes represents the virtual environment parameter range for achieving system stability and the compensation range for the controller parameters.

Figure 5.

Small parameter stability range.

Stability analysis method comparison

Consistent with the aforementioned stability analysis methods (Section 2.1), when the passivity theorem was applied to the time-domain, the real-time changes in the positivity and negativity of the system energy or the virtual environment were observed. However, the virtual environment parameters required to achieve system stability could not be determined. As described in Section 2.2, the Routh–Hurwitz stability criterion was employed to analyze the stability of the system characteristic equations. This method did not incorporate energy passivity concepts, and the system remained stable despite the estimated virtual environment energy being negative under certain virtual environment parameters. This is attributed to the damping value of the device, which consumes the energy generated in the virtual environment. Therefore, controllers equipped with energy compensation mechanisms designed using the estimation method presented in Section 2.1 are more stable because the estimation method does not consider the energy consumption of the device.

Additionally, the stability relation between the virtual environment parameters and the haptic device can be determined by combining the passivity theorem with frequency domain analysis. Because this method considers system energy consumption characteristics, virtual environment parameters set using this relation can ensure system stability. However, virtual environment parameters set according to fixed object characteristics, which fall within unstable ranges, or which change due to external interference may result in system oscillation. Therefore, a fuzzy controller is necessary to compensate for the virtual environment and ensure system stability.

Planning and design of fuzzy-passivity controller

System framework

Fuzzy theory has been developed to solve frequently occurring real-world fuzzy phenomena. Initially proposed as a quantifier tool by Zadeh in 1965, the theory was employed to express fuzzy concepts that cannot be clearly defined and has achieved outstanding results in expressing unique fuzzy phenomena in human speech.^33,34 The interaction model of VR systems with haptic feedback is considered single-port network framework. Figure 6 presents the system framework, the main components of which are the human operator, the haptic interface, standard passivity controller, and the virtual environment. In Figure 6, $v_{h}$ is the operating speed of the human operator, $f_{h}$ is the human operator input, $v_{c}$ is the haptic interface speed, $f_{c}$ is the feedback force of the haptic interface, $v_{e}$ is the speed the feedback input into the virtual environment, $f_{e}$ is the feedback generated when interacting with the virtual environment, and $α$ is the controller.

Figure 6.

Haptic feedback system framework of the fuzzy-passivity controller.

For analysis convenience and for verifying the results, a comprehensive VR training platform based on brain treatment was developed according to the system framework (Figure 7). In the VR platform, offline processing was performed to simplify the functions of the program used to present the skull. The skull was modified into a block consisting of bone characteristics but retained the medical imaging characteristics, including the Hounsfield unit value, which was referenced for collision detection testing. In terms of the human operator, to achieve experiment consistency, objectivity, and analyzability, human input was not incorporated. During the experiment, a robot arm consisting of an artificial intelligence (AI) DC motor controller was adopted to exert unidirectional force and substitute human input. Visual Basic software was employed to complete the human–machine interface design.

Figure 7.

VR experiment platform.

Planning and design of fuzzy-passivity controller

The main factors considered in designing a fuzzy-passivity controller were as follows: (1) By combining passivity system energy control rules and known system energy capacity, the fuzzy controller does not need to infer the mathematical function of the controller. (2) The input membership function in the energy slope of the fuzzy-passivity controller can predict the advantages of the system’s energy trend.

Accordingly, the fuzzy-passivity controller design process was divided into the fuzzification interface, the fuzzy rulebases, the fuzzy interface engine, and the defuzzification interface. In particular, the fuzzification interface was designed to define the fuzzy field through input and output, including membership function size and shape. Therefore, the fuzzy controller input must first be defined according to passivity theorem. Figure 8 presents the system energy curve analysis for a given reference energy capacity of zero.

Figure 8.

System energy curve analysis.

If the total system energy is greater than zero, on the basis of the stability condition, the error was set as the difference between the total system energy curve and the command (0 N·mm):

e (t) = E_{obsv} (t)

(24)

The differential error is

\overset{\cdot}{e} (t) = \frac{de (t)}{dt} = \frac{d E_{obsv} (t)}{dt}

(25)

Figures 9 and 10 present the two aforementioned input definitions, and Figure 11 presents the expert-designed input and output membership functions. The input membership function adopts a triangle design principle, whereas the output membership function employs a single value. This is because the single value output and the triangle design principle-based output do not have significant differences; their application can carry out the extensive computation required for defuzzification.

Figure 9.

The membership functions of error input.

Figure 10.

The membership functions of error variation input.

Figure 11.

The membership function plot of output.

The Fuzzy logic rules presented in Table 1 was designed using passivity control theorem as the foundation and in accordance with the energy curves presented in Figure 8. The initial design of the fuzzyrules are as follows:

Table 1.

Fuzzy logic rules.

		Error variation $\overset{\cdot}{e} (t)$
		NL	NS	Z	PS	PL
Error $e (t)$	NL	PL	PL	PS	PS	Z
	NS	PL	PS	PS	Z	Z
	Z	PS	PS	Z	Z	Z
	PS	PS	Z	Z	Z	Z
	PL	Z	Z	Z	Z	Z

A segment: $R^{1} : IF e < 0 AND \overset{\cdot}{e} > 0 THEN u (t +) = u (t)$

B segment: $R^{2} : IF e > 0 AND \overset{\cdot}{e} > 0 THEN u (t +) = u (t)$

C segment: $R^{3} : IF e > 0 AND \overset{\cdot}{e} < 0 THEN u (t +) = u (t)$

D segment: $R^{4} : IF e < 0 AND \overset{\cdot}{e} < 0 THEN u (t +) = - u (t)$

Where $u (t +) = u (t)$ indicates that the control strategy does not change, and $u (t +) = - u (t)$ indicates that the control strategy must be opposite of the aforementioned strategy.

Simulation and integrated testing

Theory verification is divided into simulation and experiment. For the simulation, the MATLAB/Simulink toolbox was used to construct the entire system framework (Figure 12). The virtual environment system comprises the human operator, the haptic device, the virtual environment, and the controller. In Figure 12, the controller was equipped with the estimator, which is indicated by the blocks on the right.

Figure 12.

Virtual environment Simulink simulation system framework.

MATLAB

Simulink framework

Simulation environment

In the system dynamic function,

\begin{matrix} f_{c} = f_{h} + f_{e} + f_{pc} \\ = m \overset{\cdot\cdot}{x} (t) + b \overset{\cdot}{x} (t) + c sgn (x (t)) \\ f_{e} = k_{e} x_{e} + b_{e} v_{e} \end{matrix}

(26)

where $f_{c}$ is the feedback force of the haptic device, $f_{e}$ is the force generated from the virtual environment, $f_{pc}$ is the compensation force of the fuzzy-passivity controller, $m$ is the equipment weight of the haptic device, $b$ is the damping coefficient, $c$ is Coulomb’s constant, and $x$ is the deposition of the haptic device. Furthermore, $k_{e}$ is the spring coefficient of the virtual environment, $b_{e}$ is the damping coefficient of the virtual environment, $x_{e}$ is the deposition of the haptic device when interacting with virtual objects, and $v_{e}$ is the speed of interaction with the virtual object.

The simulation parameters of Simulink are as follows:

Human operator: Constant = 60 N

Haptic device: m = 2 kg, b = 35 Ns/m, c = 1 N

Virtual environment: Damping coefficient B = −50 Ns/m, spring coefficient K = 30 kN/m

Analysis of simulation results

Comparison of standard passivity controller energy compensation

In Figure 13, the y-axes of the graphs from top to bottom represent haptic device arm position, virtual environment feedback force, energy, and passivity controller feedback force, and the x-axes collectively represent time. A comparison of Figure 13 revealed the fuzzy-passivity controller to exhibit a faster compensation response to unstable systems. The design of the fuzzy rules resulted in an increase in convergent speed, thereby reducing oscillation time.

Figure 13.

Simulation results for compensation of positive and negative energy: (a)standard passivity controllers and (b) fuzzy-passivity controllers.

Compensation of energy limitations

Because of the hardware limitations of the haptic device, the software designed for the passivity controller could not provide unlimited compensation. If the compensation feedback force exceeds the hardware tolerance, the compensation fails, and the system becomes unstable, potentially damaging the device. Therefore, feedback limitations are necessary in controllers designed for virtual environments. As presented in Figure 14, when the compensation force of the passivity controller was limited to 20 N, both controllers achieved stability. However, the position and compensation effect of the fuzzy-passivity controller was superior to that of the standard passivity controller. According to the data samples, at lower compensation limitations, the passivity compensation provided by standard passivity controllers was insufficient to deplete system energy, resulting in oscillation. By comparison, the fuzzy-passivity controller achieved stability in Figure 15.

Figure 14.

Simulation results for compensation limitation at K = 30 kN/m, B = −50 Ns/m, α = 20 N: (a)the standard passivity controller and (b) the fuzzy passivity controller.

Figure 15.

Simulation results for compensation limitation at K = 30 kN/m, B = −50 Ns/m, α = 5 N: (a)the standard passivity controller and (b) the fuzzy passivity controller.

Integrated testing with the virtual brain treatment experiment platform

Experiment platform

The PHANTOM Omni Haptic Device developed by 3D Systems³⁵ was employed as the haptic device and can be manipulated in three axes with six degrees of freedom. The mean maximum continuous force output limit is 3.3 N, and the maximum instant force feedback is 4 N. The three-dimensional (3D) models presented in the virtual environment were created through computerized tomography (CT) images. Image processing technology was used to upload the CT images to the computer as 3D images. To verify the control theory, all other unnecessary interference factors were excluded prior to uploading the CT images. The skull was simplified into a block consisting of bone characteristics and only the required information for the collision detection experiment. Additionally, the block was placed at fixed coordinates in the virtual environment for consistency in experiment verification. The Hounsfield unit (HU) values of the virtual objects and air were referenced for collision detection (Table 2). The HU value of the virtual object was set to 300 in accordance with bone characteristics, and that of air was set to −1024. When the human operator interacted with the virtual object, the system would detect the changes in HU values and record the relevant contact positions to compute the feedback force.

Table 2.

HU value of each type of tissue.

Tissue	HU value
Dense Bone	1000
Red blood cell	93
Hemorrhage	60–110
blood clot	74–81
normal liver	50–80
craniopharyngioma	62
Muscle	44,56~59
Blood	42,56~58
Gray matter	38,32~44
White matter	30,24~36
Renal parenchyma	20~50,45
Heart	24
Plasma	22
Infarct	10–30
Cerebral spinal fluid	0–22
Air	−1000

The brain treatment–based VR haptic feedback system was employed as the experiment platform. To ensure consistency, a robot arm was installed at the human operator end. Comprising an AI DC motor and a human–machine interface for operation, the robot arm was used to repeat the experiment and objectively apply force. In addition to the continuous activation command, a manual command function was added to the robot to test the motor precision of each action and verify the circuit protection mechanism.

One-dimensional cutting actions using the robotic arm

To ensure consistency, objectivity, and analyzability, a robotic arm comprising an AI DC control motor was used at the human operator end to substitute human operation. When designing the graphical human–machine operating interface, we considered the operation safety of the actual environment platform. Accordingly, an internal circuit protection function was installed in the system; if the feedback circuit data exceeded the preset safety value of the experiment platform, the human–machine interface would automatically transmit a stop signal to halt motor action. In this study, the circuit protection value of the AI motor was set to 791 mA. In addition, a continuous action command was designed for the robotic arm to ensure consistency in experiment time and aid in analysis. After pressing the “activate continuous action command” button, the robotic arm would act in accordance with the original settings. If the virtual knife collided with other virtual objects, a continuous cutting action was immediately performed. In addition to the continuous action command, a manual command function was installed to test the motor precision for each action and to verify the circuit protection mechanism.

Experiment results and analysis

Testing was performed using the developed experiment platform. In addition to comparisons of standard compensation, this study investigated the influence of compensation limitations and changes in virtual environment parameters. The virtual environment parameters were as follows: spring coefficient, 710 N/m; damping coefficient, −7 Ns/m.

The controller compensation limitation experiment revealed that maximum instant force feedback tolerance of the haptic device was 4 N. Therefore, the experiment condition involved comparing the compensation response of instant force feedback between 1 and 0.5 N, as depicted in Figure 16. Under a limited compensation response of 1 N, the fuzzy-passivity controller demonstrated a shorter response time. Figure 17 indicate that under a limited compensation response of 0.5 N, the compensation provided by the standard passivity controller was insufficient to deplete the energy generated by the system, resulting in system oscillation. Overall, the compensation generated by the fuzzy-passivity controller achieved convergence stability after a short period of oscillation.

Figure 16.

Experimental results for compensation limitation at K = 710 N/m, B = −7 Ns/m, α = 1 N: (a) the standard passivity controller and (b) the fuzzy passivity controller.

Figure 17.

Experimental results for compensation limitation at K = 710 N/m, B = −7 Ns/m, α = 0.5 N: (a) the standard passivity controller and (b) the fuzzy passivity controller.

To further evaluate the stability performance of the fuzzy controller, the virtual environment parameters were changed 3.5 s into the experiment. To explore the anti-interference capabilities of the two controllers, during the parameter changing experiment, the spring coefficient was changed from 710 to 200 N/m at the 3.5 s mark. The damping coefficient was maintained at −7 N/m, and the feedback limitation was set as −4 N (Figure 18). The results revealed that although the standard fuzzy controller demonstrated a faster initial stability response, when the system sustained interferences that changed the parameters, the fuzzy-passivity controller achieved stability more quickly.

Figure 18.

Experimental results for parameter variations at B = −7 Ns/m, α = −4 N–4 N, then K from 710 to 200 N/m at the 3.5 s mark: (a)the standard passivity controller and (b) the fuzzy passivity controller.

Conclusion and future research

This study combined the passivity-based control theory with the system energy prediction and two-way energy compensation functions of fuzzy theory to design a controller with optimized system stability. A virtual medical training environment was constructed as the experiment platform to verify the designed fuzzy-passivity controller. To further ensure experiment consistency, a robot arm was designed to substitute the human operation of one degree of freedom motions. The results revealed that controllers designed by combining passivity theorem and fuzzy theory possess system energy estimation functions and two-way energy compensation advantages, thereby producing outstanding results in stability response speed and noise-interference stability. However, if the fuzzy-passivity controller is applied to virtual environments under different conditions, the input and output membership functions and fuzzy rulebase must be readjusted according to user experience. Overall, compared with standard passivity controllers, the proposed fuzzy-passivity controller enables systems to achieve stability control more rapidly.

In the future, we will apply the developed fuzzy-passivity controller to 3D interactions and other surgery simulations. Additionally, we will introduce other learning mechanisms such as back-propagation neural networks to the controller. This would enable the controller to automatically adjust the fuzzy rulebase, membership function size, and the membership function shape through learning, thereby providing potential improvements for various applications.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The current study was supported by grants from Ministry of National Defense-Medical Affairs Bureau (MND-MAB-110-080), and Ministry of Science Technology, Taiwan, under award number MOST 109-2625-M-016-001.

ORCID iD

Ping-Nan Chen

References

Kirkman

Ahmed

Albert

, et al. The use of simulation in neurosurgical education and training: a systematic review. J Neurosurg 2014; 121(2): 228–246.

Breimer

Haji

Bodani

, et al. Simulation-based education for endoscopic third ventriculostomy: a comparison between virtual and physical training models. Oper Neurosurg 2017; 13(1): 89–95.

Weghorst

Airola

Oppenheimer

, et al. Validation of the Madigan ESS simulator. Stud Health Technol Inform 1998; 50: 399–405.

Neubauer

Wolfsberger

Forster

, et al. Advanced virtual endoscopic pituitary surgery. IEEE Trans Vis Comput Graph 2005; 11(5): 497–507.

Tolsdorff

Pommert

Höhne

, et al. Virtual reality: a new paranasal sinus surgery simulator. Laryngoscope 2010; 120(2): 420–426.

Won

Hwang

Lim

, et al. Early experience with a patient-specific virtual surgical simulation for rehearsal of endoscopic skull-base surgery. Int Forum Allergy Rhinol 2018; 8(1): 54–63.

Ferraguti

Preda

Manurung

, et al. An energy tank-based interactive control architecture for autonomous and teleoperated robotic surgery. IEEE Trans Robot 2015; 31(5): 1073–1088.

Menaker

Shah

Snelling

, et al. Current applications and future perspectives of robotics in cerebrovascular and endovascular neurosurgery. J Neurointerv Surg 2018; 10(1): 78–82.

Meli

Pacchierotti

Prattichizzo

. Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction. IEEE Trans Biomed Eng 2014; 61(4): 1318–1327.

10.

Agus

Giachetti

Gobbetti

, et al. Real-time haptic and visual simulation of bone dissection. In: Proceedings IEEE virtual reality 2002, Orlando, FL, 24–28 March 2002, pp.209–216. New York: IEEE.

11.

Marshall

Payandeh

Dill

. Study on haptic rendering in a simulated surgical training environment. In: 2006 14th Symposium on haptic interfaces for virtual environment and teleoperator systems, Alexandria, VA, 25–26 March 2006, pp.241–247. New York: IEEE.

12.

Adams

Hannaford

. Stable haptic interaction with virtual environments. IEEE Trans Robot Autom 1999; 15(3): 465–474.

13.

Cogate

Stanley

Brown

. Issues in the haptic display of tool use. In: Proceedings 1995 IEEE/RSJ International conference on intelligent robots and systems. human robot interaction and cooperative robots, Pittsburgh, PA, 5–9 August 1995, pp.140–145. New York: IEEE.

14.

Hannaford

Ryu

. Time-domain passivity control of haptic interfaces. IEEE Trans Robot Autom 2002; 18(1): 1–10.

15.

Stramigioli

Secchi

van der Schaft

, et al. A novel theory for sampled data system passivity. In: IEEE/RSJ international conference on intelligent robots and systems, Lausanne, Switzerland, 30 September–4 October 2002, pp.1936–1941. New York: IEEE.

16.

Ryu

Kwon

Hannaford

. Stable teleoperation with time domain passivity control. In: Proceedings 2002 IEEE international conference on robotics and automation, Washington, DC, 11–15 May 2002, pp.3260–3265. New York: IEEE.

17.

Ryu

Hannaford

Preusche

, et al. Time domain passivity control with reference energy behavior. In: Proceedings 2003 IEEE/RSJ International conference on intelligent robots and systems (IROS 2003), Las Vegas, NV, 27–31 October 2003, pp.2932–2937. New York: IEEE.

18.

Ryu

Kwon

Hannaford

. Stability guaranteed control: time domain passivity approach. IEEE Trans Control Syst Technol 2004; 12(6): 860–868.

19.

Colgate

Schenkel

. Passivity of a class of sampled data systems: application to haptic interfaces. In: Proceedings of 1994 American control conference - ACC ’94, Baltimore, MD, 29 June–1 July 1994, pp.3236–3240. New York: IEEE.

20.

Colgate

Brown

. Factors affecting the Z-width of a haptic display. In: Proceedings of the 1994 IEEE international conference on robotics and automation, San Diego, CA, 8–13 May 1994, pp.3205–3210. New York: IEEE.

21.

Marquez

. Nonlinear control systems: analysis and design. Hoboken, NJ: Wiley-Interscience US, 2003.

22.

Vaseghi

Mobayen

Hashemi

, et al. Fast reaching finite time synchronization approach for chaotic systems with application in medical image encryption. IEEE Access 2021; 9: 25911–25925.

23.

Vaseghi

Hashemi

Mobayen

, et al. Finite time chaos synchronization in time-delay channel and its application to satellite image encryption in OFDM communication systems. IEEE Access 2021; 9: 21332–21344.

24.

Mostafaee

Mobayen

Vaseghi

, et al. Complex dynamical behaviors of a novel exponential hyper-chaotic system and its application in fast synchronization and color image encryption. Sci Prog 2021; 104: 368504211003388.

25.

Chen

Y-T

C-S

Chen

P-N

. Feedback linearization based robust control for linear permanent magnet synchronous motors. Energies 2020; 13: 5242.

26.

Shiao

Pan

. Design of a 3-Stage voltage controller for EMV actuation in SI engines. Int J Eng Technol Innov 2012; 2(4): 273–282.

27.

Venkataraman

. Design and implementation of adaptive PID and adaptive fuzzy controllers for a level process station. Adv Technol Innov 2021; 6(2): 90–105.

28.

Zhang

. Fuzzy passivity control of flexible joint robot. Adv Mater Res 2014; 834–836: 1229–1233.

29.

Zhang

. Fuzzy passivity non-fragile control of flexible joint robot. Appl Mech Mater 2014; 448–453: 3571–3575.

30.

Psillakis

Alexandridis

. A simple nonlinear adaptive-fuzzy passivitybased control of power systems. Nonlinear Dyn Syst Theory 2007; 7(1): 51–67.

31.

Salim

Mounir

. Position control of an induction machine using the fuzzy passivity mode. In: 2019 1st International conference on sustainable renewable energy systems and applications (ICSRESA), 2019, pp.1–5. New York: IEEE.

32.

Gil

Avello

Rubio

, et al. Stability analysis of a 1 DOF haptic interface using the Routh–Hurwitz criterion. IEEE Trans Control Syst Technol 2004; 12(4): 583–588.

33.

Lee

. Fuzzy logic in control systems: fuzzy logic Controller. II. IEEE Trans Syst Man Cybernet US 1990; 20: 419–435.

34.

Mamdani

. Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans Comput 1977; C-26(12): 1182–1191.

35.

3D Systems, Inc. https://www.3dsystems.com/haptics (2021, accessed 31 July 2021).

Stability of virtual reality haptic feedback incorporating fuzzy-passivity control

Abstract

Keywords

Introduction

Stability analysis of VR environments

Passivity theorem

Routh–Hurwitz stability analysis

Stability analysis method comparison

Planning and design of fuzzy-passivity controller

System framework

Planning and design of fuzzy-passivity controller

Simulation and integrated testing

MATLAB

Simulink framework

Simulation environment

Analysis of simulation results

Comparison of standard passivity controller energy compensation

Compensation of energy limitations

Integrated testing with the virtual brain treatment experiment platform

Experiment platform

One-dimensional cutting actions using the robotic arm

Experiment results and analysis

Conclusion and future research

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References