A virtual myoelectric prosthesis training system capable of providing instructions on hand operations

Abstract

This article proposes a virtual hand and a virtual training system for controlling the MyoBock—the most commonly used myoelectric prosthetic hand worldwide. As the virtual hand is controlled using the method also adopted for the MyoBock hand, the proposed system provides upper-limb amputees with operation sensibilities similar to those experienced in MyoBock control. It can also display an additional virtual hand for the provision of instructions on hand operation, such as the recommended posture for object grasping and the trajectory desirable to reach a target. In virtual hand control experiments conducted with an amputee to evaluate the proposed virtual hand’s operability, the subject successfully performed stable opening and closing with high discrimination rates (89.3±6.65%), thanks to the virtual hand’s incorporation of the MyoBock’s operational characteristics. A training experiment using the proposed system was also conducted with eight healthy participants over a period of 5 days. The participants were asked to perform the box and block test using the MyoBock hand in a real environment on the first and final days. The results showed that the number of blocks transported in 1 min significantly increased and that the participants using the instruction virtual hand changed the orientation of the hand approaching blocks from vertical to lateral. The outcomes of the experiment indicate that the proposed system can be used to improve MyoBock hand control operation both quantitatively and qualitatively.

Introduction

A number of studies on myoelectric prosthetic hands ^{1 –7} which are controlled based on electromyogram (EMG) signals, have been conducted to support the daily living of upper-limb amputees. However, to control these prostheses naturally, several months of training under the instruction of doctors and therapists are required. ⁸

In related clinical practice, EMG signal control training to recover muscle function and promote voluntary muscle contraction is first conducted. Training for daily-life tasks is then performed with an actual myoelectric prosthetic hand. ⁸ To support this stage, a variety of computer-based systems and evaluation for control ability have been proposed. ^{9 –13} However, even amputees who achieve voluntary control of EMG signals may struggle to perform tasks as desired because EMG signals are affected by the weight of the prosthesis and arm posture. Task training with the myoelectric prosthesis is therefore necessary before the unit can be prescribed. To this end, users may practice grasping objects of various sizes and shapes, tying knots in string, and other tasks. In this training, development of the ability to perform tasks quickly is not enough; it is also necessary to acquire the two skills of (1) smooth hand opening/closing and (2) natural posture (e.g. the direction of grasping approach and the manner of forearm usage). ⁸ As operation of the most common current myoelectric prostheses is restricted by a fixed wrist joint and the ability to execute pinching motion only, it is particularly difficult to perform tasks with natural posture. Accordingly, long-term training is required until users can perform tasks naturally.

Only a limited number of medical facilities can provide such training in Japan because myoelectric prostheses are very expensive. Against such a background, a number of computer-based systems that provide task training and evaluation have been proposed. ^{14 –19} These allow amputees to operate a virtual prosthetic hand to grasp/release objects and complete the box and block test (BBT) used in clinical practice. However, such systems tend to be quantitative, focusing only on considerations such as the number of successfully completed tasks or the time taken to perform them; attention to qualitative considerations such as EMG signal control ability, direction of grasping approach, and posture during tasks tends to be lacking.

Shibanoki et al. ²⁰ proposed a system for qualitative training in consideration of EMG signal control ability and posture during tasks. The system incorporates a virtual reality (VR) environment for the creation of objects in various positions and allows evaluation of users’ fine control skills such as EMG signal control ability and motor function. However, users may feel that the system’s virtual hand (VH) arrangement does not match the feel of an actual MyoBock hand during task performance because their characteristics differ. Against such a background, the authors developed a virtual training system for the MyoBock ²¹ with which users can practice VH control using the MyoBock method. However, the system cannot be used to provide training while giving feedback on appropriate control in areas such as the direction of grasping approach depending on object shapes. As a result, even if users learn to perform tasks quickly through improved hand opening/closing control skill, prosthesis control with natural posture may still be difficult.

Against such a background, this article proposes a VR-based training system for myoelectric prosthesis control aimed at improving skills including efficient control and prosthesis movement with natural posture. As control of the proposed VH is based on the characteristics of the MyoBock hand, usage of the latter will be intuitive once VH training is complete. Thanks to the display of an additional VH for the provision of instruction on aspects of hand operation (referred to as the instruction VH), such as the recommended posture for object grasping and the desirable trajectory to a target, the proposed system can also be used to provide unprecedented training for prosthesis control with natural posture (e.g. direction of grasping approach and forearm usage). Specifically, users develop natural prosthesis control by imitating the instruction VH’s manner of forearm usage. This article mainly focuses on the direction of block grasping approach and the provision of instruction for the prototype system and also covers validation of the proposed instruction. All experiments in the study were conducted in accordance with the Declaration of Helsinki (1964) with the full understanding and consent of the participants. Specifically, approval was received from the Human Research Ethics Committee of The Hyogo Institute of Assistive Technology for the amputee's participation in this study, and approval was received from the Human Research Ethics Committee of The Hyogo Institute of Assistive Technology. In the rest of the article, “A training system for the MyoBock hand” section outlines the proposed training system, “Determination of MyoBock operating characteristics” section highlights the operational characteristics of the MyoBock unit, and “Experiments” section describes the VH control experiments and the training experiments.

A training system for the MyoBock hand

Concept

Prescription of a myoelectric prosthetic hand requires the user to first learn natural grasping motion in various situations to reduce the load caused by wearing the hand. In related clinical practice, assessment of trainees’ quality of movement is based on observations such as the orientation of hand approach and natural posture. Therapists therefore provide instruction on hand operation to teach users natural control. In the teaching of natural grasping motion, it is essential to evaluate prosthesis operation ability and provide feedback as appropriate. However, the number of therapists capable of performing this work in Japan is limited. Accordingly, a training system capable of performing evaluation and providing instruction during training is necessary. The following three points are essential for the realization of such a system:

Development of a VH featuring the operational characteristics of the myoelectric prosthetic that will eventually be used by the trainee (usually a MyoBock hand);

Instruction on appropriate myoelectric prosthetic hand control during training; and

Evaluation of trainees’ myoelectric prosthesis operation ability and provision of appropriate feedback.

VH control

Figure 1 shows the structures of the VH control system and the proposed VH. Figure 2(a) gives an overview of the VH and its link structure. The hand itself and the virtual objects used were developed using Virtools. ²² The VH has a rigid link structure consisting of 22 joints (numbered 1, 2,…, 22 in the figure) based on the structure of the human hand. First, opening/closing motion is discriminated in consideration of EMG signal amplitude. Motion discrimination is performed using the threshold method adopted for the MyoBock. Control of the VH’s angular velocity is based on EMG signal amplitude and discrimination results. Angular velocity depends on the MyoBock’s dynamic mode control (DMC). The position and posture of the VH are controlled using 3-D position and posture information recorded from the trainee’s forearm. The details of these processes are outlined below. VH opening/closing is based on the method used for the MyoBock. First, EMG signals are recorded from two electrodes (l: channel number; l = 1: extensor digitorum muscle; l = 2: flexor carpi ulnaris muscle) contained in a socket and attached to the trainee’s relevant upper limb. The proposed system can provide training in consideration of prosthesis weight using the socket. Next, the measured signals are digitized using an A/D converter (sampling frequency: f_s (Hz)) and are defined as e_l(n) ( $l\in \left\{1,2\right\}$ , where n is the number of samples). The trainee’s motion $m\in \left\{o,c\right\}$ (o: opening; c: closing) is discriminated by comparing e_l(n) to the predefined threshold e^th (if e₁(n) is greater, the motion is discriminated as opening; if e₂(n) is greater, it is discriminated as closing). The behavior of the MyoBock DMC is simulated using motion discrimination and e_l(n). In the DMC method, the angular velocity of opening/closing depends on EMG signal amplitude. Accordingly, this velocity is described as

$\theta (n)=\theta (n-1)+{\omega }_m(e_l(n))\Delta t$

1

Figure 1.

VH control system. VH: virtual hand.

Figure 2.

Proposed VH and instruction VH. (a) VH structure and (b) instruction VH overview. VH: virtual hand.

where θ(n) represents the joint angles between the thumb and fingers, ${\omega }_m(e_l(n))$ is the angular velocity of opening/closing, and Δt is the sampling time. To realize pinching motion, the initial joint angles θ(n) are set as a conflict position between the thumb and fingers.

The position and posture of the VH are controlled with a 3-D position sensor. This is attached to the trainee’s cubital fossa because the proposed system is generally designed for upper-limb amputees. The recorded positions $[x^{\text{'}},y^{\prime },z^{\prime }]\text{T}\text{ and }[{\alpha }^{\prime },{\beta }^{\prime },{\gamma }^{\prime }]\text{T}$ are used to control the elbow position ${\mathit{P}}_{\text{elbow}}(n)\in {\mathit{R}}^3$ and the posture ${\mathit{\theta }}_{\text{elbow}}(n)\in {\mathit{R}}^3$ of the VH. As the Virtools coordinate system is left-handed, these right-handed system data are converted into left-handed equivalents. The 3-D position thus converted is defined as ${[x,y,z]}^{\text{T}}$ , and the posture is defined as ${[\alpha ,\beta ,\gamma ]}^{\text{T}}$ . To allow determination of object grasping and evaluate the trainee’s manual dexterity, the hand position ${\mathit{P}}_{\text{hand}}(n)$ is defined as follows

${\mathit{P}}_{\text{hand}}(n)={\mathit{P}}_{\text{elbow}}(n)+A\left[\begin{array}{c}\text{cos}\beta \text{sin}\gamma \\ \text{sin}(-\beta )\\ \text{cos}\beta \text{cos}\gamma \end{array}\right]$

2

where A is a forearm length configured in advance.

Control of the instruction VH

The proposed system displays the instruction VH (Figure 2(b)) to provide guidance on how to grasp or release objects. This approach leverages the VR environment’s capacity to support user training under instruction. First, the position of the instruction VH ${[x^{\text{target}},y^{\text{target}},z^{\text{target}}]}^{\text{T}}$ is determined from information on the position of the object $P_{\text{obj}}(n)$ . The elbow joint angle of the instruction VH ${[{\alpha }^{\text{target}},{\beta }^{\text{target}},{\gamma }^{\text{target}}]}^{\text{T}}$ is arbitrarily set in advance, and the elbow position of the instruction VH ${\mathit{P}}_{\text{elbow}}^{\text{target}}(n)$ is calculated as follows

${\mathit{P}}_{\text{elbow}}^{\text{target}}(n)={\mathit{P}}_{\text{obj}}(n)-A\left[\begin{array}{c}\text{cos}{\beta }^{\text{target}}\text{sin}{\gamma }^{\text{target}}\\ \text{sin}(-{\beta }^{\text{target}})\\ \text{cos}{\beta }^{\text{target}}\text{cos}{\gamma }^{\text{target}}\end{array}\right]$

3

The trainee looks to see whether the translucent instruction VH and the VH being controlled are superimposed and receives auditory feedback when the following conditions are satisfied

$\sqrt{{(x^{\text{target}}-x)}^2+{(y^{\text{target}}-y)}^2+{(z^{\text{target}}-z)}^2}\le E_{\text{position}}^{\text{th}}$

4

$\sqrt{{({\alpha }^{\text{target}}-\alpha )}^2+{({\beta }^{\text{target}}-\beta )}^2+{({\gamma }^{\text{target}}-\gamma )}^2}\le E_{\text{angle}}^{\text{th}}$

5

Here, $E_{\text{position}}^{\text{th}}\text{ and }{E}_{\text{angle}}^{\text{th}}$ are predefined error thresholds. Based on this method, the proposed system can be used to provide training with instruction on aspects of appropriate operation, such as adjustment of the elbow joint angle depending on object shapes. It can also be used for quantitative evaluation, as it supports grading of prosthesis control with natural posture based on differences between the movement of the instruction VH and that of the VH.

VR feedback

In addition to visual information, the proposed system also provides the user with auditory and tactile feedback to enhance realism and support the communication of spatial and tactile information. Tactile feedback is provided via vibrotactile stimulators when the VH touches a virtual object, and the user perceives vibration from real environment collision with objects via the socket. In this study, vibration of 250 Hz was used for tactile feedback because humans are the most sensitive to this frequency. ²³ Auditory feedback was also provided to trainees when the virtual object was grasped and deposited as intended.

Determination of MyoBock operating characteristics

The development of a training system for the MyoBock requires determination of its characteristics, but little related information is available. Accordingly, the characteristics of the MyoBock were investigated as described in this chapter.

This article proposes a training system for the MyoBock prosthetic hand, which is commonly used worldwide. To describe its behavior, it is necessary to appropriately determine operational parameters such as the maximum opening/closing angles θ^max, θ^min, the motion threshold e^th, and the angular velocity of opening/closing ${\omega }_m(e_l(n))$ . However, as this information is not publicly disclosed, the operational characteristics of the MyoBock hand were investigated in consideration of its DMC specification. The maximum opening/closing angles θ^max and θ^min were measured using a goniometer (GS-100; OG Wellness Technologies Co., Ltd., Okayama, Japan Corp.). To determine the motion threshold e^th and the angular velocity of opening/closing ${\omega }_m(e_l(n))$ , a constant voltage was applied to the MyoBock using a digital/analog converter (D/A) board (PCI-3338; Interface Corp., Hiroshima, Japan; resolution: 12 bit; settlement time: 10 μs; output range: −5 to 5 V), and its motion was observed. Magnetic sensors (UB-1; Hitachi Ltd., Tokyo, Japan; sampling frequency: 100 Hz) were attached to the thumb and index finger of the MyoBock to record motion based on changes in their output voltage depending on the distance between them. The maximum voltage was observed at the maximum closing angle, and the minimum voltage was observed at the maximum opening angle. The opening/closing time $T_m(e_l(n))$ were defined from the time between observation of the minimum and maximum voltages. The angular velocities of opening/closing ${\omega }_m(e_l(n))$ were defined as follows based on the maximum opening/closing angles θ^max and θ^min and the opening/closing time $T_m(e_l(n))$

${\omega }_m(e_l(n))=\frac{{\theta }^{\text{max}}-{\theta }^{\text{min}}}{T_m(e_l(n))}$

6

The experiment was conducted and repeated three times for an open and close motion of the MyoBock, in which the output voltage was increased from 0.50 to 2.0 V in 0.10 V increments. The error between the relative angle between MyoBock’s first and other fingers ${\theta }^{\text{MyoBock}}(n)$ and the VH’s relative angle ${\theta }^{\text{VH}}(n)$ during the duration of the motion was calculated as follows

${\epsilon }^{\theta }(n)=\left|\frac{{\theta }^{\text{MyoBock}}(n)-{\theta }^{\text{VH}}(n)}{{\theta }^{\text{max}}-{\theta }^{\text{min}}}\right|$

7

Figure 3 shows measurement of the maximum opening/closing angles θ^max and θ^min. The goniometer reading showed a maximum opening angle of 110°, and the maximum closing angle θ^min was defined as 0° because imaginary lines based on the angle between the first joint of the thumb and the second joint of the index finger were parallel (Figure 3). Figure 4(a) and (b) shows the relationship between input voltage and angular velocity of opening/closing ${\omega }_m(e_l(n))$ , with each plot indicating the average for three trials. The opening angular velocity was defined as a positive value, and the closing angular velocity was expressed as a negative value. In both opening and closing, it can be seen that the MyoBock began to move at around 0.50 V. Investigation was also performed with the voltage changed in increments of 0.01 V between 0.50 and 0.60 V to determine the motion threshold e^th more accurately. The result indicated a motion threshold of 0.56 V. After the MyoBock began to move, the angular velocity of opening/closing ${\omega }_m(e_l(n))$ increased quadratically until around 1.5 V and then remained largely constant.

Figure 3.

MyoBock maximum opening and closing angles.

Figure 4.

Relationships between angular velocity and input voltage for the MyoBock. (a) Opening angular velocity and (b) closing angular velocity.

From these results, the motion threshold e^th was defined as 0.56 V and the relative angle between the first joint of the thumb and the second joint of the index finger θ(n) was expressed via the following equations and parameters (Table 1)

$\theta (n)=\theta (n-1)+{\omega }_m(e_l(n))\Delta t$

8

${\omega }_m(e_l(n))=\{\begin{array}{l}0(0\le e_l(n)<e^{\text{th}})\\ a_m{e}_l^2(n)+b_m{e}_l(n)+c_m\\ (e^{\text{th}}\le e_l(n)\le 1.5)\\ {\omega }_m^{\text{max}}(1.5<e_l(n))\end{array}$

9

Table 1.

Experimentally determined MyoBock operational.

	$e^{\text{th}}(\text{V})$	${\omega }_m^{\text{max}}(°/\text{s})$	$a_m(°/{\text{V}}^2\cdot \text{s})$	$b_m(°/\text{V}\cdot \text{s})$	$c_m(°/\text{s)}$	${\theta }^{\text{max}}(°)$	${\theta }^{\text{min}}(°)$
Opening	0.56	144.3	160.0	−185.6	67.3	110
Closing		−127.7	−88.7	55.8	−12.0		0

Table 1 shows the results for each of the parameters. a_m, b_m, and c_m were estimated from the relationship between EMG signal amplitude and angular velocity ${\omega }_m(e_l(n))$ using the least-squares method. The boundary conditions were discontinuous $({\omega }_o(e_{th})=13.5\text{ and }{\omega }_c(e_{th})=-8.57\text{ for }{e}_l(n)=e^{\text{th}},\text{ and }{\omega }_o(1.5)=148.9\text{ and }{\omega }_c(1.5)=-127.9\text{ for }{e}_l(n)=1.5$ . Figure 5 compares VH and MyoBock joint angles determined from simultaneous control of the both units using identical EMG signals recorded from a non-amputee. The horizontal axis represents time, and the vertical axis indicates (from top to bottom) EMG signals, discrimination results, and joint angles (VH and MyoBock). The EMG signals were recorded from a subject’s right forearm (channel 1: extensor digitorum muscle; channel 2: flexor carpi ulnaris muscle). An electronic goniometer (B2921; Biometrics Ltd., Newport, UK; sampling frequency: 1000 Hz) was attached to the MyoBock’s first and second fingers for angle measurement. The average value for ε^θ(n) and the average correlation coefficient during the duration of the motion (the shaded area in Figure 5) were estimated as 0.141 ± 0.069 and 0.969 ± 0.017, respectively. These results show that the proposed VH’s opening/closing angles corresponded to those of the MyoBock, indicating its capacity to replicate MyoBock behavior.

Figure 5.

Comparison of VH and MyoBock joint angles. VH: virtual hand.

Experiments

VH control experiments

An amputee who had used a MyoBock hand for a long time was asked to perform the BBT ²⁴ in a virtual environment to determine whether the proposed VH would provide users with operation sensibilities similar to those experienced in MyoBock control. The BBT is used to evaluate a user’s manual dexterity based on the number of blocks transferred within a certain time. The 50-year-old male subject had undergone upper-limb amputation 14 cm below the elbow and had used a myoelectric prosthesis for 15 years. The experiment environment consisted of boxes, a partition and a ball (radius r = 3 cm) in the center of the rightmost box. The ball was judged to have been grasped when (1) the distance between the VH position P _hand(n) and the center of the object P _obj(n) was less than or equal to p^th, and (2) discrimination resulted in a closed status. The ball was released if discrimination indicated an open status while the VH was holding the ball. Here, p^th is a predetermined constant. A total of 10 one-minute sessions were conducted. The subject was asked in advance to perform as many tasks as possible and was allowed to practice for 3 min. The average rectified EMG signals were measured from two electrodes (13E200 = 60; Otto Bock HealthCare Deutschland GmbH, Duderstadt, Germany) at f_s = 100 Hz (l = 2). The electrodes were attached at the same locations used for the subject’s usual myoelectric prosthesis control (channel 1: extensor; channel 2: flexor). A 3-D position sensor (Istorak II; Polhemus Inc., Colchester, Vermont, USA) was also attached to the subject’s right cubital fossa, and the position, 3-D position, and posture were measured at 60 Hz. The subject practiced opening and closing, and the parameter for ball-grasp judgment was set as $p^{\text{th}}=\sqrt{2}r$ . Evaluation of operability with the proposed method was based on the number of successfully completed tasks and the average motion discrimination rate (i.e. the ratio of the total number of opening and closing motions to the number of motions conducted appropriately while grasping condition 1 was satisfied and the VH reached the left of the box while holding the ball).

Table 2 shows the average discrimination rate for 10 sessions (89.3±6.65% and the average number of successful tasks for 10 sessions (15.6±4.17). The average discrimination rate indicates that the subject successfully performed stable opening and closing control with high discrimination rates (approximately 90%). This is considered attributable to the fact that the operation sensibilities were similar to those of the subject’s control of the MyoBock hand in everyday life, thanks to the similarity between the proposed VH’s motion threshold and angular velocity of opening/closing to those of the MyoBock hand. These results indicate the feasibility of VR environment training without incongruity using the proposed VH with control based on the method and operational characteristics of the MyoBock hand. The average number of successful tasks (15.6±4.17) indicates that the subject was able to perform in the proposed VR environment with a certain level of success. However, the number of successful tasks in the VR environment was lower than the total of 40 that the subject was able to complete in a real environment—a difference attributed to the difficulty of perceiving distance between the hand and ball. The authors plan to introduce a 3-D display solution (such as a head-mounted display) to address this depth perception issue.

Table 2.

VH control experimental results.

Discrimination rate (%)	Number of successful tasks (times)
89.3±6.65	15.6±4.17

VH: virtual hand.

Training experiments

In this article, the BBT ²⁴ is referenced as an example of task training. The proposed VR environment consists of boxes, a partition and a cube with each side of r = 2.5 (cm) in the center of the right-hand box. The previous system proposed by Shibanoki et al. ²⁰ did not allow changes in object posture, but the object in the proposed system can be controlled by rotating its axes α_obj, β_obj, and γ_obj. The system also allows instruction on grasping for each object posture via the display of the instruction VH, which is shown from the time an object appears in the box on the right until the VH grasps it. Further, the proposed system allows instruction on the placement of objects in the left-hand box. The conditions for cube grasping were as per the experiment described in “VH control experiments” section.

Training experiments were performed over a period of five consecutive days using the proposed system. To verify the effectiveness of the instruction VH, the performances of two groups were compared. Group I consisted of four healthy males (A–D; average age: 23.5±1.73) who practiced with the VH independently without instruction (referred to as the no-instruction group). Group II consisted of four healthy males (F–I; average age: 22.5±0.58) who underwent training with the instruction VH (the instruction group). The instruction group was allowed to grasp an object only when the square root of the sum of squares of the differences between the elbow position/angle of the instruction VH and those of the VH were less than $E_{\text{position}}^{\text{th}}/E_{\text{angle}}^{\text{th}}$ , respectively. EMG signals were recorded using two dry-type electrodes (13E200 = 60; Otto Bock HealthCare Deutschland GmbH, Duderstadt, Germany) at f_s = 1000 Hz. The electrodes were attached to each subject’s right forearm (channel 1: extensor digitorum muscle; channel 2: flexor carpi ulnaris muscle). A 3-D position sensor (Istorak II; Polhemus Inc., Colchester, Vermont, USA) was also attached to right cubital fossa of each subject, and the position, 3-D position, and posture were measured at 60 Hz. Two vibrotactile stimulators (VBW32C25; Audiological Engineering Corp., Massachusetts, Somerville, USA) were attached to the outer surface of the socket near the electrode . Two motions (opening and closing) were used in the experiment. The parameters for cube-grasp detection were set as $p^{\text{th}}=\sqrt{10}r\text{ and }{E}_{\text{position}}^{\text{th}}=E_{\text{angle}}^{\text{th}}=15$ based on trial and error, and the elbow joint angle of the instruction VH was set as ${[\alpha ,\beta ,\gamma ]}^{\text{T}}=[10,30,20]$ based on consultation with an occupational therapist experienced in myoelectric prosthesis training.

The 5-day training schedule is shown in Figure 6. First, all subjects underwent training with an EMG-based rehabilitation aid (EBRA) ¹¹ to learn how to generate and maintain EMG signals for the target motions. In this session, each subject underwent three types of EMG signal control training in voluntary muscular contraction, multiple-muscle cooperation, and muscular contraction timing. All subjects conducted virtual BBTs from the second day to the fifth day using the proposed system with a total of 15 one-minute sessions per day. All participants were asked before the training to perform as many tasks as possible. In evaluation of the VH operation method, all participants were allowed to practice for 1 min in advance and took breaks every five sessions. No feedback on evaluation results was given to the subjects during the training. In evaluation to determine the effectiveness of using only the instruction VH, no information on the grasping method (such as the purpose of displaying the instruction VH or the clinically appropriate grasping method) was provided to the subjects before the experiments.

Figure 6.

Training schedule. EBRA: EMG-based rehabilitation aid.

The BBT was conducted in a real environment on the first and fifth days in order to evaluate the effectiveness of the proposed VR training for real prosthesis control. Figure 7 shows how the blocks in the experiment were arranged in columns, and the standard task involved moving blocks from the upper-right corner to the lower left to determine whether the direction of block grasping approach was improved by the proposed instruction VH. The blocks were stacked in two stages, and those on the upper-stage side were wrapped in paper to prevent them from falling down easily (Figure 7). The paper was also rendered less obstructive by cutting it on each side. All subjects were instructed not to drop blocks intentionally. They were asked to pick up the next block if they dropped one and were allowed to pick up dropped blocks only if there were no others to pick up. The upper part of the boxes was at around waist level.

Figure 7.

Block arrangement used in the BBT experiments. BBT: box and block test.

In the virtual BBT, evaluation of subjects’ ability was based on (i) task performance in consideration of the number of tasks successfully completed within a minute and (ii) ability to control posture in consideration of the elbow joint angle recorded at the time when the VH grasped the object in successfully completed tasks. Focus was placed on the rotational angle of the elbow joint α to determine which block surfaces the subjects grasped.

On the first day, EMG signal control training with EBRA was conducted to teach the subjects how to control EMG signals freely. The training was repeated until the average scores for each session were over 80. All subjects achieved the target score within around 30 min to an hour, and the training was ended. Figure 8 shows a scene from the virtual BBT training conducted after the second day, and the subjects’ levels of skill for each session are shown based on the viewpoints of (i) task performance and (ii) ability to control posture.

Figure 8.

Virtual BBT training. BBT: box and block test.

Task performance: Number of successfully completed tasks

Figure 9(a) shows the number of successfully completed tasks, and the details of the statistical tests are shown in Table 3. The figure shows the average value for each group on each training day, and the results for each training session were statistically verified via Holm’s multiple comparison testing. Figure 9(a) shows that the number of successfully completed tasks for both groups increases with the number of training sessions. This suggests that the subjects learned to perform the tasks by training with the proposed system through each training session.

Figure 9.

Experimental results obtained from the second day to the fifth day. (a) Task performance and (b) ability to control posture.

Table 3.

Details of statistical tests for the number of successfully completed tasks from the second day to the fifth day.

Group I				Group II
Holm test	DOF	NSL	p Value	Holm test	DOF	NSL	p Value
2nd–5th	118	0.0083	1.1 × 10−18	2nd–5th	118	0.0083	2.0 × 10−9
3rd–5th	118	0.010	6.9 × 10−12	2nd–3rd	118	0.010	7.5 × 10−5
2nd–4th	118	0.013	2.5 × 10−11	4th–5th	118	0.013	0.0024
2nd–3rd	118	0.017	8.6 × 10−7	2nd–4th	118	0.017	0.0030
4th–5th	118	0.025	0.00011	3rd–5th	118	0.025	0.018
3rd–5th	118	0.050	0.00035	3rd–4th	118	0.050	0.41

DOF: degrees of freedom; NSL: nominal significant level.

Ability to control posture: Difference between target elbow joint angle and subject’s elbow joint angle

Figure 9(b) shows differences between the target elbow joint angle and those of the subjects, and the details of the statistical tests are shown in Table 4. As the instruction group did not complete any tasks on the second day, the table includes a degree-of-freedom indication of 117. Specifically, Figure 9(b) indicates absolute errors between the target value for the rotational angle of the elbow joint α^target and those of the subjects α. It can be seen that the angle in the instruction group was closer to the target than in the no-instruction group. This indicates that the instruction group’s subjects managed to conduct the training with the instructed elbow joint angle. It can therefore be inferred that the proposed system helps not only to improve the number of successfully completed tasks but also the direction of grasping approach with the instruction VH. Hence, it can be used for unprecedented training in myoelectric prosthesis control with natural posture.

Table 4.

Details of the statistical tests for absolute errors between α and α^target from the second day to the fifth day.

Group I				Group II
Holm test	DOF	NSL	p Value	Holm test	DOF	NSL	p Value
2nd–4th	118	0.0083	0.27	2nd–5rd	117	0.0083	0.021
2nd–3rd	118	0.010	0.30	2nd–3th	117	0.010	0.15
2nd–5th	118	0.013	0.54	3rd–4th	118	0.013	0.17
4th–5th	118	0.017	0.65	2nd–4th	117	0.017	0.32
3rd–5th	118	0.025	0.73	3rd–5th	118	0.025	0.34
3rd–5th	118	0.050	0.90	4th–4th	118	0.050	0.66

DOF: degrees of freedom; NSL: nominal significant level.

The results of BBT performed in a real environment are shown in Figure 10 and Table 5. The horizontal axis represents each group, and the vertical axis is the mean value of the number of successfully completed tasks over three sessions on the first and fifth days. Figure 11 shows an example of how grasping posture changed after training in a real environment among instruction group subjects. It depicts the grasping posture observed just before subject F grasped a block. Comparison of the first and fifth days shows that the number of successfully completed tasks increased in both groups (Figure 10). These results indicate that control skill for the actual MyoBock can also be improved by training with the proposed VH featuring MyoBock operating characteristics. Comparison of the number of successfully completed tasks based on Welch’s t-test shows no significant difference between the no-instruction group and the instruction group. However, it was confirmed that the direction of approach in the instruction group changed from vertical to lateral as instructed in the training (Figure 11). These results indicate that the direction of grasping approach can also be changed in a real environment by the training with correction of the grasping posture based on the proposed instruction VH. It can be inferred that the proposed system is suitable for training in natural prosthesis control, although there is a need for further investigation on appropriate provision of more detailed instruction and improvement of the system to enable guidance on a series of forearm motions.

Figure 10.

Numbers of blocks in a real environment.

Table 5.

Details of statistical tests for numbers of blocks in a real environment.

Group I			Group II
DOF	F value	p Value	DOF	F value	p value
22	1.23	0.0022	22	1.01	1.9 × 10−6

DOF: degrees of freedom.

Figure 11.

BBT (subject F). (a) The first day and (b) the fifth day. BBT: box and block test.

From the above results, it was concluded that the proposed system improves not only the number of successfully completed tasks but also the direction of grasping approach and is promising as a training system for natural prosthesis control.

Conclusion

This article proposes a training system for the MyoBock hand in a VR environment. In the study, the operating characteristics of the MyoBock were investigated and a training system that provides the user with operation sensibilities similar to those experienced in MyoBock control was realized using a VH featuring MyoBock characteristics. The proposed system can also display an instruction VH that shows the user how to grasp or release an object, allowing the provision of real-time guidance on appropriate operation during training.

To verify the effectiveness of the proposed instruction VH, a training experiment using the system was conducted with eight non-amputees over a period of 5 days. The results showed that the system improved not only the number of successfully completed tasks but also the direction of grasping approach, thanks to the instruction VH.

In future research, the authors plan to engage in detailed analysis of forearm usage during task performance with a prosthesis and to improve the instruction method for guidance based on musculoskeletal kinematics. Other plans include introduction of the capacity to change the instruction method automatically based on the object position and the performance of experiments with a greater number of amputee subjects to verify the proposed system’s validity.