Towards the application of one-dimensional sonomyography for powered upper-limb prosthetic control using machine learning models

Abstract

Background:

The inherent properties of surface electromyography limit its potential for multi-degrees of freedom control. Our previous studies demonstrated that wrist angle could be predicted by muscle thickness measured from B-mode ultrasound, and hence, it could be an alternative signal for prosthetic control. However, an ultrasound imaging machine is too bulky and expensive.

Objective:

We aim to utilize a portable A-mode ultrasound system to examine the feasibility of using one-dimensional sonomyography (i.e. muscle thickness signals detected by A-mode ultrasound) to predict wrist angle with three different machine learning models – (1) support vector machine (SVM), (2) radial basis function artificial neural network (RBF ANN), and (3) back-propagation artificial neural network (BP ANN).

Study Design:

Feasibility study using nine healthy subjects.

Methods:

Each subject performed wrist extension guided at 15, 22.5, and 30 cycles/minute, respectively. Data obtained from 22.5 cycles/minute trials was used to train the models and the remaining trials were used for cross-validation. Prediction accuracy was quantified by relative root mean square error (RMSE) and correlation coefficients (CC).

Results:

Excellent prediction was noted using SVM (RMSE = 13%, CC = 0.975), which outperformed the other methods.

Conclusion:

It appears that one-dimensional sonomyography could be an alternative signal for prosthetic control.

Clinical relevance

Surface electromyography has inherent limitations that prohibit its full functional use for prosthetic control. Research that explores alternative signals to improve prosthetic control (such as the one-dimensional sonomyography signals evaluated in this study) may revolutionize powered prosthesis design and ultimately benefit amputee patients.

Background

Surface electromyography (SEMG) is an electrical manifestation of the neuromuscular activation associated with contracting muscle. ¹ It has been widely used to control powered upper limb prosthetic devices. Although controlled studies suggest that expert users of EMG driven powered prostheses can perform a wide range of tasks, the high rejection rates ² and the high incidence of overuse injuries in the contralateral arm of unilateral amputees, ³ suggest that full functionality of EMG-based powered prosthesis is often not exploited in everyday life. The use of EMG driven powered prostheses is limited by some of its inherent properties. SEMG is a noisy, non-stationary and non-linear signal, which is susceptible to interference (e.g. socket movement and sweat-related skin impedance changes). It is also difficult to use SEMG to discriminate superficial and deep muscle activities, ⁴ limiting the potential of multi-degrees of freedom (DOF) control. In addition, SEMG signal amplitudes associated with a particular task could vary considerably among individuals, which means that subject-special calibration is needed. ⁵ More importantly, multi-DOF EMG-based prostheses require a very high level of concentration to control. ⁵

Due to the advantages of being stable, easy to use, non-ionizing and able to record activities from deep muscles without cross talks from adjacent superficial muscles, ⁶ ultrasonography, has been widely used to measure muscle morphology in both static and dynamic conditions.^7,8 We previously reported that the real-time change of muscle thickness detected by ultrasound, namely sonomyography (SMG), could reliably monitor the muscle morphological changes ⁹ and therefore has the potential for both prosthetic control^10,11 and assessment of muscle function.^12,13

For a better prosthetic control, mapping the relationship between SEMG and resultant kinetic and kinematic patterns (i.e. force and angle) have been mostly done using artificial neural network (ANN) in animal or human models.^14,15 A three-layer feed-forward neural network model with the back-propagation (BP) algorithm was first utilized to map the relationship between EMG and joint angle. ¹⁶ Thereafter, similar approaches with different improvements has been widely used to study the relationships between SEMG and muscle force, ¹⁴ arm movement ¹⁷ and elbow joint torque. ¹⁸ Other ANN architectures have also been proposed in neurophysiologic and biomechanical research, including radial basis function (RBF) ANN, ¹⁹ BP through time algorithm, ²⁰ Levenberg–Marquardt algorithm, ²¹ and time-delayed ANN. ²² Another machine-learning algorithm, namely, support vector machine (SVM), has also been applied for EMG-related neuromuscular diseases diagnosis^23,24 and pattern recognition.^25,26 SVM implements the structural risk minimization (SRM) principle rather than the empirical risk minimization adopted by most traditional ANN models. In most cases, the performance of SVM modelling was significantly better than ANN approaches.

One previous study has demonstrated that two-dimensional (2D) SMG-based wrist angle prediction could be best done by SVM models. ¹¹ However, the ultrasound imaging system employed is not suitable for control purposes, as the system is too expensive and bulky for practical use. In contrast to the studies using B-mode ultrasound (i.e. 2D SMG), our most recent studies have shown that muscle thickness changes could also be measured in real time using A-mode ultrasound (i.e. 1D SMG). ²⁷ A-mode ultrasound transducers could be easily attached to the skin during dynamic activities of muscles and are of a sufficiently small size to potentially be embedded in prosthetic sockets. Hence, 1D SMG supposes to provide a more portable, compact, inexpensive and practical solution to detect muscle thickness changes than 2D SMG. The objective of this paper is to examine the feasibility of using 1D SMG to predict wrist angle with SVM and compare its prediction accuracy with other traditional ANN models (i.e. BP ANN and RBF ANN).

Methods

Subjects

Nine healthy subjects (seven male and two female) aged between 24 and 35 years were recruited into the study. Human subject ethical approval was obtained from the ethics committee of Hong Kong Polytechnic University and informed consents were obtained from all subjects prior to the experiment. All participants were right-hand dominant without any known neuromuscular disorders.

Experimental protocol

Right forearm of each subject was selected for all tests. Each subject was asked to perform wrist extension guided by a metronome (MT-40, Wittner, Germany) at extension rates of 15, 22.5, 30 cycles/minute, respectively, starting from the neutral position to full extension. For each extension rate, three repeated tests were performed with a 3-minute rest between trials and there were three wrist extension cycles in each trial.

Experiment setup

A ultrasound pulser/receiver (model 5052 UA, GE Panametrics, Inc. West Chester, OH, USA) was used to drive a 10 MHz single element ultrasound transducer (model V129, GE Panametrics, Inc., West Chester, OH, USA), and amplify the received signals. The ultrasound transducer was attached to the skin surface of the extensor carpi radialis muscle (Figure 1). The A-mode ultrasound signals were acquired at a sampling rate of 100 MHz by a high speed A/D converter card (Gage CS82G, Gage Applied Technologies, Inc, Canada). Angle signals were measured by an electrogoniometer (XM110, Penny & Giles Biometrics, Inc., UK) during wrist extension (Figure 1). The wrist angle signals were sampled at 17 Hz by a 12-bit data acquisition card (NI-DAQ 6024E, National Instruments Corporation, Austin, TX, USA) and synchronized with the A-mode ultrasound signals. SEMG signals were also collected, which is discussed elsewhere. ²⁸ The A-mode ultrasound signals were saved frame by frame along with the wrist angle signals for subsequent analysis in a PC with 2.8 GHz Pentium IV microprocessor and 512 MB RAM.

Figure 1.

(A) The ultrasound sensor (1D ultrasound transducer) placed on the extensor carpi radialis muscle of the subject during the wrist extension. An electrogoniometer was placed across the wrist joint. (B) The diameter of the 1D ultrasound transducer is approximately 7 mm.

In this study, we were interested in the A-mode ultrasound echoes reflected from the fat-muscle and muscle-bone interfaces. A cross-correlation algorithm was employed to track the displacements of the two interfaces during the wrist extension (Figure 2). When the muscle was contracting, its dimensional changes induced the variations of distance between these two interfaces, which in turn caused the A-mode ultrasound echoes to shift. The distance between the fat-muscle and muscle-bone interfaces was calculated for each frame. The percentage deformation of the muscle was defined as:

$$D=\frac{(d-d_0)}{d_0}\times 100\%$$

where d ₀ is the initial distance between the two echoes and d is the distance when the muscle was contracting.1D SMG signals were defined as the thickness changes of the extensor forearm muscle detected by A-mode ultrasound signals. More details could be found elsewhere. ²⁸

Figure 2.

Illustration of the A-mode ultrasound signals. Two red windows were used to track the movement of fat-muscle and muscle-bone interface using the cross-correlation algorithm.

Data analysis

The SVM, BP and RBF models of each subject were designed and implemented using Matlab software (Version 6.5, MathWorks, Inc., MA, USA). The models were the same as those used in our previous study. ¹¹ The 1D SMG signals and the wrist angle were employed to construct input–output pairs to train the models. Dimension of the input vector was five, which was formed by the current and past four SMG values. Similar feature vector constitution method had been used in several EMG-based kinematic models.^14,29 Data obtained at an extension rate of 22.5 cycles/minute were selected to train the models, which determine the relations between 1D SMG and wrist angles. Data from the remaining trials with extension rates of 15 cycles/minute and 30 cycles/minute were used for cross-validation tests.

Evaluation of the wrist angle predictions from the 1D SMG signals was made by calculating the relative root mean square error (RMSE) and the correlation coefficients (CC) of the measured and predicted wrist angles. Generally speaking, if relative RMSE (%) <15% and CC>0.9, the prediction is regarded as excellent. ¹⁴

The relative RMSE (%) and CC were obtained as follows:

$$RMSE=\sqrt{\frac{\sum _{n=1}^N\left(X\left(n\right)\right)-Y\left(n\right){)}^2}{N}},$$

The relative RMSE (%) was then calculated using RMSE as a percentage of the corresponding maximum actual peak angle:

$$CC=\frac{{\displaystyle \sum _{n=1}^N\left(X\left(n\right)-\overline{X}\left(n\right)\right)}*\left(Y\left(n\right)-\overline{Y}\left(n\right)\right)}{\sqrt{{\displaystyle \sum _{n=1}^N{\left(X\left(n\right)-\overline{X}\left(n\right)\right)}^2*{\displaystyle \sum _{n=1}^N{\left(Y\left(n\right)-\overline{Y}\left(n\right)\right)}^2}}}},$$

where X(n) is the actual wrist angle recorded during test. Y(n) is the predicted wrist angle. $\overline{X}\left(n\right)$ and $\overline{Y}\left(n\right)$ are the means of X(n) and Y(n), respectively. N is the number of date point of actual wrist signals.

Two-way repeated measure ANOVA with Least Significant Difference (LSD) pair-wise comparisons were used to demonstrate the effects of different movement rates (15 cycles/minute, 30 cycles/minute), models (SVM, BP ANN, RBF ANN) on the CC and RMSE (%). All the data were analyzed using SPSS (Version 15.0, SPSS Inc. Chicago, IL, USA) and statistical significance was set at 5% probability level.

Results

Examples of measured wrist angles and the angles predicted by SVM, BP ANN and RBF ANN for subject C at the extension rates of 15, 30 cycles/minute are displayed in Figure 3 respectively. The averaged RMSE (%) and CC among the nine subjects for different extension rates are shown in Figures 4 and 5.

Figure 3.

Comparison of the predicted and measured wrist angles at the extension rate of (A) 15 cycles/minute (B) 30 cycles/minute by SVM, BP ANN and RBF ANN.

Figure 4.

The mean and standard deviation of RMSE (%) of the nine subjects at extension rates of 15, 30 cycles/minute, by SVM, BP ANN and RBF ANN.

Figure 5.

The mean and standard deviation of CC of the nine subjects at 15, 30 cycles/minute extension rate by SVM, BP ANN, and RBF ANN.

The CC, RMSE and relative RMSE (%) for wrist angle prediction across subjects are shown in Table 1. The CCs were ranged between 0.94 and 0.98, which were larger than 0.9. RMSE (%) ranged between 12.58% and 20.51%. Statistical analysis showed that only model types significantly affected RMSE (%) and CC (p = 0.026 and p = 0.038). Both SVM and RBF ANN achieved significantly higher prediction accuracy in terms of RMSE (%) (p = 0.01 and p = 0.044) and CC as compared with BP ANN (p = 0.018 and p = 0.043), while no significant difference between RBF ANN and SVM was observed (p = 0.546 for RMSE (%) and p = 0.706 for CC).

Table 1.

Mean CC, RMSE, and relative RMSE (%) of wrist angle prediction by SVM, RBF ANN, and BP ANN).

Wrist extension rate (cycles/minute)	SVM			RBF ANN			BP ANN
	CC	RMSE (Degree)	RMSE* (%)	CC	RMSE (Degree)	RMSE (%)	CC	RMSE (Degree)	RMSE (%)
15	0.98	11.08	12.58	0.97	12.64	14.30	0.95	15.56	17.67
30	0.97	11.47	13.37	0.96	12.35	14.42	0.94	17.77	20.51

*

RMSE (%) as a percentage of the corresponding maximum peak angles.

Discussion

The result of the present study is consistent with that reported in our previous paper where – SMG has the great potential to describe the relationship between joint angle and muscle thickness changes.^11,28 Most prediction studies chose EMG signals for prosthetic control purposes. ³⁰ However, EMG relates more to the input of muscle contraction, i.e. the intension of an action, while the muscle architecture is a primary determination of muscle function. ³¹ The architectural changes of skeletal muscle detected using ultrasound were believed to correlate more with the output of muscle contraction.^6,10,31 Moreover, the EMG-angle relationship in the literature was complicated and remained controversial. For example, it was claimed that the EMG activity was the same at different joint angles under maximal contraction of biceps brachii muscle. ³² This is not the same with the EMG-elbow joint angle prediction model29. Therefore, the simple SMG-angle relationship could be more suitable for the kinematic signal predictions. ²⁸

Most of the studies involving modelling the muscular systems were based on ANN models.^14,33 In the current study, SVM has the highest average CC and lowest RMSE (%) compared with ANN methods, which indicated a better performance (Table 1). This is in line with the previous studies that the SVM model outperformed the ANN11,29,34 to predict the joint angle from the SEMG and SMG signals, respectively. The reason may be that SVM had better generalization power compared with BP and RBF ANN for the wrist angle prediction. This study also demonstrated that the models established for the rate of 22.5 cycles/minute could be used for the prediction of wrist angle from SMG data set obtained under other rates. In addition, CC of SVM and RBF ANN were larger than 0.9 and the RMSE (%) were smaller than 15%, thus, their performance could be regarded as excellent. ¹⁴

The result also demonstrated that 1D SMG could be used to predict joint angle using various machine learning models. A-mode ultrasound transducers could be easily attached to the skin during dynamic activities of muscles and can be small enough to be embedded in prosthetic sockets. Compared with the ultrasound imaging system, A-mode ultrasound transducer should be a less expensive (~ $400–$1,200) ³⁵ and are a more practical alternative to detect muscle thickness changes during its contraction.

Conclusion

In the present study, several factors, such as the location of ultrasound sensor, signal resolution, tracking algorithm and the frame rate of the ultrasound, may contribute to the prediction error. To improve the estimation accuracy, ultrasound frame rate should be increased and other algorithms used to track the two interfaces (fat-muscle, muscle-bone) should be explored. In addition, multiple channels of 1D SMG signal could be utilized to increase the DOF of present wrist prediction. In the future study, additional 1D SMG signals will be collected from the wrist flexor muscles to acquire the information about wrist flexion. Similarly, the estimation of wrist adduction and abduction could also be realized using more 1D SMG channels. With multiple sensors, the activities of neighbouring muscles could be differentiated. Thus, multiple channels of 1D SMG signal will be used to monitor the morphological change of different muscles at various locations in the future studies.

Future development should also focus on practical issues that are pertinent to powered prosthetic application. Given that amputees have to carry the system around, the weight, size and power requirements need to be addressed. The prototype system reported in the current study comprises of a miniature ultrasound transducer, supporting software, a personal computer, a high speed data acquisition card, and an ultrasound transmitter/receiver unit. Future efforts should be devoted to integrate the data acquisition and ultrasound modules into a control unit and to minimize its size and weight through a more compact arrangement of the PCB components so that the control unit could be strapped around the waist. In addition, a rechargeable battery should be used to power the control unit. The SVM algorithm should also be integrated into the software which is accessible through PDA or smart phone-based platforms utilizing the fast-developing wireless transmission techniques such as Wi-Fi and Bluetooth. This technology would eliminate the cumbersome wiring between the control unit and the PDA. Similar technology has been successfully utilized by our group in other applications (www.acad.polyu.edu.hk/~rcbus/).