Apply BERT optimization algorithm to improve the accuracy of speech recognition for non-native English learners

Abstract

The voice recognition of non-native tongue English learners is an important challenge in the field of speech recognition. Existing technology still has obvious defects when dealing with non-native tongue pronunciation. This study combined BERT (Bidirectional Encoder Representations from Transformers) and CNN-HMM models and introduced an attention mechanism to improve the accuracy of voice recognition of non-native tongue English learners. It used the pre-trained BERT model to extract the context of the voice signal and used the CNN (Convolutional Neural Network) for local feature extraction, and used the Hidden Markov model (HMM) to make a sequence model building model to capture ability of key features. The experimental results show that the accuracy rate of voice recognition of the BERT-CNN-HMM model in this article reaches 88.9% under normal speed, which is significantly better than 78.5% of the traditional HMM model. Under different noise levels, the accuracy of the article’s model in low noise, medium noise, and high noise environments is 86.9%, 80.5%, and 72.8%, respectively, which are higher than other comparative models. The accuracy of the models in this article remained above 83.8% when processing different accents, showing strong adaptation to strong adaptation and generality. It can be seen from the experimental results that the model of this article can significantly improve the accuracy of the voice recognition of non-native English learners and provide a new direction for further research in the field of voice recognition.

Keywords

BERT optimization algorithm convolutional neural network-hidden Markov model speech recognition non-native learners attention mechanism feature extraction model optimization

Get full access to this article

View all access options for this article.

References

Yao

Zhao

, et al. Clustering driver behavior using dynamic time warping and hidden Markov model. J Intell Transp 2021; 25(3): 249–262.

Pourtaherian

van Onzenoort

, et al. Infant facial expression analysis: towards a real-time video monitoring system using r-cnn and hmm. IEEE J Biomed Health Inform 2020; 25(5): 1429–1440.

Koller

Cihan Camgoz

Ney

, et al. Weakly supervised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos. IEEE Trans Pattern Anal Mach Intell 2019; 42(9): 2306–2320.

Dillon

Wells

. Effects of pronunciation training using automatic speech recognition on pronunciation accuracy of Korean English language learners. Engl Teach 2023; 78(1): 3–23.

Ran

Wang

Qin

. Artificial intelligence speech recognition model for correcting spoken English teaching. J Intell Fuzzy Syst 2021; 40(2): 3513–3524.

Aljohani

Alshangiti

. Investigating the efficacy of using online resources for production training in learning non-native vowel contrasts. Int J Engl Lang Educ 2023; 11(1): 31–52.

Evers

Chen

. Effects of an automatic speech recognition system with peer feedback on pronunciation instruction for adults. Comput Assist Lang Learn 2022; 35(8): 1869–1889.

Ozakin

Peng

, et al. Thanks or tanks: training with tactile cues improves learners’ accuracy of English interdental consonants in an oral reading task. Lang Learn Dev 2023; 19(4): 404–419.

Liu

Yan

Guo

, et al. Forecasting the market with machine learning algorithms: an application of NMC-BERT-LSTM-DQN-X algorithm in quantitative trading. ACM Trans Knowl Discov Data 2022; 16(4): 1–22.

10.

Shobana

Murali

. An improved self attention mechanism based on optimized BERT-BiLSTM model for accurate polarity prediction. Comput J 2023; 66(5): 1279–1294.

11.

Wulff

Mientus

Nowak

, et al. Utilizing a pretrained language model (BERT) to classify preservice physics teachers’ written reflections. Int J Artif Intell Educ 2023; 33(3): 439–466.

12.

Phan

THV

. Developing a BERT based triple classification model using knowledge graph embedding for question answering system. Appl Intell 2022; 52(1): 636–651.

13.

Viji

Revathy

. A hybrid approach of Weighted Fine-Tuned BERT extraction with deep Siamese Bi–LSTM model for semantic text similarity identification. Multimed Tool Appl 2022; 81(5): 6131–6157.

14.

Baskar

Rosenberg

Ramabhadran

, et al. Ask2Mask: guided data selection for masked speech modeling. IEEE J Sel Top Signal Process 2022; 16(6): 1357–1366.

15.

Kadetotad

Yin

Berisha

, et al. An 8.93 TOPS/W LSTM recurrent neural network accelerator featuring hierarchical coarse-grain sparsity for on-device speech recognition. IEEE J Solid State Circ 2020; 55(7): 1877–1887.

16.

Amar

Mu’in

Asmi

. Students’ perceptions on the use of tongue twisters in learning pronunciation. Ling Educ 2019; 1(2): 73–92.

17.

Thu

Win

. A review for reduction of noise by wavelet transform in audio signals. International Research Journal of Engineering and Technology 2019; 6(5): 8128–8131.

18.

Kapoor

Pathak

Rai

, et al. Speech quality enhancement through noise cancellation using an adaptive algorithm. IAENG Int J Comput Sci 2022; 49(3): 653–665.

19.

Wang

Cao

Pei

. A novel underground pipeline surveillance system based on hybrid acoustic features. IEEE Sens J 2020; 21(2): 1040–1050.

20.

Francis

. Adding noise is a confounded nuisance. J Acoust Soc Am 2022; 152(3): 1375–1388.

21.

Shayeb

JNI

Alqadi

Nader

. Analysis of digital voice features extraction methods. Int J Educ Res 2019; 1(4): 49–55.

22.

Siam

Abou Elazm

El-Bahnasawy

, et al. PPG-based human identification using Mel-frequency cepstral coefficients and neural networks. Multimed Tool Appl 2021; 80(17): 26001–26019.

23.

Acheampong

Nunoo-Mensah

Chen

. Transformer models for text-based emotion detection: a review of BERT-based approaches. Artif Intell Rev 2021; 54(8): 5789–5829.

24.

Charoenkwan

Nantasenamat

Hasan

, et al. BERT4Bitter: a bidirectional encoder representations from transformers (BERT)-based model for improving the prediction of bitter peptides. Bioinformatics 2021; 37(17): 2556–2562.

25.

Hossain

Alam Sajib

. Classification of image using convolutional neural network (CNN). Global J Comput Sci Technol 2019; 19(2): 13–14.

26.

Wang

Turko

Omar

, et al. CNN explainer: learning convolutional neural networks with interactive visualization. IEEE Trans Visual Comput Graph 2020; 27(2): 1396–1406.

27.

McClintock

Langrock

Gimenez

, et al. Uncovering ecological state dynamics with hidden Markov models. Ecol Lett 2020; 23(12): 1878–1903.

28.

Polat

. The effects of different weight functions on partial robust M-regression performance: a simulation study. Commun Stat Simulat Comput 2020; 49(4): 1089–1104.

29.

. On theoretical justification of the forward–backward algorithm for the variational learning of Bayesian hidden Markov models. IET Signal Process 2022; 16(6): 674–679.

30.

Meng

Tian

, et al. Joint extraction of entities and relations based on character graph convolutional network and multi-head self-attention mechanism. J Exp Theor Artif Intell 2021; 33(2): 349–362.

31.

DeVries

Locke

Hoda

, et al. Using a national surgical database to predict complications following posterior lumbar surgery and comparing the area under the curve and F1-score for the assessment of prognostic capability. Spine J 2021; 21(7): 1135–1142.