Abstract
After offering an overview of the deaf and hard of hearing (DHH) communities and the availability of subtitles in China, the present article explores questionnaire-based empirical research on the reception of intralingual subtitles by DHH audiences in China. The empirical research (1) has a survey on DHH users’ awareness, habits, needs, and preferences of intralingual subtitles to assess gaps between these needs and current subtitle offerings in China; (2) studies the effects of subtitle presentation rate, character identification, paralanguage information, degree of hearing loss and educational level on comprehension to improve subtitle provision and accessibility for DHH in China; (3) examines the caption speeds with which audiences are most comfortable to establish the most effective subtitle presentation speeds for enhancing viewer comfort and comprehension. It is concluded that although subtitled programs have been available on public TV in China since the 1980s, their provision is still inadequate. Nearly all the available intralingual subtitles are verbatim, which is too difficult for the poorly educated DHH to comprehend. The DHH audiences are quite unsatisfied with the existing subtitle services. Based on our empirical research, we strongly recommend that TV stations, cinemas, and online video providers should provide DHH edited subtitles with character identification and paralanguage information displayed with a lower presentation rate (about 129 Chinese characters per minute).
Keywords
Introduction
Audiovisual products are multi-semiotic constructs that convey information through auditory and visual channels in the form of verbal and non-verbal signs and codes. Although audiovisual products convey meaning through spoken and written language, voice, sounds, music, gestures, film shots, lighting, etc., the linguistic code is usually the most essential to understanding audiovisual material (Tamayo & Chaume, 2017). Traditionally, spoken language in audiovisual products has been conveyed in two ways for DHH audiences: sign language interpreting (SLI) and subtitling for the deaf and hard of hearing (SDH). In China, sign language interpreting in audiovisual products is less common than subtitling. Suppliers of audiovisual products tend to prefer subtitles because they believe subtitles are easier to produce, more widely accessible, and less costly compared to sign language (Li, 2013a). Therefore, subtitling is the main means for DHH to access audiovisual products. After offering an overview of the DHH communities and the availability of subtitles in China, the present article explores questionnaire-based empirical research on the reception of intralingual subtitles by DHH audiences in China. The empirical research (1) has a survey on DHH users’ awareness, needs, habits, and preferences of intralingual subtitles to assess gaps between these needs and current subtitle offerings in China; (2) studies the effects of subtitle presentation rate, character identification, paralanguage information, degree of hearing loss and educational level on comprehension to improve subtitle provision and accessibility for DHH in China; (3) examines the caption speeds with which audiences are most comfortable to establish the most effective subtitle presentation speeds for enhancing viewer comfort and comprehension. The article ends with a discussion of some suggestions for improving intralingual subtitles accessibility for the DHH viewers.
DHH Communities in China
General Information
In China, persons with disability are divided into six categories: visual disability, hearing disability, speech disability, physical disability, intellectual disability, and multiple disabilities. Based on the “Classification and grading criteria of disability” of the National Standard of the People’s Republic of China (GB/T26341-2010, Effective on May 1, 2011), there are four levels of hearing disability (Table 1).
Category and Grading Criteria of Hearing Disability.
According to the data released by the China Disabled Persons’ Federation, the total number of persons with disabilities in China was 85.02 million at the end of 2010, of which 20.54 million were hearing disabled (CDPF, 2012). According to the data released by the China Bureau of Statistics, the total population of China at the end of 2010 was 1.34091 billion. Therefore, the proportion of persons with disabilities is approximately 6.34%, and the proportion of hearing disabilities is approximately 1.53%. Currently, there is no official update on the data of persons with disabilities, but it is safe to conclude that with the natural population growth and the increase of the elderly in China, the number of DHHs is increasing. In 2016, a joint survey conducted by the National Health Commission of the People’s Republic of China and the China Disabled Persons’ Federation revealed that approximately 206 million individuals in China are affected by hearing impairment. This represents 15.84% of the national population (Pan, 2016). As of now, this remains the most authoritative official estimate available.
Educational Level of DHH in China
There is no data on the education level of the DHH in China, but we can get a glimpse of it from the overall education level of the disabled. People with disabilities in China generally have a low level of education. A report (CDPF, 2014) released by the China Disabled Persons’ Federation showed that in 2013, 27.3% of children with disabilities aged 6 to 14 didn’t receive compulsory education; 74.3% of persons with disabilities aged 18 and above had never attended school or only attended elementary school (Table 2).
Distribution of Educational Levels for People With Disabilities Aged 18 and Above in 2013.
The low level of education has always been an important obstacle to the personal development and social integration of the disabled. In order to improve the educational level of the disabled groups, helping them to better obtain knowledge and information through the inclusive design of the information environment has always been the focus of information barrier-free construction in the field of special education in China. “Since 1953 (except in 2000), all information accessibility policies promulgated have involved the content of information accessibility in the education field, including emphasizing the importance of braille and sign language in special schools, attaching importance to the training of sign language and braille teachers, and raising salary for special education teachers” (Li & Xiong, 2019, p. 18). The educational attainment of individuals with disabilities has markedly improved. By 2020, the enrollment rate for compulsory education among school-age children with disabilities has exceeded 95% (CDPF, 2021). Although China has made great progress in the education of the DHH, the education level of the DHH is still low and their ability to read is very poor. In China, hearing-impaired students generally have reading abilities that lag three to four grade levels behind their hearing peers. Additionally, even hearing-impaired students with cochlear implants often exhibit reading proficiency lower than that of their hearing peers (Wang & Wu, 2024, p. 30). A study showed that “most of deaf students in China still had greater difficulties in reading ordinary newspapers after graduating from senior middle school” (He, 2001, p. 46).
Availability of Subtitles in China
Subtitled Programs on TV
The intralingual subtitles are used not only in relation to regional dialects but also to Putonghua in China. Audiovisual products in China do not have subtitles for the deaf and hard of hearing in the strict sense, for almost all the intralingual subtitles are verbatim, which means just taking spoken words into the written mode. The subtitles do not include any description of non-verbal information, for example, no description is given of speaker identification, sound effects, paralinguistic features, etc. “Although the subtitles in TV programs in China are not strictly subtitles for the DHH, the provision of subtitles after all provides an alternative way for the DHH to receive TV sound information” (Li & Xiong, 2019, p. 19). In China’s TV accessibility construction, there is still a long way to go. On the one hand, closed captioning is a common practice for the provision of subtitles for the DHH, while nearly all Chinese TV stations and televisions do not have closed captioning channels, and it will be a huge project to upgrade them. On the other hand, “due to the limitations of personnel, funds, and equipment, and at the same time to ensure safe broadcasting (e.g., accuracy of subtitles), it is difficult to provide subtitles synchronously in the programs with strong timeliness, short production cycle” (NRTA, 2018).
The original TV subtitles in China date back to 1984. In 1984, CCTV’s “News Broadcast” first provided the title subtitles. But in the following years, the development of subtitling was slow. “In July 2001, CCTV decided to speed up the development of adding subtitles to TV programs, then subtitles were gradually being promoted in TV shows and dramas” (Han, 2002, p. 77). “The subtitling project of TV in China started late and the technology used was also different from European and American countries” (Li & Pan, 2012, p. 7). Nearly all subtitles are open subtitles and closed captioning is still in the infant stage of development in China. In August 2013, the State Administration of Radio and Television approved and officially promulgated the Specification of Closed Caption System for Digital Television as the national radio and television industry standard. The Specification points out that closed captioning is to protect the right of the deaf and hard of hearing to watch TV and to watch TV programs in different languages. In August 2014, the program using the closed captioning technology system was first launched on the North American Platform of Hunan Radio and Television International Channel (SD Channel). “In 2017, Hunan Satellite TV HD Channel began to broadcast with closed captioning” (Xiao et al., 2020, p. 102).
Based on the statistics on the subtitled programs broadcasted by the top 10 satellite channels in the first half of 2011, Li’s (2013b, p. 9) study showed that only about 50% of TV programs had subtitles in China as a whole. The programs with the highest proportion of subtitles were TV dramas (93.22%), followed by cultural tourism programs (88.37%) and cartoons (87.56%). The programs with the lowest proportion of subtitles were sports programs (3.42%), followed by news programs (less than 30%), and science and education programs (about 30%). 10 years later, the supply of subtitles has not increased significantly. Bai (2021, p. 87) conducted research on the supply of subtitles of all programs broadcasted on July 13, 2021, in CCTV1, and the results showed that only 64.1% of all programs that day provided subtitles. CCTV1 belongs to a central level TV station, and its proportion of programs with subtitles is slightly higher than that of provincial and county-level TV stations (NRTA, 2018).
At present, domestic TV series are generally equipped with subtitles. “Subtitles have been added to most of the TV programs of central and provincial TV stations. Movies, documentaries, cartoons, regular programs, etc., especially TV programs targeting the elderly, have been fully subtitled” (NRTA, 2018). But for programs such as live news, sports events, and evening parties, due to their strong timeliness, short production cycle, immediacy, randomness, rapid change, and especially the strict requirements for the accuracy of TV subtitles, it is still difficult to provide subtitles to the programs simultaneously. Fortunately, real-time subtitles have begun to develop on Chinese TV programs. The language programs of the 2019 CCTV Spring Festival Gala were equipped with real-time subtitles, which was a breakthrough practice in the accessibility of television. In TV programs without verbatim subtitles, to allow DHH to obtain information and understand the content of the program, the TV stations display subtitles of important information such as column titles, news titles, names of news figures, names of creators and performers, lyrics content, sports event results, etc.
In 2020, the State Administration of Radio and Television released guiding opinions on strengthening the construction of the public service system of radio and television. The Guiding Opinions aim “to promote equalization; to promote the provision of subtitles on all-time satellite TV channels across the country; to realize the simultaneous broadcast of key news programs in sign language” (NRTA, 2020). It is believed that China will continue to promote the provision of subtitles to TV programs, to create a barrier-free viewing environment for the DHH and protect their viewing rights.
Subtitles in the Cinema
Currently, all domestic movies and imported movies in China have Chinese subtitles. Similar to TV subtitles, domestic movie subtitles are also verbatim subtitles, the exact and complete written transcription of speech.
The earliest practice of subtitling in cinema in China appeared in the 1920s. According to The first translated film–A dance unforgettable (1994, p. 62), in 1922, the Peacock Film Company in Shanghai provided Chinese subtitles for the first time for foreign films. By the spring of 1933, almost all foreign films screened in China were equipped with Chinese subtitles. “However, as the dubbing in China matured in the late 1940s, subtitling had been ignored in China for more than 40 years. Starting from the end of the 20th century, subtitling, which had been silent for half a century in China, finally returned” (Deng, 2018, p. 46). Subtitling has regained popularity in China mainly due to technological advancements in multimedia, network, and video editing, the improvement of the language level of the new generation of young audiences (including Chinese and foreign languages), and the low cost of subtitling itself and Fan subgroups.
Subtitles in Audiovisual Products on Internet
The new media of network technology and digital technology have gradually been extended and popularized worldwide, and the use of new media has been gradually changing people’s communication methods and lifestyles. According to a report (CINIC, 2018) released by the China Internet Network Information Center, the number of Internet users in China reached 772 million (51.94% of the population in China) in 2017. The emergence of new media enables the DHH to break through the barriers of time, space, and language in social interactions, have the opportunity to obtain more information, and greatly enhance the freedom of social interaction. H. J. Yu’s (2018, pp. 26–27) investigation on the use of new media by DHH middle school students and college students in Henan Province showed that new media had become an indispensable part of the daily life of the DHH. DHH students used new media frequently, about 4 to 6 times a day; and the time they spent on new media was long, between 3 and 5 hr a day. The most common behaviors of DHH students using new media were watching videos and communicating with friends.
Although digital technology and new media platforms provide convenient technical support for the DHH, there are few audiovisual products specifically for the DHH on the Internet in China. “The supply of barrier-free audiovisual products in new media platforms has not been significantly improved compared with the traditional media era. The problems such as lack of subtitles for the DHH, few sign language programs, and almost no audiovisual description services on the internet still exist.” (Li & Xiong, 2017, p. 205).
Empirical Research on Reception of Intralingual Subtitles by the DHH in China
We used the online Tencent Questionnaire platform for data collection. The questionnaire consists of two sections. The first section involved 11 multiple-choice questions to collect the data on users’ awareness, habits, and preferences of intralingual subtitles. In the second section, we used six video clips to (1) test the effect of presentation rate, character identification and paralanguage information, degree of hearing loss, and educational level on comprehension and to (2) examine the presentation rate with which audiences were most comfortable.
Literature Review
Reception research on SDH started in the early 1970s and has greatly developed worldwide, especially in the US and European countries over the past five decades.
The research on subtitling in China mainly focuses on interlingual subtitling. “Research on SDH has been neglected, and there remains a notable paucity of focus in this area” (W. Lu & Li, 2022, p. 93). In recent years, there has been a rise in intralinguistic SDH studies; however, the number of such studies remains relatively limited. The research primarily focuses on media accessibility. Qiu and Zhang (2022) observed that the hearing impaired in China are gradually “moving away from” or “avoiding TV,” a trend attributed to the “invisibility” of television content for them. This issue manifests in the lack of subtitles, which affects content accessibility, and the failure of available content to meet the specific needs of the hearing-impaired. Yang et al. (2022) conducted an empirical study on the reception of existing subtitling services among the hearing impaired and found that, in China, the primary challenges to subtitle accessibility are difficulties in comprehending subtitle language and a lack of paralinguistic information. Bai (2022) examined barrier-free information dissemination in China from the perspective of media ecology and found that the DHH individuals remain a neglected group. She identified ongoing issues in media dissemination, such as incomplete subtitle information and low intelligibility of sign language programs. Some studies focus on the socio-cultural functions of intralingual subtitles. S. W. Lu and Lu (2022) explored the socio-cultural significance of intralingual subtitles used in Web-only shows in China from a postmodern perspective. They found that, compared to those used in traditional TV, subtitles in Web-only shows tend to be more diversified and creative in styles and forms. These subtitles challenge traditional media power structures and reflect a postmodern approach to consumption.
SDH is being studied across various fields worldwide. Linguistic aspects such as subtitle reading comprehension, text editing, and paralinguistic information are widely discussed.
The DHH, whether their native language is oral or sign language, encounter difficulties in understanding the written language in text and the subtitles in audiovisual products. There were many studies into the reading comprehension skill of the DHH in various countries and languages—Chinese (Liu, 2010; Zhang et al., 2020), English (Holt, 1994; Kyle & Harris, 2006), Dutch (Wauters et al., 2006), Persian (Rezaei et al., 2016), Japanese (Takahashi et al., 2017), Polish (Aleksandrowicz, 2019), and more. All of such studies showed that the DHH had a considerably lower reading comprehension skill than their hearing peers. For instance, Kyle and Harris (2006) reported that deaf children typically experienced a delay of at least 5 years in their reading skills when leaving school. Liu (2010) analyzed the deaf students’ Chinese language papers for the entrance examination at Nanjing Special Education College from 2007 to 2009 and found that DHH students generally had problems such as an inability to fully understand vocabulary connotations, unfamiliarity with common sentence patterns, poor reading comprehension, etc., and their average scores had not yet reached the level of the fifth or sixth grades of primary school. Based on language assessments and analyses conducted in seven special education schools in Yunnan Province, a study by Zhang et al. (2020) found that hearing-impaired students exhibit average language proficiency in elementary school which is approximately 4 to 5 grade levels below that of their hearing peers. In junior middle school, their proficiency typically lags 5 to 6 grade levels behind. Furthermore, no significant improvement in proficiency was observed during senior middle school. Parault and Williams (2010) tested 24 deaf American adults and reported that the reading skills of the participants were below the sixth-grade level.
The linguistic considerations such as text reduction and simplification were widely discussed in SDH. Due to the low reading ability of DHH, text editing and simplification are quite necessary to allow the audience to understand what characters say on screen. Many studies have shown that text reduction and simplification had a positive effect on comprehension (Baker, 1985; Cambra et al., 2009; Gottlieb, 1994; Kovačič, 1994; Lorenzo & Pereira, 2011; Neves, 2009; Tamayo & Chaume, 2017; Ward et al., 2007; Zárate, 2010). Neves (2009) recommended that it is quite necessary to keep the syntax as simple as possible to improve comprehension, while Zárate (2010) stated that lexical factors should be taken into account because DHH children had a more limited vocabulary as compared to hearing children. Baker’s (1985) experiment showed that the reduction in subtitle language complexity combined with the reduction in subtitle presentation rate (60 wpm) resulted in improved comprehension for British school children. Cambra et al. (2009) found that Spanish deaf children had difficulty accessing information in subtitles due to their poor reading skills and too fast subtitle presentation rates.
Meantime, in intralingual subtitling for the DHH, the issue of whether to edit subtitles remains an ongoing debate. On the one hand, the DHH requires verbatim captioning because they want full and equal access to the information presented in the auditory channel (Robson, 2004, p. 20). On the other hand, given that “hearing status and literacy tend to covary” (Burnham et al., 2008, p. 392), the DHH have difficulty in reading verbatim subtitles (Cambra et al., 2009, p. 425; Neves, 2008, p. 136). At the same time, some researchers have found that unreduced text can enhance comprehension. Zárate (2021, p. 40) argued that “the majority of d/Deaf and hard of hearing people have some access to sound, and therefore an accurate transcription of dialogue may be easier to follow than an edited one.”Bianchi et al. (2020, p. 26) proposed that “two extremes should be avoided when producing subtitles: verbatim subtitles of rapid dialogues, as they may affect usability; and adapted subtitles of easy dialogues, as they may affect understanding.”
In recent years, the eye-tracking method has been used to explore the effect of subtitling speed. In Miquel Iriarte’s (2017) study, tracking and analyzing participants’ eye movements showed that while subtitling speed influenced the distribution of attention between the subtitles and the image, it did not significantly affect reactions and repercussions. Szarkowska et al. examined the influence of text editing (edited vs. verbatim subtitles) and subtitle presentation rates on comprehension and reading patterns for both interlingual and intralingual subtitles. The results of the eye-tracking study showed “no benefit of editing down the text of subtitles, particularly in the case of intralingual subtitling and deaf viewers” (Szarkowska et al., 2016, p. 183).
Besides subtitle reading comprehension and text editing, Zárate emphasized that “it must be remembered that SDH subtitlers and captioners have to deal with audiovisual, multimodal texts which contain not only dialogue lines but also paralinguistic information, music, and sound effects” (Zárate, 2021, p. 39). Scholars have been committed to the research on the influence of paralinguistic information such as character recognition, sound effects, mood/tone, and music on comprehension (Vy & Fels, 2010; Harkins et al., 1996; King et al., 1994; Tamayo, 2015; Zárate, 2014; Zarate & Eliahoo, 2014). Their research has shown that paralinguistic information plays a positive role in comprehension. Harkins et al. (1996) concluded that viewers generally prefer to have more non-speech information included in the captions. AlBkowr and Haider (2023) concluded that the inclusion of paralinguistic information in the subtitles enhanced the DHH audience’s understanding of the movie. King et al. (1994) found that comprehension was higher when captions were color-coded for speaker identification than when captions were black-and-white. Vy and Fels (2010) found that using the speaker’s name for speaker identification was useful for DHH viewers while Amin et al. (2022) found that DHH viewers’ preferences for speaker-identification methods might vary based on the number of speakers present on screen.
Research Purposes
This empirical study aims to provide recommendations for enhancing subtitle accessibility for DHH in China through the following three interconnected purposes:
1) Survey DHH users’ preferences: To identify DHH users’ needs and preferences regarding intralingual subtitles and to assess gaps between these needs and current subtitle offerings in China.
2) Analyze factors affecting comprehension: To examine how factors such as subtitle presentation rate, degree of hearing loss, educational level, character identification, and paralanguage information impact subtitle comprehension, with the goal of improving subtitle provision and accessibility for DHH in China.
3) Determine optimal caption speeds: To establish the most effective subtitle presentation speeds for enhancing viewer comfort and comprehension.
Rationale and Hypotheses
As mentioned in the literature, numerous studies in the US and European countries have shown subtitle presentation rate, text reduction, character identification, paralanguage information, hearing loss status, and educational level are important parameters that can significantly affect the comprehension of subtitled audiovisual products. Nearly all intralingual subtitles in China are verbatim subtitles without character identification and paralanguage information, which may make it quite difficult for the poorly educated DHH individuals to understand the subtitles. With these questions in mind, we formulated the following hypotheses:
1) Text editing and comprehension: Edited subtitles will enhance subtitle comprehension more effectively than verbatim subtitles, particularly for DHH individuals with limited educational backgrounds.
2) Character identification and paralanguage information: Subtitles that include character identification and paralanguage information, when displayed at a moderated presentation rate, will improve comprehension more significantly than those without such features.
3) Impact of hearing loss: The degree of hearing loss will moderate the effect of subtitle features on comprehension, with greater hearing loss leading to reduced comprehension of both verbatim and edited subtitles.
4) Educational level: Higher educational levels will be associated with better comprehension of subtitles, but the impact of subtitle editing on comprehension will be more pronounced for those with lower educational levels.
Participants
A group of 62 volunteers from 17 provinces in China took part in the experiment, out of whom 19 were deaf (severe and extremely severe hearing loss), and 43 were hard of hearing (mild and moderate hearing loss). Among them, 25 were female and 37 male. Participants were recruited through social media WeChat and Tencent QQ with the help of special schools and deaf associations. The participants were informed about the confidentiality and anonymity of their responses. Ethical approval for the study was given by first author’s institution.
We recruited participants from different age groups and with different educational background, from school students to senior citizens, with a view to ensure a heterogeneous and ecologically valid sample of target viewers (Tables 3 and 4).
Age Distribution of Participants.
Educational Level Distribution of Participants.
Questionnaire and Materials
Tools: Data collection was conducted using the online Tencent Questionnaire platform. Data analysis was performed with the Tencent QQ questionnaire analysis tool and Microsoft Excel. To investigate the correlation between hearing loss and comprehension scores, we used MATLAB’s univariate linear regression code to calculate score fitting for participants with varying degrees of hearing loss. Similarly, we assessed the correlation between educational levels and comprehension scores using MATLAB’s univariate linear regression code. To determine the optimal caption speed, we employed the Lagrange interpolation formula with MATLAB R2020b.
Questionnaire and Materials
The questionnaire contains two sections. The first section involved 11 multiple-choice questions to survey users’ awareness, habits, preferences, and expectations of subtitles.
In the second section, we used six video clips to (1) assess how presentation rate, character identification, paralanguage information, hearing loss, and educational level affect comprehension; and (2) identify the most comfortable presentation rate for audiences.
Each clip lasts about 1 to 2 min. The clips were subtitled at different speeds: 88, 103, 125, 160, 238, and 264 Chinese characters per minute (cpm). The reading speed for Chinese texts of average complexity ranges from 150 to 180 characters per minute (H. K. Yu, 2015). Considering that deafness may be associated with lower reading skills (Trezek & Mayer, 2019) and that excessively slow subtitles may impair film comprehension (Szarkowska et al., 2021), we hypothesized that for DHH viewers, subtitle speeds below 100 cpm could be too slow, while speeds between 100 and 200 cpm would be optimal, and speeds above 200 cpm might be too fast. The selected subtitle speeds thus spanned from 88 to 264 cpm to encompass this hypothesis.
To account for the differences in the degree of hearing loss of the participants, the experimental videos in this study had no audio to test whether hearing loss negatively impacts the comprehension of subtitles.
The video clips represented two genres: three feature films/TV series and three news programs, which are among the most popular genres in Chinese daily life. The news clips were sourced from CCTV1 programs, and the feature film/TV series clips were from movies and TV series broadcast by CCTV8. Each clip was carefully discussed and selected by our research team to ensure that the difficulty of understanding was not very different. Each clip was a self-contained scene and its understanding did not depend on familiarity with previous sequences of the clip.
Two clips had verbatim subtitles without character identification and paralanguage information (same as subtitles available on TV in China), two clips had verbatim subtitles with character identification and paralanguage information, and two clips featured edited subtitles with reduced reading speeds and included character identification and paralanguage information. The reductions and omissions targeted both elements of spoken discourse, such as false starts, repetitions, hesitations, reformulations, and vocatives, as well as attributive adjectives, intensifiers, expletives, adverbials, and other modifiers with minimal propositional content.
Procedure
Participants were individually tested online through the Tencent Questionnaire. They signed a consent form online to take part in the study and then completed the questionnaire, which took about 20 min.
First, the participants were required to answer 11 multiple-choice questions about their awareness, habits, and preferences of SDH. Next, they were instructed to watch six videos carefully, as they would have to answer questions related to the videos. Each video was viewed only once. After viewing each video, participants answered two multiple-choice questions testing their comprehension of the clip and one question requiring them to give their opinion about the speed (fast, OK, slow). The questions were carefully prepared to test the comprehension of information that was only available in subtitles and was impossible to infer from the image. In total, participants answered 18 questions concerning audiovisual materials.
Results and Discussion
Based on the survey and experimental results, there is a significant gap between the subtitle provision for DHH in China and their actual needs. Existing subtitles present considerable challenges for DHH viewers. The following analysis examines three key areas—users’ awareness, habits, and preferences; subtitle comprehension; and comfortable caption speed—and provides recommendations to enhance subtitle accessibility for DHH in China.
Users’ Awareness, Habits and Preferences of Intralingual Subtitles
We designed the questionnaire to collect data on users’ awareness, habits, and preferences of intralingual subtitles.
The first four questions were about participants’ awareness and general knowledge of SDH. When asked whether they knew SDH, 58.1% of the participants said they did not know; when asked whether they knew how subtitles were produced, 87.1% of the participants said they did not know. When asked whether they thought it was technically feasible to produce closed captions for the DHH, 4.8% thought it was not feasible, and 41.9% said they did not know. When asked whether they thought that TV, movies, online videos, and etc. should provide closed captions for the DHH, 80.6% said that SDH should be provided. The results showed that the DHH in China didn’t know much about SDH. The society should raise the awareness of caring for the DHHs, and provide SDH services for the DHHs with the existing mature digital technology. The DHHs should also voice their needs and demands to protect their rights to obtain audiovisual products.
The next seven questions were about users’ habits, preferences, and expectations of intralingual subtitles. The results showed that:
1) The most commonly used video media among the participants were mobile phones and TV. Specifically, 91.94% of participants reported using mobile phones, 64.52% use TV, 46.77% use computers, and 24.19% use theaters.
2) When asked about their preferred video genres (with multiple choices allowed), the top three favorites among DHH participants were TV series (64.52%), movies (62.90%), and news (62.90%), which showed that news programs occupied a very important place in the life of DHH. However, TV news programs in China rarely have subtitles. TV stations should apply digital technology to produce pre-recorded and especially real-time news subtitles to meet their needs. The following preferences were: Internet short videos (53.23%), military programs (41.94%), entertainment programs (40.32%), sports (37.10%), life programs (35.48%), interviews (22.58%), and financial programs (16.13%).
3) Participants reported spending relatively long periods watching videos each day. Specifically, 41.94% spent more than 3 hr, 17.74% spent 2–3 hr, 17.74% spent 1–2 hr, and 22.58% spent 0–1 hr. The possible reason may be that the DHH rely more on videos for information and entertainment. In 2018, the National Bureau of Statistics organized the second national time use survey, which showed that the average time Chinese people spend watching TV is 1 hr and 40 min per day (NBS, 2019).
4) When asked about the purposes of using subtitles, most participants (75.81%) reported using them “to help understand the video content,” and 53.23% used them “to learn written Chinese further.” Only 3.23% said they “rarely use subtitles.”
5) When asked about their satisfaction with existing video subtitles, omly 6.45% of participants said they were satisfied, while 56.45% thought that subtitles in general were fair. Additionally, 37.10% were dissatisfied, and 0% did not know.
6) The top 3 difficulties encountered by the participants when using subtitles were “too fast” (35.48%), “unable to obtain paralanguage information” (29.03%), and “difficult to identify the speaker” (22.58%). Other issues included “the characters are too small” (12.90%), “I don’t know some Chinese characters” (19.35%), and “the sentence is too difficult” (9.68%). Only 1.61% reported “no difficulty.”
7) When asked about their preferences for verbatim or edited subtitles, 66.13% of the participants preferred edited subtitles.
Based on this survey results, we strongly recommend that TV stations, cinemas, and online video providers should provide DHH edited subtitles with character identification and paralanguage information displayed with a lower presentation rate.
Comprehension
In this study, we aimed to test whether the subtitle presentation rate, the degree of text editing, the degree of hearing loss and educational level, character identification, and paralanguage information affect comprehension. By examining the participants’ comprehension of different types of clips, we sought to provide empirical evidence that edited subtitles with character identification and paralanguage information help DHH understand subtitles more effectively.
To investigate how different subtitle types and presentation rates influence comprehension for DHH individuals, we conducted the following comparisons:
1) Verbatim subtitles with and without character ID and paralanguage: We assess whether subtitles with character ID and paralanguage enhances comprehension when presentation rates are comparable (238 cpm vs. 264 cpm and 160 cpm vs. 125 cpm).
2) Edited subtitles with ID and paralanguage versus verbatim subtitles: We evaluate whether edited subtitles with character ID and paralanguage, presented at reduced speeds (103 and 88 cpm), improve comprehension more effectively than verbatim subtitles displayed at various speeds (238, 160, 264, and 125 cpm).
To test the differences in comprehension, we conducted an analysis with groups (hearing loss; educational level), presentation rate (88–264 cpm), and type of subtitles (with and without character identification and paralanguage information) as independent factors. The dependent variable was the scores for two comprehension questions after each video (50 points per question, for a total of 100 points). Tables 5 to 7 present the results of this analysis.
Average Comprehension Scores for Each Clip by Participants.
Comprehension Scores for Each Clip by Participants With Different Hearing Impairment Levels (Full Score: 100).
Comprehension Scores for Each Clip by Participants With Different Educational Level (Full Score: 100).
As shown in Table 5, participants did not score high on comprehension (average = 55.38). Consistent with our initial hypothesis, comprehension is higher (average = 65.32) in the case of edited subtitles with character identification and paralanguage information displayed with a lower presentation rate (88 cpm). Verbatim subtitles without character identification and paralanguage information displayed with a high presentation rate (238 cpm) are the most difficult ones for participants to comprehend (average = 41.94). Subtitles with character identification and paralanguage information at a high speed (264 cpm) do help participants understand the content of audiovisual products (average = 58.87).
The data showed a correlation between different presentation rates and comprehension scores. Although the caption speed and score are not inversely proportional exactly (Figure 1), it showed the benefits of a lower presentation rate on comprehension scores. In each group, participants scored slightly higher in the clips with lower presentation rate (Table 5): verbatim subtitle without ID and paralanguage group, average scores 41.94 (238 cpm) versus 42.74 (160 cpm); verbatim subtitle with ID and paralanguage group, average scores 58.87 (264 cpm) versus 63.71 (125 cpm); edited subtitle with ID and paralanguage group, average scores 59.68 (103 cpm) versus 65.32 (88 cpm).

Curve chart of participants’ average scores for clips at different speeds.
The data also showed a correlation between the degree of hearing loss and comprehension scores (Table 6). The trend was that participants with less degree of hearing loss scored higher (Figure 2), which is in line with our hypothesis that the degree of hearing loss negatively affects subtitling comprehension. The relatively small number of participants in some categories, particularly the extremely severe hearing loss group, may affect the representativeness of the means. The data showed the participants with extremely severe hearing loss scored the highest (average 60.42), which may be related to the low number of participants (only four) and higher levels of education (two Junior middle school, two Bachelor degree), and the two participants with Bachelor degree scored high.

Curve chart of average comprehension scores by participants with different levels of hearing impairment.
Educational level has a positive effect on the understanding of subtitles, as shown in Table 7. The overall trend is shown in Figure 3. However, the data also shows that participants with primary school education scored higher, which may be because that the number of participants with elementary educational level is quite small (only two participants).

Curve chart of average comprehension scores by participants with different educational level).
We also calculated the score fitting of participants with different degrees of hearing loss using MATLAB’s univariate linear regression equation code. Due to the small number of participants with extremely severe hearing loss (four participants), which deviated greatly from the average sample size of other types, it was excluded from the calculation. The calculation process is shown in Figure 4. The results, shown in Figure 5, reveal that the degree of hearing loss is negatively correlated with comprehension scores. The overall data can be fitted as a negatively correlated linear regression curve, in which the correlation coefficient |R| = .8935. This indicates that about 89% of the fluctuation of y can be explained by the fluctuation of x, and the R-value is close to 1, indicating that the regression equation is significant and the model fitting ability is good. Consequently, we conclude that the degree of hearing loss is strongly correlated with comprehension scores, and the general trend is that participants with less degree of hearing loss scores higher.

Calculation processes (degrees of hearing loss).

Result of linear regression curve fitting (degrees of hearing loss).
We calculated the score fitting of participants with different educational levels using MATLAB’s univariate linear regression equation code. Due to the small number of participants with primary education level (two participants), which deviated greatly from the average sample size of other types, it was temporarily excluded from the calculation. The calculation process is shown in Figure 6. The results, shown in Figure 7, indicate that there is a positive correlation between educational attainment and comprehension scores. The overall data can be fitted as a positively correlated linear regression curve, in which the correlation coefficient R = .89824. This suggests that approximately 90% of the fluctuation of y can be explained by the fluctuation of x, and the R-value is close to 1, indicating that the regression equation is significant and the model fitting ability is good. Therefore, we conclude that there is a strong correlation between educational level and comprehension scores, and the general trend is that the participants with higher educational level scored higher.

Calculation processes (degrees of educational levels).

Result of linear regression curve fitting (degrees of educational levels).
Comfortable Caption Speed
The six videos were subtitled at different speeds and participants were asked to give their opinion about the speed (fast, OK, slow). The caption speeds tested were 88, 103, 125, 160, 238, and 264 cpm. To prevent sudden extreme changes that might skew participant assessments. The order of these speeds was randomly varied ensuring that extreme speeds did not follow one another ( e.g., an 88-cpm segment was never followed by a 264-cpm segment).
In the score coding used, “2” indicates the caption speed is “OK,” as defined as “caption speed is comfortable to me.”“3” indicates the caption speed is “fast,” as defined as “captions should be slower.”“1” indicates the caption speed is “slow,” as defined as “Captions should be faster.” Table 8 shows that a mean score of “2” corresponds to a caption speed of between 88 and 160 cpm.
Scores at Each Caption Speed (N = 62).
Based on the mean and standard deviation data, calculations were performed using the Lagrangian interpolation formula and the Matlab R2020b application tool (see Figure 8 for the calculation process). The data indicates that the mean caption speed that “is comfortable to me” is about 129 cpm, much slower than that of subtitles available in China (e.g., the speech speed of CCTV1’s “News Broadcast” program is about 300 cpm).

Calculation process.
Limitations of This Study and Implications for Future Research
A primary limitation of this study is the small sample size. Participants were recruited through social media platforms, special schools, and deaf associations, leading to a sample that may not fully represent the diversity of the DHH population in China. Specifically, the sample had a higher education level than the general DHH population, with 96.77% having education at or above junior middle school. Additionally, the distribution of participants across 17 provinces was uneven, which may affect the generalizability of the findings.
Another limitation is the complexity introduced by multiple independent factors, including hearing loss, educational level, presentation rate, subtitle types, and text editing degree. The interactions among these factors make it challenging to pinpoint the primary influences on subtitle comprehension and may obscure the relative importance of each factor. Future research could benefit from focusing on fewer variables or employing more refined methods better to understand each factor’s contribution to subtitle comprehension.
Additionally, the study used 1 to 2 min video clips, which may not accurately reflect the experience of viewing full-length films or television programs. Future research should aim to recruit a more diverse and representative sample and use longer video segments to better assess subtitle comprehension for extended content. Expanding recruitment channels and collaborating with a wider range of institutions could help achieve a more representative sample.
Conclusion
This study highlights progress and ongoing challenges in providing subtitles for the DHH audience in China. Despite some advancements, current subtitling practices remain inadequate, primarily featuring verbatim subtitles without character identification or paralanguage information. These limitations hinder comprehension, particularly for those with lower educational levels.
The study confirmed our hypotheses: Edited subtitles improve comprehension more than verbatim ones, particularly for those with lower educational backgrounds. Subtitles with character identification and paralanguage enhance comprehension more than those without these features. Greater hearing loss reduces comprehension of both verbatim and edited subtitles. Higher education levels correlate with better subtitle comprehension, with more pronounced benefits from editing for those with lower education.
To enhance accessibility to audiovisual products for the DHH, it is essential to use edited subtitles with character identification and paralanguage information. Policy recommendations include clearer regulations, increased funding for barrier-free content, and government support to ensure accessible formats and investment in technology. Further research on subtitle readability and accessibility, with a multidisciplinary approach, should also be encouraged.
Footnotes
Acknowledgements
We would like to thank everyone who participated in this study.
Author Note
It should be noted that the SDH services in Taiwan, Hong Kong and Macau are different, and the SDH services described in this article do not include that in the above regions.
Ethical Considerations
The Ethics Review Committee of Lanzhou Jiaotong University approved our interviews (approval: 2022001) on March 15, 2022. Respondents gave written consent for review and signature before starting interviews.
Consent to Participate
The participants were informed about the confidentiality and anonymity of their responses. Participants were individually tested online through the Tencent Questionnaire. They signed a consent form online to take part in the study and then completed the questionnaire.
Author Contributions
Hui Yang determines the topic of the article, writes and revises the article, and controls the direction of the article. Weiju Shen, Xinyan Ma, Bo Liu, Yi Wang collects data, participates in research design and writes the article, and revises the article. All authors contributed to the article and approved the submitted version.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Gansu Provincial Science and Technology Plan (Key Research and Development Program—Social Development Field), Project No. 24YFFA024.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author.
