Cut-off value of the eHEALS score as a measure of eHealth skills among rural residents in Gansu,China

Abstract

Background

In this study, our aim was to quantify eHealth literacy and determine a cut-off value for eHEALS scores that signifies adequate eHealth literacy among rural residents in Gansu, China.

Methods

This cross-sectional investigation encompassed 18 rural areas across three cities—Lanzhou, Wuwei, and Dingxi—in Gansu. A total of 451 residents were recruited and underwent evaluations for both eHealth literacy and eHealth skills, utilizing the eHEALS and self-constructed tasks, respectively. A receiving operator characteristic curve was plotted with eHealth skills as the dependent variable and eHealth literacy levels as the independent variable, aiming to determine a cut-off value for eHEALS indicating adequate eHealth literacy and evaluate its predictive capacity.

Results

Among the 451 respondents, 10.9% did not possess a personal electronic device with Internet access, while 6.4% owned but had never used them. Within the remaining 373 residents, the mean eHealth literacy score was 25.85 (SD:10.93), item scores ranged from 3.12 (SD:1.45) to 3.42 (SD:1.60). Completion rates for three eHealth skills varied from 39.1% to 59.8%. The cut-off value was 29.5 determined by the Youden index. The area under the receiver operating curve was 0.829, with a sensitivity of 86.7%, and a specificity of 66.8%.

Conclusions

Our findings emphasized that eHealth literacy among rural residents remains at a low level. Moreover, we identified a cut-off value of 29.5 for eHEALS scores that signifies adequate eHealth literacy within this demographic.

Keywords

eHealth literacy cut-off value rural residents eHealth skills China

Introduction

The World Health Organization (WHO) emphasized that health literacy plays a critical role in determining health outcomes and has the potential to empower individuals, ensuring health equity.¹ eHealth services have the capacity to alleviate the strain on healthcare systems by offering prompt, high-quality healthcare information.² Furthermore, they enhance healthcare quality by fostering collaborative decision-making and empowering citizens, patients, and healthcare professionals.^3,4 In recent decades, the proliferation of the Internet had increased the utilization of eHealth services in disease prevention and health promotion. Individuals were increasingly adopting appropriate self-care practices and managing their health through eHealth.^5,6 Individuals across different regions can readily access healthcare information and utilize eHealth services, contributing to a more equitable distribution of healthcare resources. Amidst the COVID-19 pandemic, when stringent control measures were enforced and healthcare resources were strained, eHealth emerged as a solution for delivering remote individual health services,⁷ facilitated the implementation of outbreak-related protective measures,^8,9 consequently mitigating the risk of infection and transmission. eHealth gained widespread interest, catalyzing a broader public recognition of its significance.^10,11 Overall, the advocacy and adoption of eHealth yield substantial benefits. Nevertheless, in certain contexts, the dissemination of disease-related misinformation, individual misinterpretation, and subsequent adverse behaviors could lead to unfavorable health outcomes.^12–14 The concept of eHealth literacy was created to assess the skills of individuals using eHealth, which is defined as the ability to seek, find, understand, and appraise health information from electronic sources and using the knowledge gained to address or solve a health problem.¹⁵ Proficiency in adeptly employing eHealth services––reflecting a high level of eHealth literacy––is a pivotal factor in both the execution of eHealth services and the enhancement of healthcare.

Since its introduction, the concept of eHealth literacy has attracted increasing attention worldwide. Research has spanned various domains, including surveys on eHealth literacy levels,^16,17 evaluation of influencing factors,^18,19 analysis of the correlations between eHealth literacy and health behaviors, mental health, lifestyle, and quality of life,^20–22 as well as the development and evaluation of interventions to enhance eHealth literacy,^23,24 etc. However, studies have been restricted in terms of the population, primarily concentrating on adolescents, university students, and urban populations, with less focus on rural residents. This is concerning because people in rural areas tend to have poorer health and lower health literacy due to geographical isolation, socioeconomic disadvantages, inadequate healthcare infrastructure, subpar sanitation, and limited access to medical services, etc.^25–27 There is a significant urban–rural disparity exists in the distribution of healthcare resources and individual health literacy.²⁸ Presently, 500 million Chinese people (36.11%) still reside in rural regions.²⁹ As of June 2021, the 48th Statistical Report on China's Internet Development indicated that over 99 percent of administrative villages had access to optical fiber and 4G connections, attaining a state of “same network and same speed” across rural and urban areas. With the improvement of communications infrastructure, the cost of internet use in rural areas has gradually declined, and the urban–rural Internet penetration rate has narrowed to 19.1 percent (78.3% vs. 59.2%), which provides a guarantee for the use of eHealth by rural residents.³⁰ However, not everyone possesses substantial internet access or eHealth skills to utilize the benefits of eHealth, the eHealth services require a certain level of eHealth literacy of the user to be equitably adopted.³¹ Factors such as advanced age, lower income, and limited education contribute to a lower level of eHealth literacy among China's rural inhabitants compared to their urban counterparts.³² A review suggested that eHealth literacy frequently receives inadequate attention in interventions aimed at socially vulnerable groups, thereby exacerbating healthcare disparities.³¹ Amidst China's COVID-19 prevention and control efforts, the urban–rural gap could potentially increase. As a valuable means for rural residents to access healthcare services and enhance their well-being, eHealth literacy warrants more attention.

Norman et al. developed the eHealth literacy scale (eHEALS) to assess individual's eHealth literacy.³³ The scale has been translated into various languages, and its validity has been demonstrated across diverse populations.^34–39 In China, Ma et al. evaluated the psychometric properties of the eHEALS within a rural population, using the classical test theory and item response theory methods. Their findings endorsed the applicability of eHEALS for measuring eHealth literacy among rural inhabitants.⁴⁰ As a subjective measure of perceived confidence and skills in finding, assessing, and using electronic health information to solve health problems,³³ it is worth considering how perceived skills (as measured by eHEALS) can be used to assess respondents’ objective eHealth skills. Quantitatively, a Dutch study found that eHealth literacy is not always a good predictor of actual eHealth skills.³⁶ In rural areas, the majority who cannot migrate to urban areas is less educated, in poorer health, and less likely to use the internet to access eHealth services.^41,42 There might be a greater disparity between eHealth literacy and actual eHealth skills. Qualitatively, a study in Taiwan formulated a scale for digital health literacy assessment (DHLA) grounded in eHEALS and found the different cut-off values for DHLA could distinguish those at low, medium, and high risk of misjudging health information on the internet.⁴³ These findings suggest that a qualitative cut-off value of eHEALS might be more suitable for assessing actual eHealth skills. Furthermore, adopting a cut-off value could yield additional benefits and greater practical significance. With a cut-off value, we can efficiently distinguish between high and low eHealth literacy levels among residents in a specific area, thus determining their possession of eHealth skills and identifying the key target demographic for intervention. Moreover, the cut-off value can be directly employed to appraise intervention efficacy or serve as a target for interventions. By assessing the discrepancy between current and desired levels, specific interventions can be tailored in terms of type, intensity, and scope to amplify intervention effectiveness. To the best of our knowledge, there is no relevant study to explore an appropriate cut-value for eHEALS. Some studies have employed values such as 20, 26 (without explicit criteria)^,44,45 or 32 (80% of the total score) to distinguish between high and low levels of eHealth literacy,^18,46,47 which still needs to be validated and examined in future research. Therefore, it's necessary to determine a cut-off value for eHEALS to assess the acquisition of eHealth skills.

For obtaining a cut-off value, we need to adopt an appropriate methodology for evaluating participants’ eHealth skills. In a Dutch study, eHealth skills of 88 urban and rural residents were assessed through nine assignments encompassing four facets of eHealth skills—operational, formal, informational, and strategic. These tasks spanned 17 scenarios and took around 1.5 h to complete. The study then evaluated the correlation between eHealth literacy and eHealth skills. The tasks were designed to be accessible and relevant to the general user population (e.g. find the Web address of a health clinic, or search for information on vitamins).³⁶ A UK study evaluated practical health skills by surveying 54 participants (comprising university students and staff) on their completion levels for six health-related questions including analgesic medication, obesity, diabetes, nutrition, and influenza retrieval tasks, and assessed the relationship with eHealth literacy (e.g. you have a headache and buy a packet of Co-Codamol tablets from the chemist, which are the two largest ingredients in Co-Codamol tablets, or diabetic retinopathy is an eye complication associated with diabetes, can you name three common symptoms of this condition).⁴⁸A study in Taiwan measured the risk of misunderstanding health information by constructing an online health information database containing 310 items across categories like food safety, drugs, medical devices, cosmetics, medical treatment, preventative health, disease screening, health promotion, and general medicine. Then selected five items at random from this database and had online participants determine the correctness of these items (e.g. it is helpful to drink bone broth, which can supplement calcium, or smoking e-cigarettes is not addictive).⁴³ These studies employed objective methods to evaluate eHealth skills. Nevertheless, findings from study designs with smaller sample sizes might not be representative of real-world scenarios and may not be applicable to large-scale investigations. Additionally, specific health issues may not align with the realities of rural China, rendering the surveys less practically significant. Lastly, taking longer to complete surveys could lead to diminished participant engagement and completion rates. Given these circumstances, it might be impractical to replicate similar studies in rural China. Consequently, we need to develop assessments that are adapted to specific health topics and simplify the assessment approach as much as possible, in order to assess the eHealth skills of rural residents in China.

Therefore, the objectives of this study were (a) to investigate the eHealth literacy levels among rural Chinese residents (b) the first to determine a cut-off value for adequate eHealth literacy by assessing the eHealth skills in real-world settings.

Methods

Study design

This cross-sectional study was conducted in Gansu, a major agricultural province situated in northwest China. Gansu encompasses 12 prefecture-level cities (Lanzhou, Jiayuguan, Jinchang, Baiyin, Tianshui, Wuwei, Zhangye, Pingliang, Jiuquan, Qingyang, Dingxi, Longnan) and two autonomous prefectures (Linxia Hui Autonomous Prefecture and Gannan Tibetan Autonomous Prefecture). The study employed a multistage stratified sampling approach to recruit participants. In the first phase, based on the GDP levels from the Gansu Development Yearbook 2020, 14 prefecture-level cities were divided into three classes—high (Jiayuguan, Lanzhou, Jinchang, Jiuquan, Zhangye), medium (Qingyang, Gannan, Baiyin, Wuwei, Pingliang) and low (Tianshui, Longnan, Linxia, Dingxi). One city was then randomly selected at each class, three cities, Lanzhou, Wuwei, and Dingxi were selected. In the subsequent phase, two districts were randomly picked from each of these three cities. Following this, 18 villages and towns were conveniently selected, considering the degree of on-site cooperation. This process led to the recruitment of participants from rural areas across six districts in Gansu Province: Yuzhong and Gaolan from Lanzhou, Gulang and Tianzhu from Wuwei, and Anding and Tongwei from Dingxi. Specific townships including Dinyuan, Lianta, Sanjiaocheng, Xiaguanying, Chengguan, Qingshui, Shenchuan, Shidong, Dingning, Sishui, Dachaigou, Huazangsi, Saishensi, Songshan, Lijiapu, Chankou, Beichengpu, and Biyu.

Participants and procedure

The recruitment process was conducted through two methods: (a) household surveys led by village doctors and (b) Face-to-face surveys conducted among clinic attendees at village health centers. During the survey, enumerators worked in pairs to address any encountered problems, ensuring the quality and efficiency of the survey. Participants were provided with a comprehensive briefing regarding the survey's objectives, content, anonymity, and voluntary nature before granting their consent to participate. Those who agreed to participate in the study were requested to complete a questionnaire on their own or with the assistance of an enumerator, the entire process typically required approximately 30 min. Upon finishing the questionnaire, the participants would receive a gift (worth approximately RMB 25) as a reward for their participation, as well as compensation for their time. During the 1-month survey period (29 March–28 April 2021), 451 subjects consented to participate in survey and completed the questionnaire. After checking the questionnaire's completeness and reasonableness, the subjects were all included in the study. The inclusion criteria were as follows: age ≥18 years, local permanent residents, and have the ability to complete the questionnaire on one's own or under the assistance of an enumerator.

Measures

Demographics

The following data regarding general sociodemographic characteristics were collected: age, gender, ethnicity, place of domicile, education level, marital status, occupation, living with children, and personal monthly income.

eHEALS

The eHEALS, as a classic eHealth literacy measurement tool, has been widely used across various cultures and populations. Its simplicity, relatively ease of comprehension, and quick completion time make it suitable for rural residents. Furthermore, its measurement validity has been confirmed in rural China.⁴⁰ Consequently, eHEALS was used to measure eHealth literacy.³³ The assessment encompassed tests to evaluate the ability to access internet health information and usability of services (1–5), judgment ability (6–7), and decision-making ability (8). Sample items included: “I know how to find helpful health resources on the Internet,” “I feel confident in using information from the Internet to make health decisions.” Each item was rated on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The total score was calculated by summing all 8 items, ranging from 8 to 40, with higher scores representing a higher level of eHealth literacy. The Chinese⁴⁹ version has been widely used to evaluate eHealth literacy levels in various populations. The Cronbach's α was 0.962.

Ehealth skills

Three tasks were constructed to assess participants’ practical eHealth skills (Cronbach's α = 0.831). For the tasks, various types of diseases were considered, including acute and chronic infectious diseases, as well as acute non-infectious diseases. Additionally, factors including general disease information, access to specific health services, and emergency disease response scenarios were incorporated. The participants conducted searches using their electronic devices, and subsequently, investigators judged task completion. A participant who completed all three tasks was considered to have good eHealth skills. The three self-constructed tasks are described below.

Firstly, regarding the choice of acute infectious diseases, the study was conducted during a period when China was implementing dynamic zero-control measure for COVID-19, it was beneficial to understand the dynamic development of the epidemic in terms of the degree of implementation of one's own protective measures. Additionally, COVID-19 was a hot topic within Chinese society during that period, information regarding the disease was rapidly spreading across various channels. Therefore, we wanted to investigate whether rural residents were able to access correct and effective information amidst the diversified available sources:

Task 1: Search for official notification data on yesterday's (the day before the respondent was surveyed) nationwide COVID-19 epidemic situation.

The judgment criteria were as follows: the respondents were considered to have completed the task if they used one of the following sources for their search, official government web pages (e.g. the National Health Commission or Chinese Centre for Disease Control and Prevention, epidemic bulletin by Dr Ding Xiang (a professional healthy lifestyle coach in China's pan-health field)), the official websites of Tencent, Sina, or Baidu (Internet news platforms with a wide audience in China for real-time updates on the latest developments in COVID-19), or official WeChat public account. If the participants retrieved data consistent with the above channels, the task was also considered complete, otherwise, it was considered incomplete.

Secondly, regarding the choice of chronic infectious diseases, we initially considered HIV (human immunodeficiency virus) and TB (tuberculosis). However, the focus of HIV epidemic in China was not on rural areas, we ultimately included TB as part of task construct. Despite the great progress in TB control, China had approximately 780,000 new cases of TB, with the third highest (after India and Indonesia) worldwide in 2021.⁵⁰ And according to a national epidemiological sample report conducted in 2010, the incidence of active TB cases in rural areas (569 per 100,000) was significantly higher than that in urban areas (307 per 100,000).⁵¹ Considering the distinctive pattern of TB treatment in China, specific medical institutions are officially set up to diagnose and treat TB. Therefore, we focused our questions on search for specific TB medical institution sites:

Task 2: Search for designated TB clinics within your area.

This task was judged based on the following criteria. Participants conducted searches on official websites of the Health Commission of Gansu Province or Gansu Provincial Centre for Disease Control and Prevention using the keyword “tuberculosis sentinel.” If the participants found a list of clinics through this search method or other means, the task was considered complete, otherwise, the task was considered incomplete.

Thirdly, as China experiences ongoing socioeconomic development and an accelerated aging process, the incidence of cardiovascular disease cases within the nation continues to rise. According to the China Cardiovascular Health and Disease Report 2021, cardiovascular disease stood as the primary contributor to total deaths among urban and rural residents in 2019, with two out of every five deaths were attributed to cardiovascular disease. Particularly in rural areas, this proportion reached 44.26%.⁵² Confronted with this substantial disease burden, our objective was to assess whether rural residents could acquire the right information and undertake effective countermeasures when confronted with sudden cardiovascular diseases:

Task 3: If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it.

The criteria for evaluating this task were as follows. Given that there is no single standardized answer for this task. Participants who derived solutions from healthcare professionals, encompassing actions such as promptly discontinuing ongoing activity, administering oral rapid-acting heart relievers or nitrates (if accessible), seeking urgent medical attention, or getting content similar to the answer, were considered to have accomplished the task. If no search was conducted or the search results showed advertisements without effective information extraction, the task was considered incomplete.

Following the survey, the investigators would demonstrate the correct search process to the participants or provide suggestions for improvement based on their search processes, with the hope that they could implement correct and effective measures in similar situations that may occur in the future.

Statistical analysis

The receiver operating characteristic (ROC) curve is a two-dimensional curve with sensitivity as the vertical axis and specificity as the horizontal axis. Each point on the curve reflects the corresponding sensitivity and specificity corresponding to the observed sample. The area under the curve (AUC) refers to the size of the area under the ROC curve, which is a standard used to evaluate the quality of a classification model and reflects the models’ accuracy. Typically, AUC values ranged from 0.5 to 1.0, with a larger AUC representing better model performance.

A ROC curve was plotted, with eHealth skills as the dependent variable and eHealth literacy level as the independent variable. The AUC was employed to assess the predictive power of eHealth literacy on actual eHealth skills. The Youden index was used to determine the optimal cut-off value yielding the greatest combined sensitivity and specificity for eHEALS that indicates sufficient literacy (larger Youden index represented better discriminatory results).

To evaluate the validity of the results, the cut-off value was used to divide the respondents into two groups, those with and without eHealth skills. Descriptive statistics were used to demonstrate the proportion of correct responses for each task, with means and standard deviations of the eHEALS scores and the number of tasks completed in two groups. Chi-square tests were used detect the relationship between eHealth skills task completion and groups. Analysis of variance tests were used to compare the differences in eHEALS scores and number of tasks completed across two groups. Discriminant analysis on the two groups was used to explore the probability of correct classification based on the eHEALS scores.

All the analyses were conducted using IBM SPSS (Version 25).

Results

Descriptive statistics

Slightly over half of the participants were aged below 50 years (53.7%) and female (54.8%). The majority of the participants were Han Chinese (91.4%), domiciled in the local area (99.1%), married (92.7%), and possessed a mobile phone with Internet access (89.1%). More than half had studied only until middle school or lower (64.8%) and living with their children (68.3%). Approximately half of the participants engaged in agriculture, forestry, animal husbandry, fishery, and water production tasks (55.7%) and had a monthly income less than CNY 2000 (56.5%). Detailed demographic characteristics are presented in Table 1.

Table 1.

General sociodemographic characteristics.

Sociodemographic characteristics	n	Percentage (%)
Age
≤50 years	242	53.7
>50 years	209	46.3
Sex
Male	204	45.2
Female	247	54.8
Ethnicity
Han	412	91.4
Others	39	8.6
Domicile location
Local	447	99.1
Field	4	0.9
Education level
Primary and lower	163	36.2
Middle school	129	28.6
High school/secondary	80	17.7
Specialist	46	10.2
Undergraduate	33	7.3
Marital status
Married	418	92.7
Others	33	7.3
Occupation
Agricultural, forestry, animal husbandry, fisheries, and water production tasks	251	55.7
Other	200	44.3
Living with children
Yes	308	68.3
No	143	31.7
Personal monthly income (CNY)
≤2000	255	56.5
2001–4000	121	26.8
4001–6000	49	10.9
>6000	26	5.8
Own a mobile phone with internet access
Yes	402	89.1
No	49	10.9

Ehealth literacy

Among the 451 survey respondents, 49 did not possess a personal electronic device with Internet access and 29 owned but had never used it. Consequently, 373 participants were included in the analysis. The average eHealth literacy score of the participants was 25.85 (SD:10.93). Item scores ranged from 3.12 (SD:1.45) to 3.42 (SD:1.60). The highest scoring item was “I know how to find helpful health resources on the Internet,” and the lowest scoring item was “I feel confident in using information from the Internet to make health decisions.” Specific scores are shown in Table 2.

Table 2.

Ehealth literacy score.

Item	Mean	SD
I know how to find helpful health resources on the Internet	3.42	1.60
I know how to use the Internet to answer my health questions	3.29	1.60
I know what health resources are available on the Internet	3.14	1.55
I know where to find helpful health resources on the Internet	3.20	1.56
I know how to use the health information I find on the Internet to help me	3.24	1.56
I have the skills I need to evaluate the health resources I find on the Internet	3.21	1.51
I can tell high-quality from low-quality health resources on the Internet	3.22	1.47
I feel confident in using information from the Internet to make health decisions	3.12	1.45
Total	25.85	10.93

Ehealth skills

The completion rates for the three tasks varied from 39.1% to 59.8%. The highest was “Search the official data of yesterday's national COVID-19 outbreak” (59.8%), while the lowest was “Please search online for the correct treatment for someone who is experiencing severe heartburn” (39.1%). The distribution of respondents got 0, 1, 2, and 3 items correct were 138 (37.0%), 46 (12.3%), 69 (18.5%), and 120 (32.2%), respectively. Details are provided in Table 3.

Table 3.

Completion profile of eHealth skills.

Item	n	%
Search the official notification data of yesterday's national COVID-19 outbreak	223	59.8
Search for a designated tuberculosis provider in your area	175	46.9
If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it	146	39.1
The number of respondents got 0 task correct	138	37.0
The number of respondents got 1 task correct	46	12.3
The number of respondents got 2 tasks correct	69	18.5
The number of respondents got all 3 tasks correct	120	32.2

The correlation between eHealth literacy and eHealth skills

The eHealth literacy and eHealth skills were significantly correlated, with correlation coefficients ranged from 0.518 to 0.694. There were also significant correlations between eHealth skills, the correlation coefficients ranged from 0.568 to 0.683. Specific correlations are shown in Table 4.

Table 4.

The correlation between eHealth literacy and eHealth skills.

Item	eHealth literacy	eHealth skills task 1	eHealth skills task 2	eHealth skills task 3	eHealth skills total
eHealth literacy	1	-	-	-	-
eHealth skills task 1	0.694*	1	-	-	-
eHealth skills task 2	0.607*	0.683*	1	-	-
eHealth skills task 3	0.556*	0.568*	0.611*	1	-
eHealth skills total	0.518*	0.565*	0.733*	0.859*	1

The numbers represent the correlation coefficients of the row and column elements in which they are located, *p < 0.001, -: repetition correlation coefficient.

Cut-off points

The AUC for the three eHealth skills predicted by the eHealth literacy ranged from 0.823 to 0.886, with corresponding cut-off values ranged from 22.5 to 28.5. Taken together, the AUC predicted by eHEALS scores was 0.829, while the cut-off value was determined as 29.5 by the Youden index. The sensitivity was 86.7%, and the specificity was 66.8%. Details are shown in Table 5. The receiver ROC curve is presented in Figure 1.

Figure 1.

Receiver operator characteristic curves for eHealth literacy levels to predict eHealth skills. ((a) Task 1, (b) Task 2, (c) Task 3, (d) Total; the red line on the ROC curve corresponds to the diagonal line, which has an AUC of 0.5, indicating the performance of a random classifier that would achieve accuracy equivalent to random guessing. The blue line represents the performance of the classifier in our study, whose ROC curve lies above the diagonal line, indicating better performance than random guessing).

Table 5.

Cut-off values for eHealth literacy levels to predict eHealth skills.

eHealth Skills	AUC	95% CI	Cut-off value	Youden index	Sensitivity (%)	Specificity (%)
Item 1	0.886	0.850–0.923	22.5	0.628	92.8	70.0
Item 2	0.846	0.807–0.885	28.5	0.549	81.1	73.7
Item 3	0.823	0.782–0.864	26.5	0.534	89.0	64.3
Total	0.829	0.788–0.869	29.5	0.535	86.7	66.8

Validation of the cut-off value

The differences in eHealth skills task completion, eHEALS scores, and the number of tasks completed across two groups were all statistically significant. To be specific, in the group with eHealth skills, the proportions of each eHealth skills task completed were all higher than in the group without eHealth skills, ranged from 49.7% to 58.5%, the proportions of all eHealth skills tasks completed was 46.7% higher than in the group without eHealth skills, and the eHEALS scores (34.83 vs 16.68) and number of tasks completed (2.27 vs 0.64) were both higher than in the group without eHealth skills. And according to the discriminant analysis result, 92.2% of original participants were correctly classified. Details are shown in Table 6.

Table 6.

Comparing tasks completion and eHEALS scores in groups with or without eHealth skills.

Item	Possess eHealth skills		χ2/F	p
Item	Yes (n = 188)	No (n = 185)	χ2/F	p
eHEALS scores (Mean/SD)	34.84 (3.63)	16.68 (7.78)	843.147	<0.001
Search the official notification data of yesterday's national COVID-19 outbreak (n/%)	167 (88.8%)	56 (30.3%)	132.996	<0.001
Search for a designated tuberculosis provider in your area (n/%)	139 (73.9%)	36 (19.5%)	111.111	<0.001
If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it (n/%)	120 (63.8%)	26 (14.1%)	96.983	<0.001
The number of respondents got all three tasks correct (n/%)	104 (55.3%)	16 (8.6%)	93.073	<0.001
The number of tasks completed (Mean/SD)	2.27 (0.96)	0.64 (1.01)	253.708	<0.001
Discriminant analysis results	-	-	-	-
Possess eHealth skills /Yes (n/%)	188 (100%)	0 (0%)	-	-
Possess eHealth skills /No (n/%)	29 (17.8%)	156 (88.2%)	-	-

92.2% of participants were correctly classified; -: not involved.

Discussion

The study findings emphasize the low levels of eHealth literacy among rural residents in Gansu. Moreover, this study is the first to determine a cut-off value of 29.5 for adequate eHealth literacy

Compared with previous studies conducted in Zhengzhou(13.76)⁵³ and Taian (15.80),³² the eHealth literacy level has improved in Gansu. This improvement could potentially be ascribed to variances in study design; the mean age of our study's respondents was comparatively younger. In rural settings, the health literacy of older residents tends to decrease as a result of a multitude of factors, including personal, interpersonal, and social/community elements.^18,53,54 Consequently, in rural areas, younger populations were more likely to have higher levels of eHealth literacy. Moreover, there were disparities in timing of the studies, with the development of communication technology and the economy, the popularity of smartphones has facilitated access to eHealth services. The COVID-19 pandemic has contributed to the development of eHealth services, likely contributing to the elevation of the population's eHealth literacy levels.^11,55,56 In the early stages of the COVID 19 pandemic, medical consultations in hospitals under the control of the National Health and Wellness Commission grew 17 times compared to the previous year.⁵⁷ At Alibaba's Ali health, each doctor received an average of 100 patient consultations online per day.⁵⁸ And Ping An Good Doctor, the most popular online e-health APP in China, had 67.3 million users monthly.⁵⁹ Despite the progress made, approximately 17.3% (78/451) of the respondents in the study did not own an electronic device with Internet access or had one but did not use it at all. And eHealth literacy remained at low levels among the respondents who have their own electronic devices. A nationwide web-based survey conducted in February 2020 showed that the eHealth literacy of whole population was 30.32,¹⁰ which was similar to the results of two surveys among university students (30.16 and 30.68, respectively) in China,^60,61 while in our study, eHealth literacy was only 25.85. Low levels of eHealth literacy continue to be a health concern in rural China. In the future, more interventions need to be developed to improve eHealth literacy among rural residents.

Our findings suggested that the actual eHealth skills of respondents are probably not sufficient to fulfill their daily eHealth service requirements. Among the three tasks examined, the best performance was in the COVID-19 epidemic search results. A possible reason was the relative familiarity with the COVID-19 or higher levels of interest, underscoring the crucial role of eHealth in facilitating the control and management of the COVID-19 epidemic.^27,62–64 However, the respondents exhibited relatively weaker eHealth skills in tasks involving other diseases. Thus, there is a general and broad-based requirement to promote general eHealth skills training among residents. The normalization of COVID-19 epidemic prevention and control provided an opportunity for eHealth development including training the public in general eHealth skills.

Additionally, our study also identified a cut-off value for eHEALS scores that indicates adequate eHealth literacy among rural residents in Gansu. The scale is a good predictor of actual eHealth skills (AUC = 0.829) with a cut-off value of 29.5. In validation section, it was ascertained that the value was effective in differentiating between individuals with high and low levels of eHealth literacy. That is, in rural areas, a respondent with eHealth literacy scores greater than 29.5 is likely to possess good eHealth skills. In future eHealth intervention practices, based on this cut-off value, it is possible to quickly categorize respondents and identify priority intervention groups. Furthermore, the cut-off value can be used as a desired goal to guide the development of appropriate intervention strategies and the evaluation of intervention resource investment. Lastly, it can be used to evaluate the effectiveness of the interventions. Following the above steps, the efficiency of interventions may improve, which is helpful in improving the eHealth literacy of rural residents and making eHealth more useful in rural areas. Applying this cut-off value to assess the eHealth literacy of participants indicated that the eHealth literacy levels of rural residents are generally low, probably making it difficult for them to meet daily healthcare service needs. Detailed and comprehensive eHealth literacy interventions alongside the enriching eHealth literacy assessment criteria need to be explored and developed to guide residents toward theoretical to practical behavioral changes. This method and criteria should be validated and examined across other populations and regions in future research.

Conclusion

Our findings indicated the eHealth literacy among rural residents remains at a low level; interventions should be developed to improve the eHealth literacy level within this population. Additionally, we determined a cut-off value of 29.5 for adequate eHealth literacy among rural residents, this method and criteria should be validated and examined across other populations in future research.

Limitations

This study had several limitations. Firstly, due to the absence of a gold standard, we used three self-developed tasks related eHealth skills as a reference standard for assessing eHealth literacy. While we incorporated a wide range of disease elements, the skill assessment tasks might not comprehensively represent the eHealth skill diversity of the respondents. Secondly, due to lack of experience with household surveys, we neglected to record those individuals who declined to participate. Consequently, we were unable to report survey response rates and compare the characteristics of non-participants with those of responders potentially introducing incompleteness or bias into the sample. Thirdly, our sample was obtained by convenience sampling, which might compromise its representativeness and lead to sampling errors. Finally, our sample was from Gansu Province. Consequently, the outcomes may not accurately reflect the eHealth literacy levels of rural residents in other parts of China. The respondents’ eHealth literacy levels might be below the national average due to various factors, including economic level or geographic location.

Footnotes

Acknowledgements

We would like to express our sincere thanks to all the participants in this study. Special thanks to enumerators Junxian Zhao, Tianhu Liang, Meng Luo, Wenpin Hu, Jia Xu, Ya Wang, Shuyi Li, Guoyan Li, Yijun, Liu, Xinyue Li, Xinye Guo, Shuyi, Li, Zhongyu Ma, Hongbing Zhu, and Yuxuan Xin, from School of Public Health, Lanzhou University, for their contribution to data collection.

Contributorship

XY and JL involved in conceptualization; YZ in formal analysis; JL in funding acquisition; XY and HZ in investigation; YZ, XW, MX and JL in writing the original draft; YG, WC, MX and JL in writing the review and editing.

Data availability statement

The datasets used analyzed during the current study available from the corresponding author on reasonable request.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of Sun Yat-sen University ([2019] 140). Informed consent was obtained from all subjects involved in the study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the China Medical Board, grant number:18-301 (Guarantor: Jinghua Li) and Natural Science Foundation of Gansu Province, China, grant number: 22JR5RA506 (Guarantor : Xiaohui Wang).

Guarantor

Jinghua Li.

ORCID iD

Jinghua Li

References

World Health Organization . Shanghai declaration on promoting health in the 2030 Agenda for Sustainable Development. Health Promot Int 2017; 32: 7–8.

Wynn

Gabarron

Johnsen

, et al. Special issue on E-health services. Int J Environ Res Public Health 2020; 17: 2885.

Risling

Martinez

Young

, et al. Evaluating patient empowerment in association with eHealth technology: scoping review. J Med Internet Res 2017; 19: e329.

Wynn

Oyeyemi

Budrionis

, et al. Electronic health use in a representative sample of 18,497 respondents in Norway (the seventh Tromsø study - part 1): population-based questionnaire study. JMIR Med Inform 2020; 8: e13106.

Hao

. The development of online doctor reviews in China: an analysis of the largest online doctor review website in China. J Med Internet Res 2015; 17: e134.

Zhang

Wang

, et al. Analysis of requirements for developing an mHealth-based health management platform. JMIR Mhealth Uhealth 2017; 5: e117.

Chowdhury

Sunna

Ahmed

. Telemedicine is an important aspect of healthcare services amid COVID-19 outbreak: its barriers in Bangladesh and strategies to overcome. Int J Health Plann Manage 2021; 36: 4–12.

Hong

Park

Heo

, et al. Effect of e-health literacy on COVID-19 infection-preventive behaviors of undergraduate students majoring in healthcare. Healthcare (Basel, Switzerland) 2021; 9: 573.

Bacon

Hawley

, et al. Relationship between coronavirus-related eHealth literacy and COVID-19 knowledge, attitudes, and practices among US adults: web-based survey study. J Med Internet Res 2021; 23: e25042.

10.

Liu

. Social media use, eHealth literacy, disease knowledge, and preventive behaviors in the COVID-19 pandemic: cross-sectional study on Chinese Netizens. J Med Internet Res 2020; 22: e19684.

11.

Wang

Sun

Liu

, et al. The use of E-health during the COVID-19 pandemic: a case study in China's Hubei province. Health Sociol Rev: The J Health Sec Austral Sociol Assoc 2022; 31: 215–231.

12.

Oyeyemi

Wynn

. Giving cell phones to pregnant women and improving services may increase primary health facility utilization: a case-control study of a Nigerian project. Reprod Health 2014; 11: 8.

13.

Oyeyemi

Gabarron

Wynn

. Ebola, Twitter, and misinformation: a dangerous combination? BMJ (Clinical Research ed) 2014; 349: g6178.

14.

Gabarron

Serrano

Wynn

, et al. Tweet content related to sexually transmitted diseases: no joking matter. J Med Internet Res 2014; 16: e228.

15.

Norman

Skinner

. Ehealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 2006; 8: e9.

16.

Bergman

Nilsson

Dahlberg

, et al. Health literacy and e-health literacy among Arabic-speaking migrants in Sweden: a cross-sectional study. BMC Public Health 2021; 21: 2165.

17.

Tümer

Sümen

. E-health literacy levels of high school students in Turkey: results of a cross-sectional study. Health Promot Int 2022; 37: daab174.

18.

Shi

Zhang

, et al. In the digital age: a systematic literature review of the e-health literacy and influencing factors among Chinese older adults. Z Gesundh Wiss 2021; 31: –9.

19.

Shiferaw

Tilahun

Endehabtu

, et al. E-health literacy and associated factors among chronic patients in a low-income country: a cross-sectional survey. BMC Med Inform Decis Mak 2020; 20: 81.

20.

Kim

. The relationship between E-health literacy and health-promoting behaviors in nursing students: a multiple mediation model. Int J Environ Res Public Health 2021; 18: 5804.

21.

Özkan

Özer

Özmen

, et al. Investigation of the perceived coronavirus threat, E-health literacy, and psychological well-being in Turkey. Soc Work Public Health 2022; 37: 122–134.

22.

Khasawneh

Nusair

Arabyat

, et al. The association between e-health literacy and willingness to deprescribe among patients with chronic diseases: a cross-sectional study from Jordan. Telemed J e-Health: The Official J Am Telemed Assoc 2022; 28: 1001–1008.

23.

Pourrazavi

Kouzekanani

Bazargan-Hejazi

, et al. Theory-based E-health literacy interventions in older adults: a systematic review. Arch Public Health 2020; 78: 72.

24.

Zhou

Wang

. Improving cancer survivors’ e-health literacy via online health communities (OHCs): a social support perspective. J Cancer Survivorship: Res Practice 2020; 14: 244–252.

25.

The Lancet . Rural health inequities: data and decisions. Lancet (London, England) 2015; 385: 1803.

26.

Aljassim

Ostini

. Health literacy in rural and urban populations: a systematic review. Patient Educ Couns 2020; 103: 2142–2154.

27.

Tran

Phan

, et al. Health literacy, eHealth literacy, adherence to infection prevention and control procedures, lifestyle changes, and suspected COVID-19 symptoms among health care workers during lockdown: online survey. J Med Internet Res 2020; 22: e22894.

28.

Zhu

Österle

. Rural-urban disparities in unmet long-term care needs in China: the role of the hukou status. Soc Sci Med (1982) 2017; 191: 30–37.

29.

Cheng

Duan

. The changing trends of internal migration and urbanization in China: new evidence from the seventh national population census. China Population Dev Stud 2021; 5: 275–295.

30.

The 48th Statistical Report on China’s Internet Development . Available online:https://cnnic.cn/n4/2022/0401/c136-5299.html. (accessed on 11 August 2023).

31.

Cheng

Beauchamp

Elsworth

, et al. Applying the electronic health literacy Lens: systematic review of electronic health interventions targeted at socially disadvantaged groups. J Med Internet Res 2020; 22: e18476.

32.

Dong

Qin

, et al. Analysis of electronic health literacy and its influencing factors among residents aged ≥15 years in Tai'an City. China Public Health 2021; 37: 1319–1322.

33.

Norman

Skinner

. eHEALS: the eHealth literacy scale. J Med Internet Res 2006; 8: e27.

34.

Paramio Pérez

Almagro

Hernando Gómez

, et al. Validation of the eHealth literacy scale (eHEALS) in Spanish university students. Rev Esp Salud Publica 2015; 89: 329–338.

35.

Chung

Nahm

. The Korean eHealth literacy scale (K-eHEALS): reliability and validity testing in younger adults recruited online. J Med Internet Res 2018; 20: e138.

36.

van der Vaart

van Deursen

Drossaert

, et al. Does the eHealth literacy scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011; 13: 86.

37.

Del Giudice

Bravo

Poletto

, et al. Correlation between eHealth literacy and health literacy using the eHealth literacy scale and real-life experiences in the health sector as a proxy measure of functional health literacy: cross-sectional web-based survey. J Med Internet Res 2018; 20: e281.

38.

Lotto

Maschio

Silva

, et al. eHEALS as a predictive factor of digital health information seeking behavior among Brazilian undergraduate students. Health Promot Int 2021; 38: daab182.

39.

Wangdahl

Dahlberg

Jaensson

, et al. Psychometric validation of Swedish and Arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a Swedish context: a study protocol. BMJ Open 2019; 9: e029668.

40.

. The psychometric properties of the Chinese eHealth literacy scale (C-eHEALS) in a Chinese rural population: cross-sectional validation study. J Med Internet Res 2019; 21: e15720.

41.

Qin

. Healthy migrant and salmon bias hypotheses: a study of health and internal migration in China. Soc Sci Med (1982) 2014; 102: 41–48.

42.

Call

Erickson

Dailey

, et al. Attitudes toward telemedicine in urban, rural, and highly rural communities. Telemed J e-Health: The Official J Am Telemed Assoc 2015; 21: 644–651.

43.

Liu

Yeh

Wang

, et al. Relationship between levels of digital health literacy based on the Taiwan digital health literacy assessment and accurate assessment of online health information: cross-sectional questionnaire study. J Med Internet Res 2020; 22: e19767.

44.

Richtering

Hyun

Neubeck

, et al. Ehealth literacy: predictors in a population with moderate-to-high cardiovascular risk. JMIR Hum Factors 2017; 4: e4.

45.

Guo

Zauszniewski

, et al. A reliability and validity study of the electronic health literacy scale among stroke patients in China. Top Stroke Rehabil 2023; 30: 272–280.

46.

Meng

Shen

. Current situation of eHealth literacy and health behaviors of college students in Nanjing. Chin J Health Educ 2018; 34: 254–257.

47.

Xie

Zhang

Mao

, et al. How to predict the electronic health literacy of Chinese primary and secondary school students?: establishment of a model and web nomograms. BMC Public Health 2022; 22: 1048.

48.

Quinn

Bond

Nugent

. Quantifying health literacy and eHealth literacy using existing instruments and browser-based software for tracking online health information seeking behavior. Comput Hum Behav 2017; 69: 256–267.

49.

Zhou

, et al. Psychometric validation and cultural adaptation of the simplified Chinese eHealth literacy scale: cross-sectional study. J Med Internet Res 2020; 22: e18613.

50.

Global tuberculosis report 2022 . Available online:https://www.who.int/teams/global-tuberculosis-programme/tb-reports/global-tuberculosis-report-2022. (accessed on 13 March 2023).

51.

Wang

Cheng

Chen

, et al. Report of the fifth national epidemiological sample survey on tuberculosis in 2010. China J Anti-Tuberculosis 2012; 34: 485–508.

52.

Summary of China cardiovascular health and disease report 2021. Cardiovasc Dis Prevent Control 2022; 22: 20–40.

53.

Liu

Zhang

, et al. Current situation and influencing factors of e-health literacy among rural older adults in Zhengzhou. Modern Prevent Med 2020; 47: 283–309.

54.

Guo

Zhao

Guo

, et al. Socioeconomic disparities in eHealth literacy and preventive behaviors during the COVID-19 pandemic in Hong Kong: cross-sectional study. J Med Internet Res 2021; 23: e24577.

55.

Wynn

. E-Health in Norway before and during the initial phase of the COVID-19 pandemic. Stud Health Technol Inform 2020; 272: 9–12.

56.

Al-Ruzzieh

Ayaad

Qaddumi

. The role of e-health in improving control and management of COVID 19 outbreak: current perspectives. Int J Adolesc Med Health 2020; 34: 139–145.

57.

How can e-health in China enter the post-Coivd era? Available online:https://finance.sina.com.cn/tech/2020-12-31/doc-iiznctke9472110.shtml (accessed on 13 March 2023).

58.

E-health development research report of China in 2020(Full Edition) . Available online:https://zk.cn-healthcare.com/doc-show-39773.html (accessed on 13 March 2023).

59.

China’s online health platforms boom in wake of coronavirus . Available online:https://www.ft.com/content/22b22543-0fb5-4a8a-8ec0-e3fd067a5190 (accessed on 13 March 2023).

60.

Cui

Kaminga

, et al. Associations between health literacy, eHealth literacy, and COVID-19-related health behaviors among Chinese college students: cross-sectional online study. J Med Internet Res 2021; 23: e25600.

61.

Qin

Shi

, et al. Associations of COVID-19 risk perception, eHealth literacy, and protective behaviors among Chinese college students following vaccination: a cross-sectional study. Front Public Health 2021; 9: 776829.

62.

Al-Ruzzieh

Ayaad

Qaddumi

. The role of e-health in improving control and management of COVID 19 outbreak: current perspectives. Int J Adolesc Med Health 2020; 34: 139–145.

63.

Tebeje

Klein

. Applications of e-health to support person-centered health care at the time of COVID-19 pandemic. Telemed J e-Health: The Official J Am Telemed Assoc 2021; 27: 150–158.

64.

Chong

Cheng

Chan

HYL

, et al. COVID-19 pandemic, infodemic and the role of eHealth literacy. Int J Nurs Stud 2020; 108: 103644.