Digital Health/e-Health Literacy among University Students in the COVID-19 Era: A Systematic Review

Abstract

OBJECTIVE

During the severe acute respiratory syndrome coronavirus 2 pandemic, could be observed an established use of online information in the field of coronavirus disease worldwide. As a systematic review study, the present investigation aimed to evaluate related studies about digital health/e-health literacy among university students in the coronavirus disease 2019 era.

METHODS

Three electronic bibliographic databases (PubMed/Medline, Scopus, and Google Scholar) were searched from 2020 until June 2022, and articles were screened according to pre-established inclusion criteria.

RESULTS

Fifteen studies were included in this systematic review study. All of the studies were cross-sectional in design, and in total, 45,255 students were evaluated. The majority of studies report health literacy scores among university students that are lower compared to reference samples. Students’ health literacy is influenced by different variables (age, gender, socioeconomic background, sources of online information, well-being, and satisfaction with data).

CONCLUSION

Digital health literacy (DHL) shapes health behaviors and actions. To enhance DHL, multidisciplinary teams from diverse fields can design curricula suitable for students. The internet’s role in DHL is crucial, but it can also spread misleading content. Therefore, professionals should provide clear, evidence-based information and encourage critical data evaluation. Future studies should use robust sampling methods, consider students with limited internet access, and address the unique needs of specific populations, such as those with disabilities and low socioeconomic status.

Keywords

digital health e-health literacy university students COVID-19 era systematic review

Introduction

The emergence of coronavirus disease 2019 (COVID-19) in Wuhan, China, and turning into a pandemic within months with millions of deaths worldwide caused various medical, psychological, and socio-economic problems.¹ Since the beginning of the disease, there has been an overabundance of true and false information from official and unofficial sources, confusing patients and healthcare workers.² Also, the broad access to the internet and social media platforms leads to an overflow of false information compared to factual information.³ The World Health Organization (WHO) declared that the COVID-19 infodemic leads to challenges in controlling the pandemic and harms physical and psychological health. Promoting accurate information, preventing the spread of misinformation, and increasing health literacy (HL) can help overcome this challenge.⁴

Digital health literacy (DHL) refers to the ability to search electronic information sources and evaluate and use this information health-relatedly.⁵ The DHL has become increasingly important during the COVID-19 pandemic as individuals rely on digital tools and online information for health-related purposes. Studies have shown that individuals with higher levels of DHL are more likely to engage in preventive behaviors such as vaccination uptake.^6–8 While DHL is crucial to promoting health-related outcomes, previous studies reported its inadequacy in different countries. Different studies employed various measurement scales to assess DHL, with the most common ones being the Digital Health Literacy Index (DHLI) and the eHealth Literacy Scale (eHEALS).

Additionally, some studies utilized scales like COVID-HL and e-health literacy (e-HL). The DHLI, available in English and Dutch, contains 21 items rated on a 4-point Likert scale and is intended for the general population.⁹

In contrast, the eHEALS, initially in English, has been translated into multiple languages and comprises 8 items rated on a 5-point Likert scale. Researchers have applied the eHEALS to assess DHL in various populations, including healthy individuals, patients, and healthcare professionals.¹⁰ For example, a study examined e-HL among Chinese students during the pandemic and found a positive association between e-HL scores, adherence to infection prevention and control methods, and lifestyle changes.⁷ Also, another study in Germany reported that 43% of participants were confident in using the internet for health-related information, and higher HL correlated with younger age, higher income, and more education.¹¹ Moreover, a systematic review of 8 studies showed that higher levels of DHL were associated with increased adherence to guidelines and more appropriate preventive behaviors.¹²

Due to the pandemic, the universities were closed, and face-to-face learning was replaced with virtual and online learning, which resulted in increased internet use for obtaining health-related information among students. Online learning affects the students’ education quality and psychological well-being.¹³ For example, a survey conducted in China, Malaysia, and Philippines universities showed that 92.0% of students used search engines and 88.4% of students used social media, whereas 64.7% of students used websites of doctors, and higher HL was associated with more utilization of trustworthy resources.¹⁴ Also, a study in Vietnam on 1003 students reported that 92.8% of students searched for COVID-19 information, and low HL correlated with not searching for hygiene information.¹⁵ Furthermore, a cross-sectional study of 14916 university students in Germany on DHL reported that the most common challenges were the reliability assessment of health-related information, determining whether there was a commercial interest, and finding relevant information.¹⁶

Generally, the COVID-19 pandemic caused more people to rely on accessing medical information and services through online networks. Meanwhile, students are a crucial population using digital platforms to access information and education. University students may not have adequate DHL skills, making them vulnerable to misinformation. Understanding their DHL knowledge level and examining the existing inequalities can be a way to develop interventions and educational health policies to plan to increase students’ DHL.

Given the complexity of these challenges, the primary objective of this study is to assess students’ levels of DHL while recognizing their pivotal role in disseminating and utilizing health-related information. By gaining insights into the strengths and weaknesses of students’ DHL, this research aims to provide valuable information for developing strategies and interventions to enhance students’ DHL. Ultimately, this endeavor will promote more informed and responsible behavior, particularly in health crises like the COVID-19 pandemic. It is worth noting that, to the best of our knowledge, this study represents the first systematic review of DHL among students during the COVID-19 pandemic.

Research design and methods

This systematic review study reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement.¹⁷ Through this systematic review, we purposed to investigate related studies about digital health/e-HL among university students and report such systems’ status during COVID-19.

Registration of review protocol

The protocol for this systematic review was registered and visible in the Open Science Framework (https://doi.org/10.17605/OSF.IO/S8C7Q).

Data sources and searches

Study participants were of either sex or age with no restrictions on health status. No language restriction was applied. Exclusion criteria of the systematic review were as follows: (1) reviews, editorials, abstracts, case reports, practice guidelines, and randomized control trial studies; (2) studies conducted in the general population (non-student). Reference lists of relevant and previous review articles were hand-searched for other relevant studies. Included and excluded studies were collected following the PRISMA flow diagram (Figure 1).

Figure 1.

Flow chart of study selection for inclusion in the systematic review.

Search strategies are provided in Table 1. One of the authors performed a complete search in the references of retrieved articles (Arezoo Faridzadeh). In PubMed, Scopus, and Google Scholar databases up to June 2022 to find all related articles. EndNote 8X software was used to manage retrieved references. Duplicate articles were removed. The screening was initially conducted by titles and then abstracted by 2 authors separately (Reyhaneh Mehrabani Natanzi and Hanieh Karimi). In the next step, the full text of the articles was evaluated by 2 independent reviewers (Arezoo Faridzadeh Ali Faraji). We read the articles independently and in duplicate and determined whether they met inclusion criteria. Discrepancies were resolved by consensus and referring to the original article in consultation with a third author (Niloofar Deravi). Figure is based on PRISMA flow diagram (From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71)

Table 1.

Search strategy of PubMed, Scopus, and Google Scholar databases.

Databases	Search strategy	Results
PubMed 11 June 2022	(“digital health literacy” [Title/Abstract] OR “eHealth literacy” [Title/Abstract]) AND (“college student” [Title/Abstract] OR “university students” [Title/Abstract] OR “university student” [Title/Abstract] OR “University Health Service” [Title/Abstract] OR “Student Health Services” [Mesh Terms] OR “medical student” [Title/Abstract] OR “medical students” [Title/Abstract] OR “nursing students” [Title/Abstract] OR “nursing student” [Title/Abstract] OR “pupil nurse” [Title/Abstract] OR “pupil nurses” [Title/Abstract]) AND (“COVID-19” [MeSH Terms] OR “COVID 19” [Title/Abstract] OR “COVID-19 Virus Disease” [Title/Abstract] OR “COVID 19 Virus Disease” [Title/Abstract] OR “COVID-19 Virus Diseases” [Title/Abstract] OR “Disease, COVID-19 Virus” [Title/Abstract] OR “Virus Disease, COVID-19” [Title/Abstract] OR “COVID-19 Virus Infection” [Title/Abstract] OR “COVID 19 Virus Infection” [Title/Abstract] OR “COVID-19 Virus Infections” [Title/Abstract] OR “Infection, COVID-19 Virus” [Title/Abstract] OR “Virus Infection, COVID-19” [Title/Abstract] OR “2019-nCoV Infection” [Title/Abstract] OR “2019 nCoV Infection” [Title/Abstract] OR “2019-nCoV Infections” [Title/Abstract] OR “Infection, 2019-nCoV” [Title/Abstract] OR “Coronavirus Disease-19” [Title/Abstract] OR “Coronavirus Disease 19” [Title/Abstract] OR “2019 Novel Coronavirus Disease” [Title/Abstract] OR “2019 Novel Coronavirus Infection” [Title/Abstract] OR “2019-nCoV Disease” [Title/Abstract] OR “2019 nCoV Disease” [Title/Abstract] OR “2019-nCoV Diseases” [Title/Abstract] OR “Disease, 2019-nCoV” [Title/Abstract] OR COVID19[Title/Abstract] OR “Coronavirus Disease 2019” [Title/Abstract] OR “Disease 2019, Coronavirus” [Title/Abstract] OR “SARS Coronavirus 2 Infection” [Title/Abstract] OR “SARS-CoV-2 Infection” [Title/Abstract] OR “Infection, SARS-CoV-2” [Title/Abstract] OR “SARS CoV 2 Infection” [Title/Abstract] OR “SARS-CoV-2 Infections” [Title/Abstract] OR “COVID-19 Pandemic” [Title/Abstract] OR “COVID 19 Pandemic” [Title/Abstract] OR “COVID-19 Pandemics” [Title/Abstract] OR “Pandemic, COVID-19” [Title/Abstract])	22
Scopus 11 June 2022	(TITLE-ABS-KEY-AUTH(“university students”) OR TITLE-ABS-KEY-AUTH(“college student”) OR TITLE-ABS-KEY-AUTH(“university student”) OR TITLE-ABS-KEY-AUTH(“college students”) OR TITLE-ABS-KEY-AUTH(“University Health Service”) OR TITLE-ABS-KEY-AUTH(“Student Health Services”) TITLE-ABS-KEY-AUTH(“medical students”) OR TITLE-ABS-KEY-AUTH(“medical student”) OR TITLE-ABS-KEY-AUTH(“nursing students”) OR TITLE-ABS-KEY-AUTH(“nursing student”) OR TITLE-ABS-KEY-AUTH(“pupil nurse”) OR TITLE-ABS-KEY-AUTH(“pupil nurses”) AND (TITLE-ABS-KEY-AUTH(eHealth) OR TITLE-ABS-KEY-AUTH(“public health”) OR TITLE-ABS-KEY-AUTH(“digital health”) OR TITLE-ABS-KEY-AUTH(“virtual health”) OR TITLE-ABS-KEY-AUTH(“health information”) OR TITLE-ABS-KEY-AUTH(“social media”) OR TITLE-ABS-KEY-AUTH(“digital health literacy”) OR TITLE-ABS-KEY-AUTH(“health literacy”) OR TITLE-ABS-KEY-AUTH(“eHealth literacy”) OR TITLE-ABS-KEY-AUTH(“social network”) OR TITLE-ABS-KEY-AUTH(internet)) AND (TITLE-ABS-KEY-AUTH(“COVID-19”) OR TITLE-ABS-KEY-AUTH(“COVID 19”) OR TITLE-ABS-KEY-AUTH(2019-nCoV) OR TITLE-ABS-KEY-AUTH(2019 nCoV) OR TITLE-ABS-KEY-AUTH(“Coronavirus Disease 19”) OR TITLE-ABS-KEY-AUTH(“2019 Novel Coronavirus Disease”) OR TITLE-ABS-KEY-AUTH(“2019 Novel Coronavirus Infection”) OR TITLE-ABS-KEY-AUTH(COVID19) OR TITLE-ABS-KEY-AUTH(“Coronavirus Disease 2019”) OR TITLE-ABS-KEY-AUTH(“Disease 2019, Coronavirus”) OR TITLE-ABS-KEY-AUTH(“SARS Coronavirus 2 Infection”) OR TITLE-ABS-KEY-AUTH(SARS-CoV-2) OR TITLE-ABS-KEY-AUTH(SARS CoV 2))	53
Google Scholar 11 June 2022	allintitle: COVID University OR students OR student OR college OR Services OR medical OR nursing OR nurse OR health “digital health literacy” allintitle: COVID University OR students OR student OR college OR Services OR medical OR nursing OR nurse OR health “eHealth literacy”	41

Eligibility criteria

The authors of the current investigation included published articles with data regarding DHL among university students. Studies investigating this topic on students other than university students, Reviews, case reports, case series, letters to the editor, abstracts, and posters were among the studies that were excluded. Articles without open access and papers written in languages other than English were also disqualified.

Data extraction and quality assessment

For all studies, we extracted information on study characteristics, type of study, population characteristics, theoretical frames, scales used, HL definition, and quality assessment score. Any disagreements or disputes between the 2 evaluators were settled by agreement, then by the third evaluator (Niloofar Deravi), and lastly. Three authors (Mohammad Sadegh Fallahi, Mehrnaz Salahi, and Arezoo Faridzadeh) independently assessed the risk of bias. The JBI critical appraisal tools¹⁸ were used to judge study quality for each type of study. We considered the studies that received half of the total score or higher to be at low risk of bias and studies that scored less than half of the total score to be at high risk, and we eliminated them.

Results

Study selection

After the systematic search, 116 articles were found in 3 databases (53 in Scopus, 22 in Medline/PubMed, and 41 in Google Scholar); 98 remained after removing the duplicates. After screening the titles and abstracts of the studies, 55 articles were excluded following the criteria mentioned in our study, while 43 articles were selected for the full-text screening. After the article selection, 15 studies were included; Figure 1 shows the process.

Investigated Studies were conducted in Portugal, Germany, South Korea, Vietnam, China, the United States, Pakistan, Slovenia, Spain, Puerto Rico, Ecuador, England, Malaysia, the Philippines, and Denmark. All the included studies were cross-sectional. The included studies had a total enrollment of 42785 students, with a sample size ranging from 256 to 14916. Theoretical frameworks for HL were the definition of WHO,¹⁹ Norman and Skinner,²⁰ Sørensen et al,²¹ van der Vaart and Drossaert,⁹ Berkman et al,²² Institute of Medicine,²³ Van Den Broucke et al,²⁴ Dunn and Hazzard,²⁵ and Brørs et al.²⁶ To evaluate DHL, a variety of scales were utilized, including The eHEALS,²⁷ the DHL Instrument,⁹ and COVID-19 Health Literacy Questionnaire.²⁸

The quality scores of each study ranged from 4/8 to 8/8, indicating variations in research quality and methodology among the different investigations. These findings illuminate the multifaceted nature of DHL and its relationship with a diverse set of determinants, offering valuable insights for educators, healthcare professionals, and policymakers seeking to enhance DHL among university students worldwide.

Possible determinants of HL

This study encompasses a series of cross-sectional investigations conducted across diverse countries to explore the determinants of DHL among university students. Each of these studies employed various theoretical frameworks and assessment scales to delve into the factors influencing students’ DHL levels; its details and thematic analysis are presented in Table 2.

Table 2.

Summary of the investigated studies.

First author (year)	Country	Design	Participants	Sex (female)	Scales used	Determinants of digital health literacy	Theoretical frames	Reference
Rosário et al (2020)	Portugal	Cross-sectional	3084 university students	75.7%	COVID-HL (67)	[+] Male gender [−] Using search engines, Wikipedia, and social media [+] Using websites of public bodies	Van der Vaart and Drossaert ⁹	²⁹
Htay et al (2022)	China, Malaysia, and the Philippines	Cross-sectional	5302 university students	75%	COVID-HL	[+] Using trustworthy online information sources (websites of public bodies, health portals, websites of doctors or health insurance companies, and news portals	Van der Vaart and Drossaert ⁹, Norman and Skinner ¹⁰	¹⁴
Zakar (2021)	Pakistan	Cross-sectional	1747 university students in Punjab, Pakistan	52.7%	COVID-HL-Q, DHLI	[+] Female gender [+] Higher sense of coherence [+] Giving importance to information [−] Satisfaction with information	Institute of Medicine ³⁰	³¹
Hong (2021)	South Korea	Cross-sectional	274 university students majoring in healthcare	86.5%	e-HL	[+] Majoring in nursery [+] Perceiving health status as healthy [+] High health concerns	Norman and Skinner ¹⁰	³²
Qin (2021)	China	Cross-sectional	3785 students of Chinese college	40.26%	m-eHEALS	[+] Spending longer time on social media	Brørs et al ²⁶	⁴⁷
Tran (2022)	Vietnam	Cross-sectional	1851 nursing students from 8 universities	93.1%	eHEALS	[+] Male gender [+] Higher ability to pay for medication	Norman and Skinner ¹⁰	³³
Patil (2021)	United States	Cross-sectional	256 college students in the United States	42%	SILS, DHLI	[−] Disability [−] Using YouTube and TikTok [+] Using LinkedIn and Tumblr	Berkman et al ³⁴, Norman and Skinner ¹⁰	⁶
Vrdelja (2021)	Slovenia	Cross-sectional	3621 university students in Slovenia	70.0%	DHLI	[+] Using websites of public bodies and Wikipedia and other encyclopedias [−] Using social media	Norman and Skinner ¹⁰	³⁵
Dadaczynski (2021)	Germany	Cross-sectional	14916 university students aged between 18 and 72 years	48.5%	DHLI	[−] Female gender [+] Higher age [+] Using websites of public bodies [−] Using social media	Sørensen et al ²¹, van der Vaart and Drossaert ⁹	¹⁶
Nguyen et al (2021)	Vietnam	Cross-sectional	1003 university students	70.1%	DHLI (16)	[+] Searching for information on “Hygiene regulations” and “infection control.” [−] Average information satisfaction [+] Subjective well-being	Dunn and Hazzard ²⁵	¹⁵
Rivadeneira (2022)	Spain, Puerto Rico, and Ecuador	Cross-sectional	2318 university students	70.03%	DHLI	[+] Male gender [+] Higher age [+] Higher educational status [+] Satisfaction with information	WHO ⁴	³⁶
Frings et al (2022)	England	Cross-sectional	691 university students from 25 universities	71.5%	DHLI, MacArthur Scale (69), Dark Future Scale (70), the Future Anxiety scale (71)	[−] Future anxiety [+] Satisfaction with information	Norman and Skinner ¹⁰	³⁷
Martins et al (2022)	Portugal	Cross-sectional	1815 university students	75.1%	DHLI	Not mentioned.	van der Vaart and Drossaert ⁹	³⁸
Bak (2022)	Denmark	Cross-sectional	1518 students from University College South	83.4%	DHLI	[−] Younger age [+] Higher subjective social status [−] Using social media [−] Getting help from close social network	Sørensen et al ²¹	³⁹
Chun (2022)	South Korea	Cross-sectional	604 undergraduate students	72.2%	DHLI	[+] Younger age [+] Higher level of social support [+] Higher subjective social support [+] Sufficient pocket money [−] Low well-being	Norman and Skinner ¹⁰, van der Vaart and Drossaert ⁹	⁴⁰

HL: health literacy; SILS: Single-Item Literacy Screener; RLP: remote learning proficiency; DR-T: demands-resources theory; DHLI: Digital Health Literacy Instrument; DHLI: Digital Health Literacy Instrument; eHEALS: eHealth literacy scale; PMHS: Positive Mental Health Scale (PMHS); AIPC: Adherence to Infection Prevention and Control Procedures; S-COVID-19-S: suspected COVID-19 symptoms; P-eHEALS: the Persian translate of the e-Health Literacy scale; WHO: World Health Organization.

Studies using COVID-HL

In the study by Rosário et al (2020) conducted in Portugal, involving 3084 university students, their research was grounded in the theoretical framework of Van der Vaart and Drossaert.⁹ They utilized the COVID-HL scale⁴¹ to identify determinants of DHL. Their findings revealed that being male and favoring the use of websites from public bodies were positively correlated with higher DHL levels, while the use of search engines, Wikipedia, and social media had a negative impact.⁴²

Htay et al (2022) expanded the geographical scope of their study to include China, Malaysia, and the Philippines, involving 5302 university students. Guided by theoretical frameworks such as Van der Vaart and Drossaert⁹ and Norman and Skinner,¹⁰ they employed the COVID-HL scale. Their findings underscored the significance of trustworthy online information sources, such as websites of public bodies and health portals, in enhancing DHL.¹⁴

Zakar (2021) conducted a cross-sectional study involving 1747 university students in Punjab, Pakistan, incorporating multiple theoretical frameworks and scales. Their findings demonstrated that being female, possessing a higher sense of coherence, giving importance to information, and experiencing satisfaction with information sources were positively linked to higher DHL.³¹

Studies using DHLI

Patil (2021) carried out research in the United States involving 256 college students, drawing from theoretical frameworks by Berkman et al³⁴ and Norman and Skinner.¹⁰ Their study utilized the SILS (Single-Item Literacy Screener) and DHLI scales, revealing that individuals with disabilities and extensive users of platforms like YouTube and TikTok exhibited lower DHL levels, whereas using platforms like LinkedIn and Tumblr was associated with enhanced DHL.⁴³

Vrdelja (2021) conducted a cross-sectional study in Slovenia involving 3621 university students, with their research grounded in Norman and Skinner’s theoretical framework.¹⁰ Their study highlighted that students who utilized websites of public bodies and Wikipedia while avoiding excessive use of social media exhibited higher DHL levels.⁴⁴

Dadaczynski (2021) managed a comprehensive cross-sectional study in Germany involving 14,916 university students aged 18 to 72 years, based on theoretical frameworks by Sørensen et al²¹ and van der Vaart and Drossaert.⁹ Their study employed the DHLI scale and identified that being female, older age, and utilizing websites of public bodies positively influenced DHL, while excessive use of social media had a negative impact.⁴⁵

Nguyen et al (2021) conducted a cross-sectional study in Vietnam involving 1003 university students grounded in the theoretical framework of Dunn and Hazzard.²⁵ Their research employed the DHLI scale and revealed that information-seeking behavior related to “Hygiene regulations” and “infection control” was significant for DHL, and subjective well-being positively influenced DHL, while average information satisfaction had a negative impact. These studies contribute to our comprehensive understanding of the multifaceted determinants of DHL among university students across various countries.¹⁵

Rivadeneira (2022) performed a cross-sectional study in Spain, Puerto Rico, and Ecuador involving 2318 university students. The research was grounded in the WHO theoretical framework. Employing the DHLI scale, the study identified several determinants of DHL, including male gender, higher age, higher educational status, and satisfaction with information sources. With a quality score of 7/8, Rivadeneira’s study underscores the role of demographic factors and information satisfaction in influencing DHL among university students in multiple countries.³⁶

Frings D et al (2022) conducted a cross-sectional study in England, encompassing 691 university students from 25 universities. The theoretical framework for this research was based on Norman and Skinner.¹⁰ Multiple scales were utilized, including the DHLI, MacArthur Scale, Dark Future Scale, and the Future Anxiety scale. The study identified that lower levels of future anxiety and higher satisfaction with information sources positively impacted DHL.³⁷

Martins et al (2022) conducted a cross-sectional study involving 1815 university students in Portugal. Although the specific determinants were not mentioned, their study received a perfect quality score of 8/8, reflecting a robust methodology. While the exact determinants remain unspecified, Martins et al’s research highlights the importance of conducting high-quality studies to further our understanding of DHL among university students in Portugal.³⁸

Moving to Denmark, Bak et al performed a cross-sectional study encompassing 1518 students from University College South. The research, grounded in the theoretical framework of Sørensen et al,²¹ employed the DHLI scale. The study identified that younger age, higher subjective social status, avoiding excessive use of social media, and seeking help from close social networks were positively associated with DHL. With a quality score of 8/8, Bak’s study underscores the influence of demographic factors, social status, and media usage patterns in shaping DHL among university students in Denmark.⁴⁶

Chun et al conducted a cross-sectional study in South Korea involving 604 undergraduate students. The research was grounded in the theoretical frameworks of Norman and Skinner¹⁰ and van der Vaart and Drossaert,⁹ utilizing the DHLI scale. The study found that younger age, higher levels of social support, higher subjective social support, sufficient pocket money, and overall well-being were positively associated with DHL. In contrast, low well-being had a negative impact.⁴⁰

Studies using eHEALS

Tran (2022) performed a cross-sectional study in Vietnam, which included 1851 nursing students. Their research was guided by the theoretical framework of Norman and Skinner,¹⁰ and they utilized the eHEALS scale. The study revealed that male gender and a higher ability to pay for medication were positively associated with enhanced DHL among nursing students.³³

Qin et al employed the Mobile eHEALS (m-eHEALS) tool to evaluate the HL of 3785 college students in China. Their findings revealed that the average eHEALS score was 46.61 ± 8.16. Notably, college students who devoted more time to social media exhibited higher levels of e-HL. Furthermore, the study identified a positive correlation between 3 eHEALS domains—self-perception, information acquisition, and interactive judgment—and the knowledge, attitude, and practice scores concerning COVID-19 vaccination.⁴⁷

Studies using e-HL

In a study conducted by Hong et al in Korea, which involved 274 undergraduate students majoring in healthcare fields such as nursing, clinical pathology, and occupational therapy, the e-HL tool was utilized to evaluate e-HL. The findings indicated that the overall mean e-HL score was 3.62 ± 0.60. Notably, students majoring in nursing demonstrated higher e-HL scores compared to those in other fields. Furthermore, students who perceived themselves as having good health and those with elevated health concerns exhibited higher e-HL scores. Moreover, the study revealed a positive correlation between the overall e-HL score and its domains, including functional, communicative, and critical aspects, with scores related to infection-preventive behaviors.³²

Discussion

The DHL of students is affected by different determinants. In this review, we found strong evidence regarding the involvement of gender, age, educational status, socioeconomic status, well-being, social media usage, internet access, satisfaction with information, and sense of coherence in determining the DHL.

During the COVID-19 pandemic, the Internet became the primary source of healthcare information, and due to the shift to online education, students spent more time using the internet for educational and non-educational purposes. As a result, students’ sources of information could significantly affect their DHL.⁴⁸ In this review, using the websites of public bodies was positively associated with DHL in 4 studies.^14,16,29,35 Websites of public bodies provide more reliable and expert-driven information, which makes them more trustworthy, and the recipient has less worry about the correctness of information and can use it in their everyday lives.²¹ In contrast, using social media more often was generally associated with limited DHL,^16,29,35,39 except 1 study by Qin et al⁴⁷ reported that spending longer on social media positively correlated with DHL. Due to the interactive character of social media platforms, they could act as sources of misinformation and disinformation, leading to harmful behaviors.³⁵ The types of social media platforms could also have different impacts on DHL. Qin et al reported that students with high DHL scores used official and public social media more frequently and aggregated social media less frequently than students with low DHL.⁴⁷ This was also consistent with the findings of Patil et al,⁶ suggesting that using LinkedIn and Tumblr was significantly more common in students with adequate DHL than using YouTube and TikTok.⁶

Moreover, well-being was consistently reported to have a positive association with DHL,^15,32,40 whereas disability had a negative correlation.⁶ There is a 2-way association between DHL and well-being. A low DHL could result in low well-being due to the reduced ability to collect adequate healthcare information, critically analyze it, and apply it to daily life. A study of 2236 participants in Hong Kong reported that HL in finding and understanding information significantly correlated with fewer physical symptoms, and higher levels of HL in applying information significantly correlated with greater subjective well-being. The authors concluded that HL directly and indirectly influences physical health and well-being.⁴⁹ On the other hand, low well-being could negatively affect HL by limiting the ability to search and use healthcare information.⁵⁰

There are inconsistencies between studies regarding some of the determinants of DHL. For example, 4 studies^16,29,33,36 with a total of 22,169 participants reported that men had higher scores in DHL compared to women, whereas a study³¹ with 1747 participants reported higher DHL scores in women. In the study by Dadaczynski et al, women had more problems in the domains of information searching and evaluating reliability regarding COVID-19-related information. The authors suggested that in Germany, women are involved in care responsibilities and more concerned about healthcare information than men and search more for healthcare information on the internet. Also, women are more critical and worried about the online information on COVID-19, which could explain reporting lower scores on the mentioned domains of DHL.¹⁶ However, several previous studies on HL showed that women had higher HL scores than men. For instance, a systematic review and meta-analysis of 24 studies assessing HL in people with migration backgrounds reported that women achieved higher HL scores than men.⁵¹ Also, a study on 300 subjects in Iran showed that among the aspects of HL, men achieved higher scores in reading skills and accessing and using health information whereas women had higher scores in health knowledge.⁵² Further studies with robust methodologies are required to compare the aspects of DHL between genders.

Age is another controversial factor among studies of DHL in students. Two studies^36,39 reported that higher age is associated with higher scores on DHL, whereas Chun et al⁴⁰ reported younger age as a positive determinant of DHL. The positive association between the students’ age and HL was also reported in previous studies.^53,54 A recent systematic review investigating HL among university students suggested that older students have more experience interacting with the health care system and communicating with professionals, which promotes their knowledge, self-confidence, and HL.⁵⁵

The included studies have some limitations which should be considered. All the studies are cross-sectional, and causality between factors cannot be determined. Also, the studies are conducted on highly educated students, which could limit the generalizability of these results to the general population. All the studies used online surveys for data collection, and five studies^{14,15,29,33,40} used convenience sampling, limiting their representativeness of the population of students in that country.

Moreover, the data on DHL were collected using self-reported questionnaires, increasing the risk of bias. Also, the studies conducted in the early phases of the pandemic may have an overestimation of DHL because the information on COVID-19 was easier to understand and presented mainly by official websites, and the students had higher adherence compared to the subsequent phases of the pandemic when there were more conflicting information and enduring extended restrictions.^16,39 Moreover, the studies were performed in different universities and different populations, which may affect their results. For instance, females are generally more represented in the included studies, and some studies such as Tran et al,³³ the number of female participants was significantly higher than males, which may affect their results, especially on the role of gender as a determinant of DHL.

Regarding the theoretical frames used in DHL studies, Norman and Skinner¹⁰ introduced “the lily model,” consisting of 6 fields encompassing e-HL. These fields include traditional literacy, information literacy, media literacy, HL, computer literacy, and scientific literacy. Building upon this model, van der Vaart and Drossaert⁹ assess 6 essential skills. These skills encompass operational skills, navigation skills, information search skills, evaluating online information’s reliability and relevance, adding self-generated content to web-based applications, and protecting and respecting privacy while using the internet. These DHLI skills can be linked to various aspects of the lily model. For example, operational skills, defined as the ability to “use the computer and internet browser,” align with computer literacy. Similarly, navigation skills, information search skills, and the evaluation of online information’s reliability and relevance fall under the purview of information literacy and media literacy. However, Norman and Skinner do not explicitly address skills such as adding self-generated content to web-based applications and ensuring privacy and respect while using the internet.

Furthermore, concerning the HL component of Norman and Skinner’s model, their definition is based on a 1999 article by the American Medical Association (AMA).⁵⁶ The AMA defines HL as “a constellation of skills, including the ability to perform basic reading and numerical tasks required to function in the healthcare environment. Patients with adequate HL can read, understand, and act on healthcare information.” In contrast, the theoretical frameworks employed by the Institute of Medicine Committee on Health Literacy,³⁰ Berkman et al,³⁴ and Dunn and Hazzard²⁵ rely on the definition proposed by Ratzan and Parker.⁵⁷ This definition suggests that HL is “the capacity to obtain, process, and understand” information within a health-related context. These definitions share common ground in aspects of HL, such as reading (obtaining) and understanding. However, it has been argued that the AMA’s definition emphasizes skills, while Ratzan and Parker’s definition emphasizes HL goals.³⁴ In addition, the definition by WHO⁴ encompasses aspects like “access, understand, evaluate, and use” in the context of HL, aligning more closely with goal-oriented definitions. In summary, while the theoretical frameworks employed in the articles reviewed vary, they do overlap in numerous aspects, potentially facilitating the derivation of generalized conclusions.

Furthermore, studies used different scales for measuring the DHL of their participants. The most commonly used scales were DHLI and eHEALS; however, some studies used COVID-HL (which utilizes parts of DHLI) and e-HL. DHLI was developed initially in English and Dutch for the general population and had 21 items scored on a 4-point Likert scale.⁹ eHEALS was initially developed in English⁵⁸ and was further translated into other languages such as Chinese,⁵⁹ German,⁶⁰ and Persian.⁶¹ eHEALS has 8 items scored on a 5-point Likert scale and was used in diverse populations including healthy people, patients, and health professionals.⁵⁸ There are also conceptual and theoretical framework differences between eHEALS and DHLI. eHEALS was developed based on the components of the Lily model, including traditional, computer, information, health, media, and science literacies and Social cognitive theory.¹⁰ In contrast, DHLI was derived from a formative study on actual performance tests.⁶² A systematic review evaluating the different measures of e-HL reported that eHEALS had a moderate quality of evidence for comprehensibility, but the quality was deficient for DHLI. Also, eHEALS had a generally moderate quality of structural validity, but this quality was low for DHLI.⁶³ In addition, e-HL has 31 items divided into 3 categories, including functional, communicative, and critical e-HL, and they are scored based on a 5-item Likert scale. e-HL has an excellent internal consistency with Cronbach’s α of 0.91.³²

Currently, the COVID-19 pandemic has subsided. However, this does not diminish the importance of DHL assessment. Even in the post-COVID-19 era, digital technologies remain strong in providing health information and services. Ensuring university students’ proper DHL as the young and future generation of society is necessary for a more informed and healthy society.

Every day, we see the development of digital technologies worldwide, especially in health and medicine. As professionals and future researchers, students are essential to this development. Also, as health care recipients, increasing their DHL knowledge can improve their care of themselves and others. The increase of DHL in students is the basis for their future adaptation to the world of digital health technologies.

In addition, the present review has some limitations. First, only English studies already published or available were considered. Second, the general quality of included studies is high; however, some studies are medium to low quality. Third, the studies used various sampling methods and assessment tools and were conducted in different countries, limiting the comparison between them.

The findings of this review underscore the need for future studies with robust sampling methods, such as cluster sampling, and a longitudinal design to pinpoint the specific factors influencing DHL. Moreover, researchers should acknowledge the limitations of relying solely on online surveys and strive to include students with limited internet access and usage in their studies. Furthermore, it is crucial to address the unique needs of specific populations, including individuals with disabilities, those experiencing low physical or psychological well-being, and those with low socioeconomic status. Policies aimed at enhancing DHL should be tailored to accommodate these groups.

Conclusion

DHL plays a crucial role in shaping behaviors and actions related to health and well-being, with significant implications for individuals and society. Numerous factors influence students’ DHL, providing valuable insights for university officials and government organizations to develop more inclusive and effective programs aimed at enhancing students’ DHL levels. To achieve this, multidisciplinary teams comprising experts from diverse fields, such as medicine, psychology, and computer sciences, can contribute by designing curricula suitable for a wide range of students. During the pandemic, the internet emerged as the primary source of healthcare information. While it offers unparalleled access to vast amounts of information, it also harbors the potential for disseminating misleading content. Therefore, professionals and official organizations must prioritize providing clear, evidence-based information. Additionally, individuals should be encouraged to adopt a more critical approach when evaluating the validity of the information they encounter and sharing it on social media platforms.

Footnotes

Acknowledgments

We appreciate all the authors of included studies.

FUNDING

The authors received no financial support for the research, authorship, and/or publication of this article.

DECLARATION OF CONFLICTING INTERESTS

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author contributions

Study design and conception: ND and PB. Data search: ArF. Study selection: ArF, RM, HK, AlF, and ND. Data extraction: SI, MaF, and ND. Quality assessment: MSF, ArF, and MS. Statistical analysis and interpretation: MB and NN. Drafting the manuscript: MSF, ArF, MS, RM, HK, AlF, SI, MaF, MB, NN, SAM, MR, SFS, MoF, PB, and ND. Critical revision: SAM, MR, SFS, MoF, and PB. All authors have approved the submitted version.

References

Solomon

Normandin

Bhattacharyya

, et al. Neuropathological features of COVID-19. N Engl J Med. 2020;383(10):989‐992.

Hua

Shaw

. Coronavirus (COVID-19) “infodemic” and emerging issues through a data lens: the case of China. Int J Environ Res Public Health. 2020;17(7):2309.

Vosoughi

Roy

Aral

. The spread of true and false news online. Science. 2018;359(6380):1146‐1151.

WHO. Managing the COVID-19 infodemic: promoting healthy behaviours and mitigating the harm from misinformation and disinformation. World Health Organization; 2020. https://www.who.int/news/item/23-09-2020-managing-the-covid-19-infodemic-promoting-healthy-behaviours-and-mitigating-the-harm-from-misinformation-and-disinformation

van Kessel

Wong

BLH

Clemens

Brand

. Digital health literacy as a super determinant of health: more than simply the sum of its parts. Internet Interv. 2022;27:100500.

Patil

Kostareva

Hadley

, et al. Health literacy, digital health literacy, and COVID-19 pandemic attitudes and behaviors in U.S. college students: implications for interventions. Int J Environ Res Public Health. 2021;18(6):3301.

Cui

Kaminga

Cheng

. Associations between health literacy, eHealth literacy, and COVID-19-related health behaviors among Chinese college students: cross-sectional online study. J Med Internet Res. 2021;23(5):e25600.

Griebel

Enwald

Gilstad

Pohl

A-L

Moreland

Sedlmayr

. eHealth literacy research—quo vadis? Inf Health Soc Care. 2018;43(4):427‐442.

van der Vaart

Drossaert

. Development of the digital health literacy instrument: measuring a broad spectrum of Health 1.0 and Health 2.0 skills. J Med Internet Res. 2017;19(1):e27.

10.

Norman

Skinner

. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. 2006;8(2):e9.

11.

De Santis

Jahnel

Sina

Wienert

Zeeb

. Digital health literacy and COVID-19 pandemic: results of a nationwide survey in Germany. Eur J Public Health. 2021;31(Supplement_3):ckab164.274.

12.

Ameri

Dastani

Sabahi

, et al. The role of E-health literacy in preventive behaviors for COVID-19: a systematic review. J Health Literacy. 2022;6(4):88‐97.

13.

Dodd

Dadaczynski

Okan

McCaffery

Pickles

. Psychological wellbeing and academic experience of university students in Australia during COVID-19. Int J Environ Res Public Health. 2021;18(3):866.

14.

Htay

MNN

Parial

Tolabing

, et al. Digital health literacy, online information-seeking behaviour, and satisfaction of COVID-19 information among the university students of East and South-East Asia. PLoS One. 2022;17(4):e0266276.

15.

Nguyen

LHT

MTH

Tran

LTM

, et al. Digital health literacy about COVID-19 as a factor mediating the association between the importance of online information search and subjective well-being among university students in Vietnam. Front Digit Health. 2021;3:739476.

16.

Dadaczynski

Okan

Messer

, et al. Digital health literacy and web-based information-seeking behaviors of university students in Germany during the COVID-19 pandemic: cross-sectional survey study. J Med Internet Res. 2021;23(1):e24097.

17.

Moher

Shamseer

Clarke

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1‐9.

18.

JB Institute. Critical appraisal tools. https://jbi.global/critical-appraisal-tools.

19.

World Health Organization. Managing the COVID-19 infodemic: promoting healthy behaviours and mitigating the harm from misinformation and disinformation; 2020.

20.

Norman

Skinner

. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. 2006;8(2):e506.

21.

Sørensen

Van den Broucke

Fullam

, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12(1):1‐13.

22.

McCormack

Berkman

Davis

. Health literacy: what is it. J Health Commun Int Perspect. 2010;15:9‐19.

23.

Kindig

Panzer

Nielsen-Bohlman

Health literacy: a prescription to end confusion. National Academies Press; 2004.

24.

Van Den Broucke

Levin-Zamir

Schaeffer

, et al. Digital health literacy in general populations–an international comparison. Eur J Public Health. 2020;30(Supplement_5):ckaa165.

25.

Dunn

Hazzard

. Technology approaches to digital health literacy. Int J Cardiol. 2019;293:294‐296.

26.

Brørs

Norman

Norekvål

. Accelerated importance of eHealth literacy in the COVID-19 outbreak and beyond. Eur J Cardiovasc Nurs. 2020;19(6):458‐461.

27.

Norman

Skinner

. eHEALS: the eHealth literacy scale. J Med Internet Res. 2006;8(4):e507.

28.

Zakar

Iqbal

Zakar

Fischer

. COVID-19 and health information seeking behavior: digital health literacy survey among university students in Pakistan. Int J Environ Res Public Health. 2021;18(8):4009.

29.

Rosário

Martins

MRO

Augusto

, et al. Associations between COVID-19-Related Digital Health Literacy and Online Information-Seeking Behavior among Portuguese University Students. Int J Environ Res Public Health. 2020;17(23):8987.

30.

Institute of Medicine Committee on Health Literacy. In: Nielsen-Bohlman

Panzer

Kindig

, eds. Health Literacy: A Prescription to End Confusion. National Academies Press (US); 2004.

31.

Zakar

Iqbal

Zakar

Fischer

. COVID-19 and health information seeking behavior: digital health literacy survey among university students in Pakistan. Int J Environ Res Public Health. 2021;18(8):4009.

32.

Hong

Park

Heo

Jung

Lee

Hwang

. Effect of e-Health literacy on COVID-19 infection-preventive behaviors of undergraduate students majoring in healthcare. Healthcare (Basel). 2021;9(5):573.

33.

Tran

HTT

Nguyen

Pham

TTM

, et al. Predictors of eHealth literacy and its associations with preventive behaviors, fear of COVID-19, anxiety, and depression among undergraduate nursing students: a cross-sectional survey. Int J Environ Res Public Health. 2022;19(7):3766.

34.

Berkman

Davis

McCormack

. Health literacy: what is it? J Health Commun. 2010;15(Suppl 2):9‐19.

35.

Vrdelja

Vrbovšek

Klopčič

Dadaczynski

Okan

. Facing the growing COVID-19 infodemic: digital health literacy and information-seeking behaviour of university students in Slovenia. Int J Environ Res Public Health. 2021;18(16):8507.

36.

Rivadeneira

Miranda-Velasco

Arroyo

Caicedo-Gallardo

Salvador-Pinos

. Digital health literacy related to COVID-19: validation and implementation of a questionnaire in Hispanic university students. Int J Environ Res Public Health. 2022;19(7):4092.

37.

Frings

Sykes

Ojo

, et al. Differences in digital health literacy and future anxiety between health care and other university students in England during the COVID-19 pandemic. BMC Public Health. 2022;22(1):1‐9.

38.

Martins

Augusto

Martins

MRO

, et al. Adaptation and validation of the digital health literacy instrument for Portuguese university students. Health Promot J Austr. 2022;33(Suppl 1):390‐398.

39.

Bak

Krammer

Dadaczynski

, et al. Digital health literacy and information-seeking behavior among university college students during the COVID-19 pandemic: a cross-sectional study from Denmark. Int J Environ Res Public Health. 2022;19(6):3676.

40.

Chun

Park

Choi

Yoon

Okan

Dadaczynski

. Validating the digital health literacy instrument in relation to COVID-19 information (COVID-DHL-K) among South Korean undergraduates. Int J Environ Res Public Health. 2022;19(6):3437.

41.

Dadaczynski

Okan

Rathmann

. COVID-19 Health Literacy Survey: University Students (COVID-HL-Survey). Questionnaire and Scale Documentation. Version 1. Interdisciplinary Centre for Health Literacy Research and Fulda University of Applied Sciences, Public Health Centre, Bielefeld University; 2020.

42.

Rosário

Martins

Augusto

, et al. Associations between COVID-19-related digital health literacy and online information-seeking behavior among Portuguese university students. Int J Environ Res Public Health. 2020;17(23):8987.

43.

Patil

Kostareva

Hadley

, et al. Health literacy, digital health literacy, and COVID-19 pandemic attitudes and behaviors in US college students: implications for interventions. Int J Environ Res Public Health. 2021;18(6):3301.

44.

Vrdelja

Vrbovšek

Klopčič

Dadaczynski

Okan

. Facing the growing COVID-19 infodemic: digital health literacy and information-seeking behaviour of university students in Slovenia. Int J Environ Res Public Health. 2021;18(16):8507.

45.

Dadaczynski

Okan

Messer

46.

Bak

Krammer

JØ

Dadaczynski

47.

Qin

Shi

Duan

, et al. Social Media use, eHealth literacy, knowledge, attitudes, and practices toward COVID-19 vaccination among Chinese college students in the phase of regular epidemic prevention and control: a cross-sectional survey. Front Public Health. 2021;9:754904.

48.

Jahan

Hosen

Mamun

, et al. How has the COVID-19 pandemic impacted internet use behaviors and facilitated problematic internet use? A Bangladeshi study. Psychol Res Behav Manage. 2021;14:1127‐1138.

49.

Zhang

PPL

Chung

JWY

. How different health literacy dimensions influences health and well-being among men and women: the mediating role of health behaviours. Health Expect. 2021;24(2):617‐627.

50.

Virlée

van Riel

ACR

Hammedi

. Health literacy and its effects on well-being: how vulnerable healthcare service users integrate online resources. J Serv Market. 2020;34(5):697‐715.

51.

Chakraverty

Baumeister

Aldin

, et al. Gender differences of health literacy in persons with a migration background: a systematic review and meta-analysis. BMJ Open. 2022;12(7):e056090.

52.

Heizomi

Iraji

Vaezi

Bhalla

Morisky

Nadrian

. Gender differences in the associations between health literacy and medication adherence in hypertension: a population-based survey in Heris County, Iran. Vasc Health Risk Manag. 2020;16:157‐166.

53.

Rababah

Al-Hammouri

Drew

Aldalaykeh

. Health literacy: exploring disparities among college students. BMC Public Health. 2019;19(1):1401.

54.

Budhathoki

Pokharel

Jha

, et al. Health literacy of future healthcare professionals: a cross-sectional study among health sciences students in Nepal. Int Health. 2019;11(1):15‐23.

55.

Kühn

Bachert

Hildebrand

, et al. Health literacy among university students: a systematic review of cross-sectional studies. Front Public Health. 2021;9:680999.

56.

Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. JAMA. 1999;281(6):552‐557.

57.

Ratzan

Parker

Selden

Zorn

. National Library of Medicine Current Bibliographies in Medicine: Health Literacy. National Institutes of Health; 2000.

58.

Norman

Skinner

. eHEALS: the eHealth literacy scale. J Med Internet Res. 2006;8(4):e27.

59.

Zhou

Wong

Chang

Wang

. Psychometric validation and cultural adaptation of the simplified Chinese eHealth literacy scale: cross-sectional study. J Med Internet Res. 2020;22(12):e18613.

60.

Soellner

Huber

Reder

. The concept of eHealth literacy and its measurement. J Media Psychol. 2014;26(1):29‐38.

61.

Lin

Broström

Griffiths

Pakpour

. Psychometric evaluation of the Persian eHealth literacy scale (eHEALS) among elder Iranians with heart failure. Eval Health Prof. 2020;43(4):222‐229.

62.

van der Vaart

Drossaert

de Heus

Taal

van de Laar

. Measuring actual eHealth literacy among patients with rheumatic diseases: a qualitative analysis of problems encountered using Health 1.0 and Health 2.0 applications. J Med Internet Res. 2013;15(2):e27.

63.

Lee

Chae

. eHealth literacy instruments: systematic review of measurement properties. J Med Internet Res. 2021;23(11):e30644.