Sage Journals: Discover world-class research

Abstract

In addition to the infectious disease, the COVID-19 pandemic was accompanied by an infodemic that worsened the outbreak. Increased digital health literacy (DHL) skills and information-seeking behaviors are crucial for accessing, understanding, and evaluating online health information, especially during a pandemic, where misinformation imposes immediate health consequences. This study explores the predictors of DHL among Romanian students and their information-seeking behaviors during the COVID-19 pandemic. Data was collected through a cross-sectional study design over 3 months (December 2020–February 2021) and a self-reported web-based questionnaire from 1,381 university students from Romania assessing DHL and information-seeking behaviors. We used descriptive statistics, Cronbach Alpha coefficients, and univariate and multivariate binary logistic regressions to explore DHL’s predictors in the Romanian sample. Results showed that overall, the study sample had high levels of DHL. However, almost half of the respondents reported having problems assessing whether the information searched related to coronavirus is reliable or not, and 95% shared someone else’s private information online. Age, subjective social status, gender, and chronic diseases were identified as predictors for DHL. Students’ DHL skills need to be strengthened using tailored educational programs and interventions emphasizing the quality, accuracy, and timely provision of online information on official public health platforms. Additional actions include fostering collaborations between educational institutions and public health authorities, alongside developing smartphone applications for real-time guidance on health topics. It is of the utmost importance to address the spread of the infodemic through different strategies, to combat misinformation among university students.

Keywords

digital health literacy health literacy COVID-19 pandemic students

Introduction

In addition to the infectious disease, the control of the COVID-19 pandemic was further hindered by the massive spread of misinformation, commonly referred to as the “infodemic.” This term describes the overload of false or misleading information shared during the disease outbreak. Amidst an infodemic, confusion, risk-taking behaviors, and distrust in health authorities prevail, potentially aggravating and prolonging the health emergency (Rubinelli et al., 2022). Dr. Tedros, the General Director of the World Health Organization, has emphasized that along with the response efforts taken to combat the spread of the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), the infodemic represents a concomitant battle that is highly contagious. It also only jeopardizes the global actions to control the pandemic growth by disseminating an overabundance of inaccurate information throughout online and offline environments (World Health Organization, 2020). This wealth of information and information streams supplied by the pandemic context put a strain on the collective and individual’s critical thinking ability to read and select appropriate data sources (Paakkari & Okan, 2020). To overcome the threat posed by the infodemic, each one’s digital health literacy level is crucial.

Digital health literacy (DHL) is defined as the “ability to seek, find, understand, and appraise health information from electronic sources” (Norman & Skinner, 2006), being able to use the information obtained to tackle and solve a health-related problem (Norman & Skinner, 2006). DHL is essential as the use of false information may immediately affect the health of one individual escalating at the community level. However, various factors can influence how a person assimilates information, acts according to guidelines, and makes thoughtful decisions (Swain, 2016). Nonetheless, citizens’ poor health literacy levels did not receive appropriate scrutiny as a major public health issue, especially in the university student population (Paakkari & Okan, 2020). Existing data shows that the level of DHL is associated with students’ health outcomes (Yang et al., 2017) and the level of skills to handle online information (Hsu et al., 2014). With the COVID-19 pandemic progression, the need for precise and reliable information has increased. People turned to online sources more than before the pandemic to satisfy this need (World Health Organization, 2020). Evidence shows that the choices of information sources may influence the individuals’ level of knowledge and, subsequently, the health beliefs and preventive behaviors they adopt (Wong & Sam, 2010; Zhao et al., 2020). Thus, strengthening and promoting preventive behaviors among populations have been pivotal to control the COVID-19 pandemic (Centers for Disease Control and Prevention, 2021; Liu, 2020).

Considering the landscape of Central and Eastern Europe, Romania’s digital literacy levels have consistently remained challenging. Based on a 2014 European report, less than half of Romania’s population (47%) used the Internet to search for health-related information, falling below the European average of 59%. Moreover, only 42% of Romanian respondents expressed confidence in their ability to use such information, and merely 36% reported knowing where to find reliable health-related content online (European Commission, 2014). This concerning trend persisted into more recent findings from 2021 and 2023, as Eurostat data indicated, Romania had the lowest percentage (28%) of individuals aged 16 to 74 possessing basic digital skills, falling behind Bulgaria (31%) and Poland (43%) (Eurostat, 2023; TRIO, 2022). Studies on COVID-19 from Romania focusing on students showed significant rates of vaccine hesitancy among students and struggle with understanding health information (Bălan et al., 2021).

The first cross-sectional study on health literacy in Romania uncovers that a significant portion of respondents (21.6%) encounter difficulties in interpreting media health information for illness prevention (Coman et al., 2022). Findings indicate that 7.5% have inadequate health literacy, 33.2% face problematic literacy, while 59.2% demonstrate sufficient literacy. Age, gender, education, and self-reported health status emerge as key determinants, while residential area shows no association. Particularly, males, older individuals, those with lower education levels, and those reporting poorer health exhibit lower health literacy levels (Coman et al., 2022). However, to our best knowledge, there are no studies in Romania that assess information-seeking behaviors and digital health literacy among students during the COVID-19 pandemic. Therefore, this study aims to explore the predictors of digital health literacy among students from Romania and their information-seeking behaviors during the COVID-19 pandemic.

Methodology

Study Design

Data were collected electronically using a cross-sectional study design over a period of 3 months (December 2020–February 2021) and using a self-reported web-based questionnaire. The questionnaire was disseminated among a nonrandomized sample of students enrolled at a university in Romania at the moment of questionnaire completion using student groups from social media and official emails sent by the universities’ administration staff. Before data collection started, the study received ethical approval from the host institution.

Study Setting, Population, and Sample

The study was disseminated entirely online and targeted students pursuing a degree at a university in Romania during the COVID-19 pandemic. Data was collected using an online questionnaire created in the Qualtrics platform and shared through social media platforms, such as Facebook and Instagram. The dissemination strategy aimed to engage Romanian student groups and organizations, including volunteer groups and university faculties, to share the survey link and promote its dissemination within their respective student networks. The final convenience study sample included 1,381 students who agreed to complete the questionnaire, and answered all the variables included in the manuscript. In order to be eligible to participate in the study, individuals had to: (1) be able to read, understand and agree on the informed consent, (2) be enrolled at a university from Romania amid the COVID-19 pandemic, (3) speak Romanian, (4) have access to the internet, and (5) be 18-year-old or older. Participants who did not meet all the inclusion criteria mentioned above were not eligible to participate in the study.

Data Collection

Dadaczynski and colleagues created the study design and the questionnaire consisting of validated scales adapted for the COVID-19 pandemic context, along with the recently developed scales by the COVID-HL network (Dadaczynski et al., 2020). Within the COVID-HL network, various countries used the same scales designed by Dadaczynski and colleagues or adaptation of them to conduct further research papers. The variables of interest for the present study are described in the following paragraphs.

The socio-demographic data included gender (Male, Female, Others), age in absolute numbers, study level (Bachelor’s degree, Master’s degree, other, e.g., PhD.), the fundamental domain of study (Mathematics and natural sciences, Engineering sciences, Biological and biomedical sciences, Social sciences, Human sciences, and art, The science of sports and physical education) (Romanian Government, 2018), subjective social status (SSS) (Adler & Stewart, 2007), financial satisfaction (Not sufficient, Less sufficient, Sufficient, Completely sufficient) and chronic disease status (Yes, No). The German version of the MacArthur Scale was used to assess the respondents’ subjective social status. The participants had to position themselves from 1 to 10 in the social hierarchy that best reflects their social status. The recoded categories involved three groups: “low SSS (1–4), medium SSS (5–7), and high SSS (8–10)” (Dadaczynski et al., 2020).

To evaluate participants’ digital health literacy, we used five subscales from the Digital Health Literacy Instrument (DHLI) (Van Der Vaart & Drossaert, 2017). The total of five subscales targeted topics on COVID-19 or related subjects for (1) online searching for information, (2) adding self-generated content, (3) reliability evaluation, (4) determining relevance, and (5) online privacy protection (Dadaczynski et al., 2021). The answers to the first four subscales consisted of a 4-point Likert scale ranging from “very difficult” to “very easy” and for the last subscale from “never” to “often” (Van Der Vaart & Drossaert, 2017). Each subscale had three items adapted to the pandemic context (e.g., “When you search the Internet for information on the coronavirus or related topics, how easy or difficult is it for you to…”).

Data Analysis

The data collected were analyzed using IBM SPSS version 23 (IBM SPSS Statistics, 2021). We used descriptive statistics to explore participants’ characteristics provided as means, standard deviations (SD), percentages (%), and valid percentages (%*) for the socio-demographic and digital health literacy levels.

We used Cronbach’s alpha to test the reliability of the five subscales of DHL. For the first four scales, the Cronbach alpha was acceptable to good (.740 < α < .768), but there was low reliability (α = .402) for the subscale for protecting privacy. Therefore, it was omitted from further analyses. The first four subscales had a satisfactory Cronbach alpha (.7 < α > .8) (Van Der Vaart & Drossaert, 2017), similar to other studies from the COVID-HL Consortium (Dadaczynski et al., 2021; Rosário et al., 2020). For further analyses, the DHLI subscales with reliable internal consistency (all subscales of DHLI except for “protecting privacy”) were dichotomized, considering the median split, after computing in a sum the items from each subscale, into two levels of DHL: “limited” and “sufficient.”

Subsequently, all statistical tests performed considered the p-value of p < .05, and if this criterion was attained, they were considered statistically significant. Using univariate and multivariate binary logistic regressions, we then analyzed associations between the four subscales of digital health literacy (dependent/outcome variable) and socio-demographic factors (independent/predictor variables). An adjusted odds ratio (OR) with a 95% confidence interval (CI) was considered to determine the strength of association between the dependent and independent variables. We have chosen binary logistic regression analysis because the outcome variable represented by the levels of digital health literacy was empirically dichotomized (as there were no predefined cut-off values), into two categories: limited and sufficient. Predictors for socio-demographic factors included “Gender,”“Age,”“Subjective social status,”“Financial satisfaction,” and “Chronic disease.”

Results

After cleaning the data by removing duplicate entries and filtering out missing responses for the DHL subscales or substantially incomplete survey responses, the final dataset included 1,381 valid surveys (out of 1,877 total surveys) completed by students aged between 18 and 39 years. Students from 38 universities and 15 cities in Romania completed the survey. In terms of geographical area, most respondents were from Bucharest (43.1%), Timişoara (15.4%), and Cluj-Napoca (11%).

Divided by gender, Table 1 illustrates the respondents’ socio-demographic characteristics.

Table 1.

Socio-Demographic Characteristics of Study Participants (n = 1,381).

Socio-demographic characteristics	Total		Female		Male
Socio-demographic characteristics	n	%	n	%	n	%
Participants	1,381	100	422	30.6*	955	69.4*
Study course	1,372	99.3*	418	99.1*	950	99.5%
Bachelor’s degree	1,151	83.9	350	83.7	798	84
Master’s degree	213	15.5	67	16	145	15.3
Other (e.g., PhD)	8	0.6	1	0.2	7	0.7
Study fundamental domain	1,354	98*	412	97.6*	938	98.2*
Mathematics and natural sciences	109	8.1	38	9.2	71	7.6
Engineering sciences	266	19.6	96	23.3	167	17.8
Biological and biomedical sciences	362	26.7	88	21.4	273	29.1
Social sciences	223	16.5	57	13.8	166	17.7
Human sciences and art	191	14.1	65	15.8	126	13.4
The science of sports and physical education	4	0.3	3	0.7	1	0.1
Missing	199	14.7	65	15.4	134	14.3
Subjective social status	1,376	99.6*	421	99.8*	951	99.6*
Low	813	59.1	250	59.4	562	59.1
Middle	534	38.8	158	37.5	373	39.1
High	29	2.1	13	3.1	16	1.7
Financial situation satisfaction	1,380	99.9*	422	100	954	99.9*
Not sufficient	224	16.2	69	16.4	154	16.1
Less sufficient	755	54.7	242	57.3	512	53.7
Sufficient	331	24	96	22.7	234	24.5
Completely sufficient	70	5.1	15	3.6	54	5.7
Chronic disease	1,290	93.4*	390	92.4*	896	93.8*
Yes	174	13.5	43	11	129	14.4
No	1,116	86.5	347	89	767	85.6
	M	SD	M	SD	M	SD
Age	21.9	3.701	22.39	4.304	21.67	3.382

Note. n = frequency of participants who answered; % = valid percent; * = percent; M = mean; SD = standard deviation.

The majority of participants were males (69.4%), and from the total respondents, significantly more were currently pursuing a bachelor’s degree (83.9%), while only a minority were enrolled in a master’s degree (15.5%). Regarding the fundamental study domain, a fourth of the respondents study biological and biomedical sciences (26.7%), followed by engineering sciences (19.6%) and social sciences (16.5%). As for the Subjective Social Status, more than half of the participants (59.1%) self-reported a low SSS and some of them (38.8%) a middle SSS. Moreover, more than half of the students were not satisfied with their financial situation, considering insufficient money at disposal (54.7%). Lastly, 13.5% of participants were suffering from a chronic disease.

Exploring the five digital health literacy subscales, Figures 1 and 2 illustrate the valid percentages of students’ answers on the dimensions of “information search” and “evaluating reliability.” Regarding the subscale of “information search,” students who responded positively to the previous question on searching for information related to COVID-19 or similar issues within the past 4 weeks, indicated they most often encounter difficulties in finding the specific information they are searching for (27.7%), closely followed by choosing from all the information they find (26.9%), while less difficult was to use proper words or search queries to find the information searched (7.4%).

Figure 1.

Responses for the “information search” subscale of the Digital Health Literacy Instrument.

Figure 2.

Responses for “evaluating reliability” subscale of the Digital Health Literacy Instrument.

Within the subscale of “adding self-generated content,” the most difficult item, as reported by students, was to express in writing their ideas, thoughts, or feelings (24.5%), then clearly formulate their health-related questions or worry when writing an online message (20.1%), followed by writing the message as such to be understandable for the audience (18.2%).

However, in Figure 2, the greatest difficulty across all dimensions of digital health literacy could be seen in “evaluating reliability” for determining the reliability of the information searched associated with health (44.1%), but also to decide if a message has a commercial interest (30.8%) while checking the same information on different sources (15.1%) was less difficult for them.

Further, in the “determining relevance” subscale, the students found it most difficult to apply the information found online in their daily life (26.9%), decide if the information is applicable to them (25.1%), and lastly to use the information found (e.g., on protective measures, hygiene regulations) to make decisions in regards to their health (16.7%).

In the last subscale, “protecting privacy,” we observed some heterogeneity among items. Therefore, 94.7% of respondents shared someone else’s information, 84.6% shared their personal information via the internet, and 63.8% of the students found it most difficult to decide who can read their posted message.

Table 2 to 5 depicts the associations between the level of digital health literacy and socio-demographic factors, both in univariate regression analysis and also upon adjustment for gender, age, subjective social status, financial satisfaction, and chronic disease predictors. We observed that younger students had higher odds of achieving a sufficient level of digital health literacy with regard the subscales of “information search” (OR = 0.949, CI 95% = [0.903, 0.998], p < .05), “adding self-generated content” (OR = 0.927, CI 95% = [0.891, 0.964], p < .05), “evaluating reliability” (OR = 0.956, CI 95% = [0.926, 0.988], p < .05), and “determining relevance” (OR = 0.962, CI 95% = [0.929, 0.995], p < .05).

Table 2.

Binary Logistic Regression Models of Socio-Demographic Predictors of Levels of Digital Health Literacy of the “Information Search” Subscale Categorized in Limited Versus Sufficient Among Students from Romania (n = 1,381).

Characteristics subscale DHL	Information search
	Model 1					Model 2
	B	Sig.	β	95% CI		B	Sig.	β	95% CI
	B	Sig.	β	Lower	Upper	B	Sig.	β	Lower	Upper
Gender
Female	−0.158	.353	0.854	0.612	1.191	−0.051	.773	0.951	0.675	1.34
Male	Ref					Ref
Age	−0.049	.047	0.952	0.906	0.999	−0.052	.041	0.949	0.903	0.998
Subjective social status	0.103	.069	1.108	0.992	1.238	0.102	.097	1.107	0.982	1.248
Financial satisfaction
Sufficient	0.111	.524	1.117	0.795	1.571	0.113	.546	1.12	0.775	1.618
Not sufficient	Ref					Ref
Chronic disease
Yes	−0.044	.846	0.956	0.611	1.497	−0.076	.742	0.927	0.59	1.457
No	Ref					Ref

Note. 95% CI = 95% confidence interval; Model 1—univariate model; Model 2—multivariate model; Nagelkerke R² = 16%.

p-Value is statistically significant at a 95% confidence level.

Table 3.

Binary Logistic Regression Models of Socio-Demographic Predictors of Levels of Digital Health Literacy of the “Adding Self-Generated Content” Subscale Categorized in Limited Versus Sufficient Among Students from Romania (n = 1,381).

Characteristics subscale DHL	Adding self-generated content
	Model 1					Model 2
	B	Sig.	β	95% CI		B	Sig.	β	95% CI
	B	Sig.	β	Lower	Upper	B	Sig.	β	Lower	Upper
Gender
Female	−0.265	.042	0.767	0.594	0.99	−0.211	.122	0.809	0.619	1.058
Male	Ref					Ref
Age	−0.080	.000	0.923	0.889	0.958	−0.076	.000	0.927	0.891	0.964
Subjective social status	0.113	.005	1.119	1.034	1.211	0.118	.009	1.125	1.03	1.23
Financial satisfaction
Sufficient	−0.031	.810	0.97	0.753	1.248	−0.130	.360	0.878	0.664	1.16
Not sufficient	Ref					Ref
Chronic disease
Yes	0.042	.809	1.043	0.74	1.47	0.026	.883	1.027	0.724	1.456
No	Ref					Ref

Note. 95% CI = 95% confidence interval. Model 1—univariate model; Model 2—multivariate model; Nagelkerke R² = 30%.

p-Value is statistically significant at a 95% confidence level.

Table 4.

Binary logistic regression models of socio-demographic predictors of levels of digital health literacy of the “Evaluating reliability” subscale categorized in Limited vs Sufficient among students from Romania (n = 1,381).

Characteristics subscale DHL	Evaluating reliability
	Model 1					Model 2
	B	Sig.	β	95% CI		B	Sig.	β	95% CI
	B	Sig.	β	Lower	Upper	B	Sig.	β	Lower	Upper
Gender
Female	−0.444	.000	0.641	0.506	0.812	−0.433	.001	0.649	0.506	0.832
Male	Ref					Ref
Age	−0.054	.001	0.948	0.919	0.978	−0.045	.007	0.956	0.926	0.988
Subjective social status	0.100	.009	1.105	1.026	1.19	0.101	.017	1.106	1.018	1.202
Financial satisfaction
Sufficient	0.082	.495	1.085	0.859	1.371	0.024	.859	1.024	0.790	1.327
Not sufficient	Ref					Ref
Chronic disease
Yes	−0.287	.087	0.751	0.541	1.042	−0.333	.05	0.716	0.513	1
No	Ref					Ref

Note. 95% CI = 95% confidence interval. Model 1—univariate model; Model 2—multivariate model; Nagelkerke R² = 32%.

p-Value is statistically significant at a 95% confidence level.

Table 5.

Binary Logistic Regression Models of Socio-Demographic Predictors of Levels of Digital Health Literacy of the “Determining Relevance” Subscale Categorized in Limited Versus Sufficient Among Students from Romania (n = 1,381).

Characteristics subscale DHL	Determining relevance
	Model 1					Model 2
	B	Sig.	β	95% CI		B	Sig.	β	95% CI
	B	Sig.	β	Lower	Upper	B	Sig.	β	Lower	Upper
Gender
Female	−0.252	.044	0.777	0.608	0.993	−0.262	.047	0.77	0.594	0.997
Male	Ref					Ref
Age	−0.039	.017	0.961	0.931	0.993	−0.039	.025	0.962	0.929	0.995
Subjective social status	0.121	.002	1.129	1.046	1.219	0.116	.008	1.124	1.031	1.224
Financial satisfaction
Sufficient	0.289	.019	1.335	1.05	1.697	0.204	.132	1.226	0.941	1.599
Not sufficient	Ref					Ref
Chronic disease
Yes	0.101	.553	1.106	0.793	1.542	0.064	.711	1.066	0.76	1.494
No	Ref					Ref

Note. 95% CI = 95% confidence interval. Model 1—univariate model; Model 2—multivariate model; Nagelkerke R² = 25%.

p-Value is statistically significant at a 95% confidence level.

Furthermore, Model 2, from Tables 3 to 5, shows that with every unit increase in the subjective social status the odds of achieving a sufficient level of DHL increase by 1.12 times for the “adding self-generated content” (CI 95% = [1.03, 1.23], p < .05), 1.1 times for the “evaluating reliability” (CI 95% = [1.018, 1.202], p < .05), and 1.12 times for the “determining relevance” (CI 95% = [1.031, 1.224], p < .05) subscales.

Moreover, compared to females, male students were more likely to achieve a sufficient level for the “evaluating reliability” (OR = 0.649, CI 95% = [0.506, 0.832], p < .05), and “determining relevance” (OR = 0.77, CI 95% = [0.594, 0.997], p < .05) subscales. Lastly, when considering all predictors, not suffering from a chronic disease was associated with increased odds of achieving a sufficient level for the “evaluating reliability” subscale (OR = 0.716, CI 95% = [0.513, 1], p < .05).

Discussion

This study explored predictors of digital health literacy among students from Romania and their information-seeking behaviors during the COVID-19 pandemic. To our knowledge, there are no previous cross-sectional studies investigating the information-seeking behaviors and the levels of digital health literacy in university students from Romania during the COVID-19 pandemic, so our study brings forward novel information relevant for the Romanian context.

While most Romanian university student respondents in our study demonstrated sufficient levels of digital health literacy during the initial months of the COVID-19 pandemic in 2020, it is essential to contextualize these findings. As abundant information about the coronavirus began circulating in the media, measures such as lockdowns, physical distancing, and mask mandates were therefore implemented by the Romanian Government (Cernicova-Buca & Palea, 2021). Despite push-based strategies to combat misinformation, primarily involving delivering information through an official online platform providing reliable information on public health issues (Ştiri oficiale, 2021), inconsistencies in communication from official authorities, conflicting messages, and non-compliance with protective measures significantly undermined the establishment of public trust (Cernicova-Buca & Palea, 2021). This highlights the importance of pull-based communication, where individuals actively seek information by themselves, requiring a high level of health literacy (Dadaczynski et al., 2021; Sørensen et al., 2015).

Our results show that the study sample had overall high levels of digital health literacy. However, almost half of the respondents reported having problems assessing whether the information searched related to coronavirus was reliable or not. Students faced challenges in finding the exact information they were looking for, making choices from all the information found, deciding if the information found applies to them and whether they can apply it in their daily life. They also struggled with deciding whether a message was written with commercial interest and expressing their opinion, thoughts, or feelings in writing. Our results are similar with studies conducted in other countries using the same scales, which showed that the reliability of information represented a problem for digital health literacy levels, but also distinguishing between commercial interest or finding the information they are looking for (Dadaczynski et al., 2021; Rosário et al., 2020; Zakar et al., 2021). Moreover, our results are similar to research from Germany, showing that female students presented lower digital health literacy skills as compared to males across subscales of “evaluating reliability” and “determining relevance” (Dadaczynski et al., 2021).

The findings of our study revealed that almost all respondents struggled to protect their privacy and others online, as they shared theirs and someone else’s private information online, either intentionally or unintentionally. Security and data protection are important problems in the circumstances of digitalization, and other studies report similar issues related to privacy while recommending a future reformulation of public measures to emphasize the role of media providers concerning protective measures (Dadaczynski et al., 2021; Van Der Vaart & Drossaert, 2017).

In line with our study’s outcomes, associations of the predictors of age, subjective social status, gender, chronic diseases, and digital health literacy levels have been reported by other studies. In most of them, older students, students reporting a low subjective social status, female students, and students suffering from a chronic disease had lower odds of achieving sufficient DHL on different subscales (Amoah et al., 2021; Dadaczynski et al., 2021; Kor et al., 2021; Zakar et al., 2021). These results are important to be taken into consideration because although some of the effect sizes were small, they have the potential to impact the health and well-being of university students.

An important finding of the multivariate regression analysis is related to the gender of the students, where female students reported having more difficulties in evaluating the reliability of COVID-19-related information and determining its relevance. Data shows that Romanian women are usually the ones in charge of the household care and health related-issues and report higher levels of health literacy. Therefore, they might be more aware of health-related information and more susceptible to challenges and difficulties of assessing the reliability of information, especially when offered conflicting information and incoherent or unclear communication (Coman et al., 2022; Muresan et al., 2022; Somogyi et al., 2022).

Another important finding of our study is related to the age of the respondents, our study showing that younger students reported higher digital health literacy skills especially for the “information search,”“adding self-generated content,” but also for the “evaluating reliability” and “determining relevance.” These findings are in line with research showing that older adults struggle with health literacy skills, but also with digital skills, making their access to online trustworthy health information even more difficult (Coman et al., 2022; Dadaczynski et al., 2021; Sørensen et al., 2015).

The results of the study raise concerns and have implications to the public health field, as students’ digital health literacy skills need to be strengthened. When students access information that is incomplete, inaccurate, or untrustworthy, and they also lack the DHL skills to assess such information, it will lead to poor decisions related to their health and risky behaviors (Dadaczynski et al., 2021). Our findings suggest that there is a need to implement different health education measures to improve students’ digital health literacy skills, and to advocate for better communication strategies at national level, strategies that are evidence-based and offer accurate and timely information in times of crisis.

Limitations

The present study has several limitations. The results are not representative of all Romania students, since we have only used a convenience sample based on how we managed to collect data during the COVID-19 pandemic. Moreover, students enrolled in university studies who do not possess an account on social media platforms or do not read emails received from their university may have been missed. Besides, the results may not be transferable to a different target population, university students being privileged to pursue higher education amid the pandemic and having different socioeconomic status compared to people their age with different levels of education. Therefore, the results of the study are not generalizable to the entire population of students from Romania and should be interpreted with caution given the limitations previously discussed. Additionally, the data collection took place at the end of 2020 and the beginning of 2021 and was highly influenced by the COVID-19 restrictions existent at that point in time. Considering the rapid change of the pandemic in knowledge, information sources, and the apparition of vaccines against COVID-19, the answers could be different if collected at another point in time.

Implications for Future Research

Future research on assessing the levels of digital health literacy among university students from Romania should continue, as poor digital health literacy has consequences on the individual itself and influences health-related choices, impacting the society and community in which they live. Furthermore, we can assume that people with lower levels of education than university students, who already encounter difficulties with digital health literacy, may be more vulnerable to lower levels of digital health literacy (Dadaczynski et al., 2021). Hence, the target group for assessing the levels of digital health literacy in Romania may be expanded to include other population categories apart from students, such as older people or children, who have access to the Internet and may benefit from increased levels of digital health literacy.

Conclusions

This study shows that overall, the targeted university students in Romania possess a sufficient level of digital health literacy regarding online information related to COVID-19. Nevertheless, a significant number of students still encounter challenges with specific capabilities to deal with health-related information on COVID-19, such as assessing the information’s reliability and making a choice from it. Action must be taken to strengthen the students’ digital health literacy skills and design interventions to improve the quality, accuracy, and timely provision of online information during health emergencies, such as the pandemic, on official public health platforms. Additionally, strategies to check in time the information delivered on the internet and increase the levels of digital health literacy of the main stakeholders that create and provide online health information. Besides, education programs to help students improve their digital health literacy skills should be considered (Centers for Disease Control and Prevention, 2021; Dadaczynski et al., 2021; World Health Organization, 2023).

Considering these insights, it is crucial to underscore the significance of addressing the infodemic more comprehensively. Solutions such as classes in which students explore alternative explanations and questions are shown to be effective in teaching them how to consume and evaluate news and information (Kaufman, 2020). Therefore, it is imperative to take into consideration a range of interventions, such as compulsory educational classes, aimed at combating the harmful dissemination of disinformation, misinformation, and ill-intended content related to COVID-19 (or other health-related threats) among university students.

Footnotes

Acknowledgements

The authors would like to acknowledge the Global Health Literacy Research Network (formerly the COVID-HL Network) for their support and extend our sincere thanks to the university leaders for their participation in the study. This research is part of the products of the International Consortium of COVID-HL Universities study, with data from Romania.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

The study involving human participants was reviewed and approved by the Institutional Review Board—Public Health (IRB-PH Protocol #2020-201113-005). The participants provided their written informed consent to participate in this study.

ORCID iD

Mădălina A. Coman

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Adler

Stewart

(2007). The MacArthur Scale of subjective social status. https://macses.ucsf.edu/research/psychosocial/subjective.php

Amoah

P. A.

Leung

A. Y. M.

Parial

L. L.

Poon

A. C. Y.

Tong

H. H. Y.

W. I.

Wong

E. M. L.

Kor

P. P. K.

Molassiotis

(2021). Digital health literacy and health-related well-being amid the COVID-19 pandemic: The role of socioeconomic status among university students in Hong Kong and Macao. Asia-Pacific Journal of Public Health, 33(5), 613–616. https://doi.org/10.1177/10105395211012230

Bălan

Bejan

Bonciu

Eni

C. E.

Ruţă

(2021). Romanian medical students’ attitude towards and perceived knowledge on COVID-19 vaccination. Vaccines, 9(8), 854. https://doi.org/10.3390/VACCINES9080854

Centers for Disease Control and Prevention. (2021). How to protect yourself & others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019-ncov%2Fprepare%2Fprevention.html

Cernicova-Buca

Palea

(2021). An appraisal of communication practices demonstrated by Romanian district public health authorities at the outbreak of the COVID-19 pandemic. Sustainability, 13(5), 2500. https://doi.org/10.3390/su13052500

Coman

M. A.

Forray

A. I.

Van den Broucke

Chereches

R. M.

(2022). Measuring health literacy in Romania: Validation of the HLS-EU-Q16 survey questionnaire. International Journal of Public Health, 67, 1604272. https://doi.org/10.3389/ijph.2022.1604272

Dadaczynski

Okan

Messer

Leung

A. Y. M.

Rosário

Darlington

Rathmann

(2021). Digital health literacy and web-based information-seeking behaviors of university students in Germany during the COVID-19 pandemic: Cross-sectional survey study. Journal of Medical Internet Research, 23(1), e24097. https://doi.org/10.2196/24097

Dadaczynski

Rathmann

(2020). COVID-19 health literacy survey: University students (COVID-HL-survey). Questionnaire and Scale Documentation.

European Commission. (2014). Flash Eurobarometer 404 “European citizens’ digital health literacy.”https://doi.org/10.2759/86596

10.

Eurostat. (2023). Individuals level of digital skills (from 2021 onwards). Retrieved May 12, 2024, from https://ec.europa.eu/eurostat/databrowser/view/ISOC_SK_DSKL_I21__custom_11448093/default/bar?lang=en

11.

Hsu

Chiang

Yang

(2014). The effect of individual factors on health behaviors among college students: The mediating effects of eHealth literacy. Journal of Medical Internet Research, 16(12), 3542. https://doi.org/10.2196/jmir.3542

12.

IBM SPSS Statistics. (2021). IBM SPSS Statistics 23. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-23

13.

Kaufman

(2020). Civic education in a fake news era: Lessons for the methods classroom. Journal of Political Science Education, 17(2), 326–331. https://doi.org/10.1080/15512169.2020.1764366

14.

Kor

P. P. K.

Leung

A. Y. M.

Parial

L. L.

Wong

E. M. L.

Dadaczynski

Okan

Amoah

P. A.

Wang

S. S.

Deng

Cheung

T. C. C.

Molassiotis

(2021). Are people with chronic diseases satisfied with the online health information related to COVID-19 during the pandemic? Journal of Nursing Scholarship, 53(1), 75–86. https://doi.org/10.1111/jnu.12616

15.

Liu

P. L.

(2020). COVID-19 information seeking on digital media and preventive behaviors: The mediation role of worry. Cyberpsychology, Behavior, and Social Networking, 23(10), 677–682. https://doi.org/10.1089/cyber.2020.0250

16.

Muresan

I. C.

Harun

Andreica

Chiciudean

G. O.

Kovacs

Oroian

C. F.

Brata

A. M.

Dumitras

D. E.

(2022). Household attitudes and behavior towards the food waste generation before and during the COVID-19 pandemic in Romania. Agronomy, 12(3), 746. https://doi.org/10.3390/agronomy12030746

17.

Norman

C. D.

Skinner

H. A.

(2006). eHealth literacy: Essential skills for consumer health in a networked world. Journal of Medical Internet Research, 8(2), e9. https://doi.org/10.2196/jmir.8.2.e9

18.

Paakkari

Okan

(2020). COVID-19: Health literacy is an underestimated problem. The Lancet Public Health, 5(5), e249–e250. https://doi.org/10.1016/S2468-2667(20)30086-4

19.

Romanian Government. (2018). Nomenclatorul domeniilor şi al specializărilor/programelor de studii universitare. Hotărâre 158/2018. https://lege5.ro/Gratuit/gi3tmmjvgq3a/nomenclatorul-domeniilor-si-al-specializarilor-programelor-de-studii-universitare-hotarare-158-2018?dp=gi2tmmjxgi2dimy

20.

Rosário

Martins

M. R. O.

Augusto

Silva

M. J.

Martins

Duarte

Fronteira

Ramos

Okan

Dadaczynski

(2020). Associations between COVID-19-related digital health literacy and online information-seeking behavior among Portuguese university students. International Journal of Environmental Research and Public Health, 17(23), 8987. https://doi.org/10.3390/ijerph17238987

21.

Rubinelli

Purnat

T. D.

Wilhelm

Traicoff

Namageyo-Funa

Thomson

Wardle

Lamichhane

Briand

Nguyen

(2022). WHO competency framework for health authorities and institutions to manage infodemics: Its development and features. Human Resources for Health, 20(1), 1–14. 10.1186/s12960-022-00733-0

22.

Somogyi

Nagy

Geambaşu

Gergely

(2022). “The children, the family, the household, and myself, these made the quarantine up for me, and I was really happy with it”—Positive evaluations of the first COVID-19 lockdown among middle-class Hungarian mothers. Journal of Family Studies, 29(4), 1904–1922. https://doi.org/10.1080/13229400.2022.2103017

23.

Sørensen

Pelikan

J. M.

Röthlin

Ganahl

Slonska

Doyle

Fullam

Kondilis

Agrafiotis

Uiters

Falcon

Mensing

Tchamov

Van Den Broucke

, & Helmut Brand. (2015). Health literacy in Europe: Comparative results of the European health literacy survey (HLS-EU). European Journal of Public Health, 25(6), 1053–1058. https://doi.org/10.1093/eurpub/ckv043

24.

Ştiri oficiale. (2021). Ultimele informaţii oficiale. COVID-19 ştiri oficiale. https://stirioficiale.ro/informatii

25.

Swain

G. R.

(2016). How does socioeconomic disadvantage drive poor health outcomes? Journal of Health Disparities Research and Practice, 9(2), 33. https://www.irp.wisc.edu/resource/how-does-economic-and-social-disadvantage-affect-health/

26.

TRIO. (2022). Romania national report summary. Retrieved May 12, 2024, from https://www.TRIO_National-Report-Summary-Romania-vs1.1.pdf (trioproject.eu)

27.

Van Der Vaart

Drossaert

. (2017). Development of the digital health literacy instrument: Measuring a broad spectrum of health 1.0 and health 2.0 skills. Journal of Medical Internet Research, 19(1), 6709. https://doi.org/10.2196/jmir.6709

28.

Wong

L. P.

Sam

I. C.

(2010). Public sources of information and information needs for pandemic influenza A(H1N1). Journal of Community Health, 35(6), 676–682. https://doi.org/10.1007/s10900-010-9271-4

29.

World Health Organization. (2020). Immunizing the public against misinformation. https://www.who.int/news-room/feature-stories/detail/immunizing-the-public-against-misinformation

30.

World Health Organization. (2023). Infodemic. https://www.who.int/health-topics/infodemic#tab=tab_3

31.

Yang

S. C.

Luo

Y. F.

Chiang

C. H.

(2017). The associations among individual factors, eHealth literacy, and health-promoting lifestyles among college students. Journal of Medical Internet Research, 19(1), 5964. https://doi.org/10.2196/jmir.5964

32.

Zakar

Iqbal

Zakar

M. Z.

Fischer

(2021). COVID-19 and health information seeking behavior: Digital health literacy survey amongst university students in Pakistan. International Journal of Environmental Research and Public Health, 18(8), 4009. https://doi.org/10.3390/ijerph18084009

33.

Zhao

Fan

Basnyat

(2020). Online health information seeking using “#COVID-19 patient seeking help” on Weibo in Wuhan, China: Descriptive study. Journal of Medical Internet Research, 22(10), e22910. https://doi.org/10.2196/22910

Digital Health Literacy Predictors and COVID-19 Online Information-Seeking Behaviors Among University Students from Romania

Abstract

Keywords

Introduction

Methodology

Study Design

Study Setting, Population, and Sample

Data Collection

Data Analysis

Results

Discussion

Limitations

Implications for Future Research

Conclusions

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

Ethical Approval

ORCID iD

Data Availability Statement

References