Abstract
The first project of the Action Network on Measuring Population and Organizational Health Literacy was the Health Literacy Population Survey 2019–2021 (HLS19). Based on the HLS19's experience, improvements were needed to the communicative health literacy questionnaire before it could be used in the second wave of the HLS survey. Our aim was to test the comprehensibility of the Hungarian version of the revised questionnaire with cognitive interviews. Two questions caused comprehension problems: (1) discussing health information with your doctor from other sources such as internet, family and friends—participants found this question too long and difficult, so we changed the sentence structure to make it more understandable; (2) expressing personal views and preferences to your doctor—the interviewees misunderstood the question, they did not realize that we were referring to health-related personal views. During the translation process, it was challenging to convey the meaning of “health concern” in Hungarian, the version with examples in parentheses was deemed the most effective. The final version of the revised questionnaire seemed to be understandable and comprehensible.
Introduction
Patient experience could be influenced by many factors, and health literacy, particularly communicative health literacy (COM-HL), could be one of them.1,2 In this study, we refer to COM-HL as “patients’ communicative and social skills that enable them to actively engage in face-to-face encounters with health care professionals, to give and seek information, derive meaning from it and apply this information in decision-making and in coproducing their health care.” 3 This definition, together with an instrument to measure this construct, was developed under the aegis of the Action Network on Measuring Population and Organizational Health Literacy (M-POHL) of the World Health Organization Regional Office for Europe. The tool was first used in the Health Literacy Population Survey 2019–2021 (HLS19). 3 Based on the experience of the HLS19, the questionnaire could have been more targeted and could benefit from the addition of harder-to-endorse items. Given that some items are underdiscriminated in a few countries, these items could be replaced. 4 In the HLS19, both a short and a long version were available, and the development of a medium length scale was also intended. As the M-POHL intends to conduct the HLS survey regularly, these shortcomings must be addressed before the COM-HL questionnaire can be used for the next data collection wave. This is why we worked on a revised version of the questionnaire as part of an international working group (WG). The WG has members from all M-POHL countries who are interested in measuring COM-HL, including Hungary. The new and/or revised items were discussed by the WG, translated into the national languages and tested for comprehensibility. The necessity of further modifications or omissions of items was decided based on experiences in different countries. The 8 countries participated in the initial phase of the revision based on their resources, meaning they finished at different times. Hungary was among the first countries started the comprehensibility tests. Our aim was to test the comprehensibility of the Hungarian version of the revised COM-HL questionnaire.
Method
The Hungarian version of the COM-HL questionnaire was pretested for comprehensibility in the form of cognitive interviews in 2 waves: the first from February to April 2024 (n = 15) and the second in September 2024 (n = 16). The aim of the first wave was to identify problematic items that needed to be omitted or modified, while the modified, potentially final version of the items was checked in the second wave. The interviews were conducted in person (n = 11 in the first wave and n = 14 in the second wave) or online (n = 4 in the first wave and n = 2 in the second wave), depending on the preferences of the participants. Convenience sampling (accessibility-based selection) was used to recruit the first participants; after which other respondents were recruited with snowball technique. Participation was voluntary without remuneration.
No personal data were collected from respondents, only their feedback, comments, and some basic demographic data (sex, age, education) and factors that might influence their answers (such as whether they speak Hungarian as their mother tongue or have any health professional education or work experience).
The study was approved by the relevant ethics committee University of Debrecen (first wave DE RKEB/IKEB 6810-2024; second wave DE RKEB/IKEB 6810A-2024). In accordance with the Declaration of Helsinki, informed consent was obtained from all participants.
Initial Questionnaire Modifications
The Instrument for measuring Communicative Health Literacy with Physicians in Health Care Services (HLS19-COM-P) 5 was used in the HLS19, its items in the English (original) version can be seen in Table 1. The introductory question was: “On a scale from very easy to very difficult, how easy would you say it is for you” with the response categories very easy-easy-difficult-very difficult. From the original version, some items were modified (Table 1).
Items of the Communicative Health Literacy Questionnaire Under Revision.
Abbreviations: HLS19, Health Literacy Population Survey 2019–2021 original item; HLS19_mod, Health Literacy Population Survey 2019–2021 modified item; WG, Communicative Health Literacy Working Group of the HLS24 Project.
These items were only used in the initial stage of the first wave.
In the current survey, we didn’t use the items 1, 2, 4, 7, and 10 of the HLS19-COM-P questionnaire (Table 1) based on the agreement of the WG. New questions were also created to address the shortcomings of the HLS19-COM-P. The new items and their different versions were prepared on the basis of discussions within the WG and the results of the cognitive interviews conducted in different countries (the details can be found in the minutes of the WG meetings, but these are not publicly available).
The research team translated the items into Hungarian, which were then translated back into English by an independent translator who was unaware of the original questionnaire. The WG leader then compared the 2 English versions, making changes until a consensus was reached.
Cognitive Interviewing
Cognitive interviews were used to assess the participants’ understanding of the questions and any difficulties they experienced when answering them, and whether they provided the information that the researchers expected when answering the questions, that is, they understood the concept to which the question applied.6,7 We used additional questions in the first phase to get deeper information on how they interpret the items and factors that can influence their answer: (1) Which doctor were you thinking of when you answering these questions?; (2) How often do you see your doctor?; (3) Do you talk to him on the phone? If yes, how often? The interviews were voice recorded and/or notes were taken, based on the interviewee's preference.
First wave of cognitive interviews: In the initial stage of the cognitive interviews, 4 people were interviewed and 15 items were included in the questionnaire. Following a discussion about our experiences with the WG members, some items were modified or deleted, resulting in a 12-item version of the questionnaire. Five people then shared their thoughts on this version. Finally, a 9-item version of the questionnaire was created and tested by interviewing 6 people. Second wave of cognitive interviews: After a consensual decision in the WG about the final 9-item version of the COM-HL questionnaire, it was tested by 16 respondents.
Concurrent probes were used to minimize memory decay and identify item specific comprehension and/or wording issues. A template was created to summarize the participants’ feedback. Both sociodemographic information and comments about each item were recorded for each participant in that template. It was used to identify problematic items, collect the reasons for that, the recommended changes to the item and see which subgroup (eg, the elderly or those with a primary education) mentioned concerns about the item. Based on this information, a recommendation was made as to whether the item should be revised, retained or discarded.
Results
Altogether 31 respondents were involved in the 2 waves of the cognitive interviews, 15 in the first and 16 in the second wave (Table 2), their mother tongue was Hungarian.
Cognitive Interview Participants’ Demographic Characteristics.
Two of the questions caused comprehension problems for the lower-educated respondents. The first was about discussing health information with your doctor from other sources such as the internet, family and friends. They perceived it too long and hard to understand, so we modified sentence structure to make it more comprehensible (the examples were inserted after the relevant word and not at the end of the sentence). This resulted in a shorter and more specific question in Hungarian. The second question was about expressing personal views and preferences to your doctor. In that case, the 2 youngest participants found it difficult to understand, they missed the fact that we consider health-related personal views (similarly to international experiences). That's why it was clarified in the new version of this item: express your “health-related” views and preferences to your doctor (originally: express your “personal” views and preferences to your doctor).
Some participants mentioned that they found it hard to choose the best answer, because they were not able to decide, so they would have preferred a “middle” option: nor easy, nor difficult. Others were never in a situation related to the question (eg, they had never tried to discuss health information with their doctor), which is why it was hard for them to answer. As the COM-HL questionnaire is part of a larger HL questionnaire package, it was not possible to modify the response categories. Respondents were unable to interpret questions about judging or evaluating their doctors’ recommendations because they usually accept or follow these without questioning their usefulness. These items were therefore omitted from the final item set.
During the translation process, it was challenging to get the meaning of “health concern” in Hungarian, different versions were tested to find one that best captured the intended meaning. In this case, the solution was a translation with examples in brackets: health concern (eg, health problems, diseases, screenings, vaccinations, or lifestyle). The other items retained the same.
The majority of the respondents said that they had their general practitioner in mind when answering the questions, around half of them met with their doctor twice per year and they preferred a personal consultation to a phone call. Some interviewees also mentioned that their perception of the consultation was strongly influenced by their physician's communication style and skills, which could have affected their answers to these items.
Discussion
In the case of cross-cultural research, it is utmost important to ensure that the participants from different population groups interpret the items in the same way. That's why a literal translation of the questionnaire is not enough but it should be adapted into an understandable and relevant form, while preserving the meaning of the original items. 8 In addition to the careful translation process, cognitive interviews are appropriate tools for identifying and correcting errors in questionnaires. 7
The 2 items of the COM-HL questionnaire identified as problematic due to comprehension problems were modified and no further problems arose in the subsequent interview phases. The number of items was reduced to create a shorter, yet still understandable, scale that covers all relevant aspects of COM-HL. The deleted items were mostly related to the evaluation or judgement of doctors’ recommendations that were not culturally acceptable. This could be related to various cultural factors, 9 such as the acceptance of medical paternalism10,11 which could also differ between generations, 12 the trust in health care providers, 13 and the beliefs about the causes of illness or the effectiveness of biomedical recommendations. 9 The present version of the questionnaire seemed to be understandable and acceptable based on the experience of the last round of the cognitive interviews.
To help respondents understand what is required of them, the introductory text mentions that they should answer based on their experiences with the doctor with whom they most often contact. Based on the WG's decision, there were no changes to the answer options, so no neutral or nonapplicable response category was included, but the respondents can skip the question if they cannot answer it.
We hope that—despite the language of the tested questionnaire—the lessons learnt from the interviews will be useful to other WGs wishing to develop or improve a tool.
Limitations
The sample size was relatively small, so the sample cannot be considered representative. This means that we may not have captured the full diversity of interpretations. However, we tried to include people with different sociodemographic characteristics in order to ensure diversity among the most important influencing factors and to be able to draw conclusions about the comprehensibility of the questionnaire and potential interpretation differences between subgroups. The participants participated only in one interview, so the modified items were not tested with the same person. We only tested one language version, which may affect the generalizability of the results. However, based on our experience, the problems we encountered were quite similar across the countries involved. This meant that we were able to contribute to the development of the final English version of the questionnaire.
Conclusion
The revised version of the questionnaire seemed to be comprehensible based on the experience of the cognitive interviews. Similar qualitative studies should be carried out to check the comprehensibility of other translations. The measurement properties should also be assessed across languages and populations to check the reliability and validity of the questionnaire—a tool which could also be useful for the improvement of patient experiences.
Footnotes
Acknowledgments
The authors would like to acknowledge the leaders and members of the Communicative Health Literacy Working Group of the HLS24 Project. The authors also thank the interviewees for their participation. The HLS19 instrument used in this research was developed within “HLS19—the International Health Literacy Population Survey 2019-2021” of M-POHL. Further information about using the HLS19 instruments can be found here: 
Ethical Considerations
The study was approved by the Regional and Institutional Ethics Committee University of Debrecen (first wave DE RKEB/IKEB 6810-2024; second wave DE RKEB/IKEB 6810A-2024).
Consent to Participate
In accordance with the Declaration of Helsinki, informed consent was obtained from all participants.
Author Contributions
Erika Szilágyi-Hornyák wrote the main manuscript text. Erika Szilágyi-Hornyák, Gabriella Nagy-Pénzes, and Gabriella Mátyás carried out the interviews. Erika Szilágyi-Hornyák prepared all tables. Éva Bíró and Róza Ádány contributed to the conception of the work and supervised the research. All authors contributed to interpreting the results. Éva Bíró and Róza Ádány reviewed the manuscript. All authors read and approved the final manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability
The data presented in this study are available on reasonable request from the corresponding author.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work was supported by the Ministry of Interior, Hungary (grant number BM/10449-2/2023). Preparation of this paper was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences (grant number BO/00933/22/5). The funders had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.