Sage Journals: Discover world-class research

Abstract

Background:

Effective benefit–risk communication to healthcare professionals plays a vital role in promoting the rational use of medicinal products. The Health Sciences Authority (HSA) of Singapore has implemented Physician Educational Materials (PEMs) and Patient Medication Guides (PMGs) as educational materials to support informed decision-making. To evaluate their effectiveness as a safety communication tool, we sought to understand prescribers’ experiences and preferences with these materials.

Methods:

Invitations to an online survey were emailed to 387 healthcare professionals who had recently purchased the 22 medicinal products with requirements for educational materials in Singapore (“Purchasers”). The survey scope included their experiences with the receipt and usage of the materials, and their preferences towards the types of content and distribution channels for these materials.

Results:

The survey invitation reached 367 purchasers, of which 89 (24%) responded, including 66 who indicated that they had prescribed the surveyed products (“Prescribers”). Although only 47% of prescribers recalled receiving the educational materials, the majority who had used them found them useful in their clinical practice. However, the PMG was used less frequently than the PEM (59% vs 96%), and there was a mismatch between current and preferred distribution methods (mostly hard copies vs electronic copies).

Conclusions:

Prescribers who had received educational materials generally perceived them as being useful in raising awareness on treatment-associated safety concerns. The publication of educational materials on the HSA website and encouraging more routine distribution of PMGs by prescribers may improve their accessibility and facilitate safety communication on medicinal products to healthcare professionals and patients.

Keywords

Benefit–risk communication educational materials medical education patient safety risk communication strategies

Introduction

In the rapidly changing and complex information environment of medicine and public health, effective benefit–risk communication on medicinal products is essential for supporting healthcare professionals in their clinical practices and decision-making. The provision of timely, evidence-based information on the safe and effective use of medicinal products, therefore, positively influences behaviour towards minimising risk and contributes to the protection of patient and public health.^1–3 It may also promote patient adherence to treatment when healthcare professionals are able to explain the risk-related issues clearly, thereby ensuring adequate understanding and accurate perception by patients on the benefits and risks of their chosen treatment.⁴

To meet the information needs of healthcare professionals and patients, health regulatory authorities in the European Union and US have incorporated safety communication tools as part of a risk management plan (RMP).^5,6 These include educational materials to supplement the information in the product labels by focusing on specific safety concerns and highlighting pertinent actions to be taken to minimise patient harm.⁷ In line with the RMP requirements implemented overseas, the Health Sciences Authority (HSA) of Singapore has, since 2008, required pharmaceutical companies to produce and distribute educational materials for medicinal products with significant safety issues identified at the pre- or post-marketing phases. These materials, in the form of Physician Educational Materials (PEMs) or Patient Medication Guides (PMGs), aimed to communicate important treatment-associated adverse effects and risk minimisation advice to healthcare professionals and patients.⁸ Upon HSA’s approval of the content, the pharmaceutical companies would disseminate the educational materials to healthcare professionals supplied with the products, who would, in turn, distribute the PMGs to their patients.

It is important to evaluate the effectiveness of the implemented educational materials where possible, taking into consideration whether the intended messages were received (and understood) by the target audience, and their impact on behaviour.⁵ To achieve this, HSA has conducted compliance checks of the companies’ distribution lists to determine if the educational materials had been provided to the intended recipients. We also regularly monitored trends in adverse events reported to the national database as a gauge of the appropriate use of medicinal products. Beyond these standard regulatory measures, healthcare professionals’ receptiveness to educational materials should also be evaluated since they are the key players in patient management and the intended primary users of these materials. In recent years, efforts have been made by overseas regulators to assess if targeted educational materials increased healthcare professionals’ awareness to important clinical safety information,^9,10 and whether the method of risk communication affected their uptake of the safety information and decisions to act on this knowledge.¹¹

To further assess and improve the effectiveness of current benefit–risk communication practices to healthcare professionals, we conducted a survey to find out prescribers’ experiences with the receipt and usage of educational materials required by HSA. Our survey also aimed to evaluate prescribers’ perception of the usefulness of the materials, and to examine their preferences towards the types of content and distribution channels for these materials.

Methods

Survey design

The scope of the survey covered the respondents’ receipt and usage of educational materials for products that they had prescribed, their attitudes towards the usefulness and content of the materials, current and preferred distribution channels, and clinical practice characteristics. Within the survey, some questions were only applicable to prescribers of the surveyed products who had received educational materials, whereas other questions were applicable to all respondents (Figure 1). To increase the response rate, the survey was designed to be completed within 5–10 minutes, where branching logic and conditional visibility logic were used to display only questions relevant to the respondent.

Figure 1.

Flow diagram of survey questions.

Prescribers were asked to recall if they had received the educational materials for each product they prescribed and, if so, the distribution channel. To facilitate recall of the educational materials, the survey included hyperlinks to view the front covers of the materials. For each product selected, recipients of the educational materials were asked if they had used the materials in their clinical practice and whether they had found them useful. The prescribers’ perception of the usefulness of educational materials was assessed with statements rated on a five-point Likert scale ranging from “Strongly disagree” to “Strongly agree”. These statements are presented in the Results section.

All survey respondents were asked to rate their opinion on the preferred types of content for the educational materials. Details on the types of content are presented in the Results section. Responses were captured using a five-point Likert scale ranging from “Not important” to “Very important”. Survey respondents were also asked about their preferred distribution channel, general comments on the materials, place of practice and clinical practice experience.

Surveyed products

The survey included 22 medicinal products. They had been registered and marketed in Singapore prior to January 2017, comprising six and 16 products registered between 2004 and 2010, and 2011 and 2016, respectively. All but one were innovator (i.e. branded) products. The 22 products all had a requirement for PEMs and/or PMGs as part of their local RMP. Of these, two products had PEM only, five had PMG only and 15 had both PEM and PMG.

The products were grouped into two sets based on their indications. Survey Form 1 contained antineoplastic agents, immunosuppressants and haemostatics for cancers and immune-mediated blood disorders (n = 11), whereas Survey Form 2 contained antihypertensives, antiarrhythmics, urologicals, antidepressants and immunosuppressants for cardiovascular, sexual, depressive and autoimmune disorders (n = 11). Both survey forms had identical questions and options, apart from the products listed within them. This allowed us to maximise the number of products listed in each survey without overcomplicating the survey logic.

Study population and survey deployment

The target study population was intended to be recent prescribers of the surveyed products. As this information was not readily available, purchasers of the products served as a proxy. We requested companies of the surveyed products to provide us with their list of purchasers (and corresponding clinics/institutions) from 2016 to 2018. For purchasers whose direct contact information was unavailable from the companies, written requests were made to various public and private hospitals to obtain their email addresses.

The survey was conducted using an online platform over three periods. Period 1 was a pilot study involving 100 purchasers using Survey Form 1; Period 2 involved the remaining purchasers of products listed in Survey Form 1; and Period 3 involved all purchasers of products listed in Survey Form 2. The pilot study served as a feasibility study to determine if further refinements to the survey questions were necessary for added clarity. Survey invitations were emailed to the study population, with up to three reminder emails to non-respondents. Each invitation provided a common weblink and QR code to the survey, which was submitted anonymously. Participation in the survey was voluntary and consent was considered obtained by virtue of survey completion. To facilitate the collation of survey returns, the respondents were requested to inform us of their survey submission separately via the invitation email so that they would not receive any further reminder emails.

Data analyses

As no changes were made to the survey questions following the pilot study, data collected over the three periods were pooled and analysed using descriptive statistics. Summaries of frequencies and percentages of survey responses were reported where applicable.

Results

The survey was sent to 387 purchasers over three periods in October 2018, January 2019 and March 2019. After removing 20 purchasers with undeliverable email addresses, the effective sample size was 367 purchasers, resulting in an overall response rate of 24% (n = 89). The respondents’ profile is presented in Table 1. Overall, the majority worked in public hospitals (71%) and had over 10 years of clinical experience (69%). Among the respondents, only 66 (74%) reported that they had prescribed at least one of the surveyed products (median: 2; range: 1 to 9), while the remaining 23 (26%) respondents reported prescribing none of the surveyed products.

Table 1.

Profile of respondents (n = 89).

	n	%^a
Place of practice
Public hospital	63	71
Private hospital	12	14
Private specialist clinic	9	10
Family medicine/General Practitioner clinic	5	6
Years of clinical experience
< 5 years	7	8
5–10 years	21	24
11–20 years	22	25
>20 years	39	44
Number of products prescribed
0	23	26
1	26	29
2	12	14
3	11	12
4 or more	17	19
Prescriber status of respondents
Prescriber of at least one surveyed product	66	74
Non-prescriber of the surveyed products	23	26

Differences in percentages may exist due to rounding.

Receipt and usage of educational materials by prescribers

Overall, 31 out of 66 prescribers reported receiving educational materials of the surveyed products (i.e. receipt rate of 47%). The receipt rates for PEMs and PMGs were similar, whereby 43% and 45% of prescribers eligible to receive the materials reported receiving at least one PEM and one PMG, respectively (Figure 2). However, the reported usage of the materials differed between their respective recipients, with the PEM (96%) being preferred over the PMG (59%).

Figure 2.

Receipt and usage of educational materials by prescribers.

Educational materials for 16 products reached their prescribers, although the percentage of prescribers who reported receiving the materials varied across the different products, ranging from 13% (one of eight prescribers) to 100% (six of six prescribers). None of the prescribers reported receiving educational materials for the remaining six products (one with PEM only, two with PMG only and three with both PEM and PMG). Of these, three products had one prescriber each, whereas the remaining three products had three, four and nine prescribers.

Prescribers’ perception of PEMs

There were 24 prescribers who had received and read at least one PEM. The majority of them agreed that they would use the PEM as a quick reference guide when prescribing the product to new patients (63%), or when seeing patients who were assessed to be at higher risk of experiencing treatment-associated adverse effects (71%) (Figure 3). Eighteen (75%) prescribers shared that the PEM was useful for seeking specific information such as patient selection criteria and monitoring parameters. However, seven (29%) prescribers (of which six had over 20 years of clinical experience) found that the information in the PEM did not add value to their current knowledge. One of them commented that products with educational materials were generally “very subspecialised” and the physician would likely be “very familiar with the usage, pharmacokinetics, pharmacodynamics, adverse events etc.”, while another preferred “getting information from independent sources e.g. journals, conferences, UpToDate, Lexicomp etc.”.

Figure 3.

Prescribers’ agreement with statements about PEMs they had received and read (n = 24).

Having insufficient time to refer to the PEM did not appear to be a major issue for these prescribers, with 12 (50%) of them disagreeing with this statement, and another one-third remaining neutral. They also generally regarded the PEM as being educational and non-promotional, with 13 (54%) prescribers not associating the PEM with promotional materials developed by pharmaceutical companies, as compared to two (8%) prescribers who did.

Prescribers’ perception of PMGs

Among the 17 prescribers who had provided at least one PMG to their patients or used it in patient counselling, the majority (94%) considered the PMG to be a reliable source of concise information for their patients (Figure 4). The prescribers generally agreed that the PMG could serve as a useful tool, such as reminding patients about the safety concerns associated with the product, educating patients on steps to take to minimise adverse effects and providing advice on patient self-monitoring (88% each). Although the adverse effects of the product were highlighted in the PMG, only five (29%) prescribers were concerned that this might deter their patients from taking the product, whereas seven (41%) prescribers disagreed with this statement. Only four (24%) prescribers felt that providing the PMG as a take-home material was of limited usefulness for patients, as they considered verbal counselling on adverse effects during their consultation to be sufficient.

Figure 4.

Prescribers’ agreement with statements about PMGs they had received and used (n = 17).

Respondents’ opinions on educational material content

All 89 respondents (including non-prescribers and non-recipients of the educational materials) were surveyed on the content typically present in the local PEMs and PMGs. The majority of respondents, ranging from 80% to 96%, rated the various types of PEM content listed in the survey as “Important” or “Very important” (Figure 5). The respondents appeared to view the PEM as a source of safety information for the product, as the topics “summary of safety concerns”, and “diagnosis and/or management of adverse effects” were rated most highly in terms of importance.

Figure 5.

Respondents’ rating on importance of content for PEMs (n = 89).

For the PMG, 82% to 96% of the respondents considered the various types of content listed in the survey as “Important” or “Very important” (Figure 6). Similar to the PEM, the respondents viewed topics related to adverse effects as being most important for inclusion into the PMG – namely, “possible adverse effects during treatment” and “signs and symptoms that might require immediate medical attention”.

Figure 6.

Respondents’ rating on importance of content for PMGs (n = 89).

Distribution channels

Of the 31 prescribers who had received educational materials, 20 (65%) received them through only one distribution channel, while 10 (32%) reported two or more channels. One prescriber indicated “Unsure”. This resulted in 45 responses on current distribution channels, whereby the most common distribution method was hard copy provided by the company’s representative (47%). Other modes of distribution encountered were by mail, supplied together with the product, by email and by internal distribution within the healthcare institution (Figure 7).

Figure 7.

Current and preferred distribution channels for educational materials. Percentages are based on the number of responses as the respondents could receive educational materials through more than one distribution channel.

When all 89 respondents were asked about their preferred distribution channels for the educational materials (multiple responses were allowed), there was a trend towards electronic copy distribution, with 57 (64%) respondents preferring only electronic copies of educational materials. Out of the 143 responses received, the most preferred methods of distribution were through the HSA website or company’s email, while the least preferred was hard copy by mail (Figure 7).

Three respondents supported the online publication of educational materials, as this would make the materials more accessible and environmentally friendly. To further improve accessibility to the educational materials, one respondent suggested a search function on the HSA website, while another suggested posting the materials on the hospital’s intranet due to the limited availability of Internet access at work.

Discussion

Communicating benefit–risk information to healthcare professionals and patients can influence their behaviour and promote the rational, safe and effective use of medicinal products.¹² In order to develop successful communication strategies, health regulatory authorities would first need to understand what information the target audience requires and how best to convey usable information.^13,14 This study sought to understand prescribers’ current experience with educational materials from 22 products, as well as all survey respondents’ preferences on the educational material content and distribution channels. It is anticipated that the feedback received would facilitate HSA in refining its review criteria for educational materials and evaluating the adequacy of current risk communication channels.

Although comparable to published data,^10,15 the relatively low receipt rate (47%) of educational materials reported by the prescribers could reflect deficiencies in the distribution process and/or a recall bias. For example, educational materials delivered together with the products to the hospital pharmacy might not be distributed internally to the hospital-based prescribers. There could also be lapses in compliance by some pharmaceutical companies in distributing the most updated version of educational materials to all prescribers of each product. The time interval between distribution of the materials and the survey period, and the last version received, could possibly impact the prescribers’ recall. Prescribers who had previously received earlier versions might not recognise the front covers of the latest versions presented in the survey, especially if there had been extensive changes to the artwork.

Despite the similar receipt rates of PEMs and PMGs, prescribers were more receptive to using the PEM than PMG. This could partly be due to their perceived relevance of the materials. Prescribers were likely to identify better with the PEM since they were the target audience and could access the PEM at their own pace and time. In contrast, the less frequent usage of the PMG could possibly be due to time constraints in going through the materials with their patients, or potential concerns over their patients’ reactions to the risk information presented in the PMG.¹⁶ Some prescribers could also have opted to provide alternative patient educational materials instead of the PMG, such as in-house materials developed by their healthcare institutions. Other considerations affecting a physician’s decision to provide educational materials to their patients include the relevance of the materials to their patients and the trustworthiness of the source of information.¹⁷ This highlights the need for HSA to further engage healthcare professionals through roadshows or HSA’s news bulletin for healthcare professionals to raise their awareness to the availability and objectives of these educational materials. By explaining that take-home materials such as the PMG could help reinforce important information regarding treatment-associated adverse effects to patients, this might promote more routine use of PMGs.

It was encouraging to note that among prescribers who had received and used the educational materials, the majority had perceived these materials as being useful in their clinical practice. Users of PEMs tended to refer to them for specific information, such as patient selection criteria or patient monitoring parameters. This reinforced the role of the PEM as a supplement to the product labels by highlighting pertinent safety-related information beyond the standard prescribing information such as indications and dosing regimens, which prescribers would already be familiar with. For prescribers who found no added value from the PEM, most of them were highly experienced (with over 20 years of clinical experience) and were, therefore, less likely to benefit from the supplemental information in the PEM. Among the users of PMGs, there were some prescribers who were doubtful about their added value to verbal counselling. Considering that patients most frequently failed to recall potential medication adverse effects, despite having good short-term recall of information provided during their physician visit,¹⁸ the combination of both verbal counselling and provision of take-home materials for patients could work synergistically to improve information retention.¹⁹

The majority of the respondents agreed with the types of content to be included in the PEM and PMG. This was reassuring as it implied that the areas of importance that HSA had chosen to highlight in these materials were aligned to the needs of our local healthcare professionals. In particular, key safety concerns and possible adverse effects during treatment were rated most highly in terms of importance for inclusion into the PEM and PMG, respectively, thus reinforcing their role as a source of risk information for healthcare professionals and their patients.

A mismatch was seen between the most common distribution channel used currently (i.e. hard copy distribution by hand) and the respondents’ preferred channels (i.e. electronic copies on HSA’s website or through the company’s email). In response to this, HSA launched the online publication of educational materials on the HSA website in April 2020 to ease accessibility to up-to-date versions of these materials. This might also further encourage the use of these materials if they were perceived as coming from a trusted source (e.g. a regulatory authority) and clearly distinguished from company-produced promotional materials.²⁰ Other potential methods of electronic dissemination of educational materials are also currently being assessed for their feasibility, such as mobile apps or QR codes on point-of-sales materials.

Although our study focused on prescribers’ experiences and preferences regarding educational materials, we acknowledge that pharmacists also play an important role in effective patient education. The counselling and advice given by pharmacists complements and reinforces information provided by prescribers and other members of the healthcare team, thereby allowing patients to achieve maximum therapeutic benefit from their medications.²¹ The interaction between pharmacists, prescribers and PMGs in improving medication management could be explored in future studies.

Strengths and limitations

Our study had several strengths. To encourage survey participation and reduce the burden on potential respondents, the survey was designed to be easy and relatively quick to complete using any Internet-enabled devices, including mobile phones, which increased accessibility and convenience. Furthermore, we employed published methods pertaining to survey administration to increase the response rate, including personalisation of the survey using last names, use of multiple reminders and making minor changes to the wordings of reminder emails (e.g. subject header or opening paragraph) over the survey life cycle.²² We also attempted to broaden the scope of feedback received from all respondents (including non-recipients of educational materials) by including some general questions on their preferred educational material content, which facilitated our understanding of their needs.

There were several limitations inherent to the nature of the study. The response rate of 24% was considered fairly low, although comparable to other published literature.⁹ The low response rate might have biased our results in that the findings might not be representative of all prescribers. However, we were unable to measure if there were differences in the characteristics of respondents versus non-respondents as the survey return was anonymous. The study was also subject to recall bias, although we attempted to facilitate recall by providing hyperlinks for prescribers to view the front covers of the educational materials.

In addition, we had assumed that purchasers of the surveyed products were an appropriate proxy for actual prescribers. However, 26% of respondents reported having prescribed none of the surveyed products. The reasons for this are not clear, but it could be that the products were purchased on behalf of group practices and, therefore, the healthcare professionals identified as purchasers for this survey were not the actual prescribers of the products. Also, given that the list of purchasers provided by the companies spanned from 2016 to 2018, it could have possibly included purchasers who used to prescribe the surveyed products but were no longer doing so.

Conclusions

Prescribers in Singapore who had received and used educational materials generally perceived them as useful communication tools for raising awareness on important safety concerns associated with medicinal products. The lower usage of PMGs compared to PEMs indicated that additional efforts should be made to encourage prescribers to distribute PMGs to their patients for reinforcing knowledge on treatment-associated adverse effects beyond the prescriber’s office. Given the relatively low receipt rate, more effective dissemination methods for educational materials should be developed locally. This has guided the publication of educational materials on the HSA website recently. Gaining an understanding of prescribers’ (or other healthcare professionals’) experiences and preferences regarding educational materials could aid in tailoring the design and implementation of benefit–risk communication tools to increase their usability, thereby promoting patient safety and increasing public confidence in the regulatory system.

Footnotes

Acknowledgements

The authors would like to express their appreciation to the healthcare professionals who participated in the survey for sharing their feedback on educational materials required by HSA.

Authors’ contributions

PH, ML, CH and JP contributed to the study conception and design. PH, ML, LT and CH carried out the survey, interpreted the data and contributed to the drafting of this manuscript. JP edited the manuscript. All authors made substantive intellectual contributions to the study. All authors read and approved the final version of the manuscript.

Availability of data

The datasets generated and/or analysed during the study are available from the corresponding author.

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Ethical approval

The National Healthcare Group Domain Specific Review Board (NHG DSRB) does not require ethical approval for quality assessment and improvement studies that meet the criteria for exemption from DSRB review.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Health Sciences Authority, Singapore under the HSA Research and Innovation Seed Grant (grant number HSARSG2018002).

Informed consent

Participation in the survey was voluntary and consent was considered obtained by virtue of survey completion.

ORCID iD

Christine Su-Ching Ho

References

Way

Blazsin

Löfstedt

, et al. Pharmaceutical benefit-risk communication tools: a review of the literature. Drug Saf 2017; 40: 15–36.

Reber

Piening

Wieringa

, et al. When direct health-care professional communications have an impact on inappropriate and unsafe use of medicines. Clin Pharmacol Ther 2013; 93: 360–365.

European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Module XV – Safety communication (Rev 1), https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-xv-safety-communication-rev-1_en.pdf (2017, accessed 19 December 2019).

Naik

Ahmed

Edwards

AGK

. Communicating risk to patients and the public. Br J Gen Pract 2012; 62: 213–216.

European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Module XVI – Risk minimisation measures: selection of tools and effectiveness indicators (Rev 2), https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-xvi-risk-minimisation-measures-selection-tools_en-3.pdf (2017, accessed 19 December 2019).

U.S. Food & Drug Administration. What’s in a REMS, https://www.fda.gov/drugs/risk-evaluation-and-mitigation-strategies-rems/whats-rems (2018, accessed 19 December 2019).

European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Module XVI Addendum I – Educational materials, https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-good-pharmacovigilance-practices-gvp-module-xvi-addendum-i-educational-materials_en.pdf (2015, accessed 19 December 2019).

Health Sciences Authority, Singapore. Guidance for industry: post-marketing vigilance requirements for therapeutic products, https://www.hsa.gov.sg/docs/default-source/hprg/therapeutic-products/guidance-documents/guidance-for-industry_post-marketing-vigilance-requirements-for-therapeutic-products-v2_01-apr-2020_final—v2.pdf (2020, accessed 10 April 2020).

Bester

Vito-Smith

McGarry

, et al. The effectiveness of an educational brochure as a risk minimization activity to communicate important rare adverse events to health-care professionals. Adv Ther 2016; 33: 167–177.

10.

Davis

Asiimwe

Zografos

, et al. Evaluation of risk-minimization activities for cyproterone acetate 2 mg/ethinylestradiol 35µg: a cross-sectional physician survey. Pharm Med 2017; 31: 339–351.

11.

Piening

Haaijer-Ruskamp

de Graeff

, et al. Healthcare professionals’ self-reported experiences and preferences related to Direct Healthcare Professional Communications: a survey conducted in the Netherlands. Drug Saf 2012; 35: 1061–1072.

12.

Council for International Organizations of Medical Sciences (CIOMS). Practical approaches to risk minimisation for medicinal products: report of CIOMS Working Group IX. Geneva: World Health Organization, 2014.

13.

U.S. Food & Drug Administration. FDA strategic plan for risk communication and health literacy 2017–2019, https://www.fda.gov/media/108318/download (2017, accessed 19 December 2019).

14.

European Medicines Agency. Benefit-risk communication to medicine users, https://www.ema.europa.eu/en/documents/report/workshop-report-benefit-risk-communication-medicines-users-how-can-regulators-best-meet-information_en.pdf (2014, accessed 19 December 2019).

15.

Brody

Liss

Wray

, et al. Effectiveness of a risk-minimization activity involving physician education on metabolic monitoring of patients receiving quetiapine: results from two postauthorization safety studies. Int Clin Psychopharmacol 2016; 31: 34–41.

16.

Herber

Gies

Schwappach

, et al. Patient information leaflets: informing or frightening? A focus group study exploring patients’ emotional reactions and subsequent behavior towards package leaflets of commonly prescribed medications in family practices. BMC Fam Prac 2014; 15: 163–170.

17.

Quinn

Vadaparampil

Malo

, et al. Oncologists’ use of patient educational materials about cancer and fertility preservation. Psychooncology 2012; 21: 1244–1249.

18.

Tarn

Flocke

SA.

New prescriptions: how well do patients remember important information?

Fam Med 2011; 43: 254–259.

19.

Curtis

Mullen

Russell

, et al. An efficacy trial of an electronic health record-based strategy to inform patients on safe medication use: the role of written and spoken communication. Patient Edu Couns 2016; 99: 1489–1495.

20.

de Vries

van der Sar

MJM

Cupelli

, et al. Communication on safety of medicines in Europe: current practices and general practitioners’ awareness and preferences. Drug Saf 2017; 40: 729–742.

21.

Sanii

Torkamandi

Gholami

, et al. Role of pharmacist counselling in pharmacotherapy quality improvement. J Res Pharm Pract 2016; 5: 132–137.

22.

Sauermann

Roach

Increasing web survey response rates in innovation research: an experimental study of static and dynamic contact design features. Res Policy 2013; 42: 273–286.