Sage Journals: Discover world-class research

Abstract

Digital technology offers several opportunities to improve access to professional expertise in primary care, and the offer of various “virtual” services has exploded in the past few years. The aim of this study was to evaluate the implementation of a direct-to-consumer on-line pharmacy consultation service (Ask Your Pharmacist - AYP) to a universal phone consultation service led by the universal public health system in Quebec (811 Info-Santé), through a direct bridge. Semi-structured interviews were conducted with clinician users of the service, and stakeholders involved in this pilot project funded by the Ministry of Economy (n = 22); documents were also analyzed, and content of the question was asked through the AYP service. Adoption of the service was low, and it suggests a poor alignment between the need and the service as implemented. Further research should investigate the mechanisms for an appropriate integration of digital services for primary care universal consultation services.

Introduction

Timely access to professional expertise in primary care is a challenge that was accentuated with the COVID-19 pandemic. Digital technology offers several opportunities to improve access to professional expertise, and the offer of various “virtual” services has exploded in the past 2 years.^1–6 Several ways of interacting at a distance with a healthcare professional have emerged in many jurisdictions, more of less integrated with the continuum of universal primary care services.^3,7‐9 In Canada, the phone has remained the main technological modality for interacting with a nurse or a physician, either through populational phone services such as 811 hotlines or through a direct access to a clinician.^6,10‐12 For example, a survey with physicians across Canada reported that only 2% of practices were supporting electronic communication with their patients in 2021.¹⁰ Similarly, 94% of physicians reported offering virtual care to their patients, while 93% were conducting the consultation by the phone.¹⁰ On the other hand, a vast array of private virtual teleconsultation services are available directly to patients (direct-to-consumer services), including with a physician, a nurse practitioner, and a pharmacist.^13,14 Pharmacists are presented as primary care professionals easily accessible (due to long opening hours) and are developing on-line services for professional counselling, either through commercial chain programs (e.g. Loblaws’ PC Health/Maple) or through dedicated on-line services (e.g. askyourpharmacist.ca).^15,16 The level of satisfaction of patients using this type of services is reported to be high, and the business model is based on commercial attractivity and high patient needs.¹⁶ The aim of this study was to evaluate the implementation of a direct-to-consumer on-line pharmacy consultation service (Ask Your Pharmacist) to a universal phone consultation service led by the universal public health system in Quebec, through a direct bridge. It followed a logic analysis that is publicly available.¹⁷

Methods

Setting

Since 1995, the 811 Info-Santé service has been deployed through a single number (referred to as 811) in order to offer quick and direct telephone access to professional expertise, 24 hours a day, 365 days a year, at no cost to users.¹⁸ Call handlers are nurses, and they should evaluate the person’s health status, make recommendations based on this evaluation, or direct the person to the most appropriate resource. Since 2015, the Ask Your Pharmacist (AYP) is available as a direct-to-consumer service available on-line. It allows citizens to communicate in an asynchronous way with a community pharmacist through a two-way messaging system, by asking a question to and receiving an answer from a participating pharmacist located close to the citizen’s place of residence. The answers to the questions may be publicly disclosed by the pharmacists, at their discretion, and become available on-line. In May 2021, there were close to 15,000 public answers accessible on AYP.

A pilot project creating an on-line bridge between Info-Santé 811 and AYP was developed and implemented in three 811 call centres from September 2020 to April 2021. The 811 nurses could redirect the calls requiring a pharmacist’s expertise to the AYP platform by directly sending the question to the pharmacists using a dedicated interface (Supplemental Figure 1). A link would be sent to the patient to register to the AYP service by e-mail, and the question would be dispatched to participating pharmacists using the AYP service. The patient would be contacted by a pharmacist to answer his/her question by e-mail. An incentive of $10 per 811-referred question answered would be offered to pharmacists. A detailed description of the service and implementation is available in French.¹⁷

Design

A descriptive evaluative study of the implementation of the 811-AYP bridge was conducted. Data sources included interviews with users and stakeholders, descriptive analysis of calls and usage of the bridge, and descriptive analysis of the content of questions asked by patients through AYP and 811-AYP bridge. This research project received approval by the research ethics committee of the CHUM and the CISSS de la Montérégie-Centre, and informed consent of participants was obtained prior to their participation. An analytic framework adapted from the model of DeLone & McLean has been used to guide this evaluation (Supplemental Figure 2).^19‐21

Data sources

Semi-structured interviews and documents

Semi-structured interviews were conducted with targeted users of the service (811 call centre nurses) and pharmacists participating in the AYP service. All targeted users were informed of the study, either by their manager for nurses (n = 90) or by the service promoter for pharmacists (n = 306). The interview guide was developed based on the analytic framework and covered the following themes: implementation (resources, training, and tools); use of the platform (number of consultations, types of questions asked); user experience (satisfaction); and the perception of its added value, facilitating and limiting factors for use and added value. Through snowball sampling, the stakeholders involved in the pilot project were also solicited, starting with the promoter. Twenty-two interviews were realized with 811 Info-Santé nurses and managers, designer and pharmacists of AYP, and health ministry managers (Supplemental Table 1). The documents related to the implementation were analyzed, including training sessions, communications, feedback between the promoter and the management team, and recruitment documents. All data from interviews and documents were extracted, organized by themes, and analyzed to describe convergent and divergent perceptions, per dimension of the framework.

Usage of AYP

All public questions asked through the AYP platform where analyzed by comparing the questions referred by the nurse through the bridge (n = 47), to the question asked by patients on-line directly (n = 5,517). Descriptive data included the date, day of the week, the time, and the period (morning, afternoon, evening, and night) of the question asked or referred, the category of question based on the groupings proposed by AYP, the time taken to answer the question in minutes (counted from the initial discussion) as well as the content of the question by frequency of words (clouds). Duplicated questions were manually identified. To create the word cloud, questions in English were translated into French (n = 86) and then standardized withdrawing “empty words” (e.g. determiners and conjunction, symbols, punctuation). The word cloud was generated using words with the highest frequency of occurrence only.^22,23 Analyses related to each data source were combined to describe the intervention and the implementation, including the experience of users, and to identify key facilitating and limiting factors. Key findings were developed and results were shared with participants and stakeholders to increase the credibility of interpretations. The comments collected were then added to the analysis.

Results

Adoption of the 811-AYP bridge

Adoption of the 811-AYP bridge was low among nurses in the three call centres, and the trend in usage shows a decrease over time during the pilot project (Figure 1). In total, 116 questions were referred to AYP by the nurses in the three 811 call centres during the study period, while the AYP platform received 9,023 on-line questions.

Figure 1.

Adoption of the 811-AYP bridge by nurses: number of questions referred by 811 nurses to AYP through the on-line bridge, by month, in the three participating 811 call centres (N total = 116).

Accordingly, only 1.3% (116/9,023) of the questions answered by pharmacists on AYP were referred from 811 call references. This observation was confirmed by interviews with targeted users. Most nurses were not using the tool in a routine manner. Interviewees reported frequency of use of between 1 and 8 times/person among evening nurses and between 0 and 4 times/person among day nurses interviewed.

“... from 8-9 o'clock (pm) the pharmacy is closed while this questionnaire (AYP) is available all the time. When it is open I refer people to their own pharmacist.” (Nurse2)

“Working during the day, as I explain, the resource [pharmacy] is always available unless it is a holiday. For me, it's more the evening colleagues who used it [the bridge]...” (Nurse8)

This observation is confirmed by detailed analysis of usage data, where most questions were referred to pharmacists during the night period (midnight until 6 am) (Supplemental Figure 3). Detailed description could be obtained for 47 of the 116 questions referred by 811 to AYP and for 5,517 of the 9,023 questions received on the AYP platform (these questions were made public by the pharmacist). The questions asked on-line directly by patients seemed broader in scope and less drug-specific than the questions referred to AYP by the 811 nurses (Figure 2). In total, 94% of the 47 questions referred from 811 to AYP were related to a medication, aligned with the reference criteria for nurses.

Figure 2.

Word cloud of questions received on the AYP platform; submitted by 811 nurses (A; N = 47) and submitted on-line by patients directly (B; N = 5,517).

User experience

The user experience was described based on the clinician’s perception on the ease of use of the system and the overall satisfaction with using the interface. Generally, the nurses’ and pharmacists’ experience was positive, although some improvements were suggested. The majority of pharmacist and nurse users at the Montérégie-Centre call centre reported that the system was easy to use. However, given the low adoption, this observation need to be interpreted with caution. Many areas for improvement were mentioned, especially by non-users, such as the length of the field for writing the question (too short, which limited them in developing the question); the need to copying and pasting fields on the patients’ demographic information from their local system to the AYP bridge; and missing or difficult information to capture (e.g. age below 14, English-speaking patient). Moreover, several nurses complained that they did not know if the referred user had opened their e-mail and signed up for AYP to get the answer to their question. Several e-mail entry errors were noted at the beginning of the project, which led to the addition of an e-mail verification feature. Nurses also reported concerns about usability on the patient’s side, potentially impeding their adoption of the tool.

“I also think that there are patients who are limited. At one point I explained to a lady and she agreed that I should switch to the platform, I took the necessary steps and then saw that she had to go on the Internet to accept the invitation so that afterwards the pharmacist could send her the answer I don’t know how far we’ve come; I’m not sure she understood.” (Nurse7)

Perceived usefulness

The majority of nurses interviewed appreciated the addition of a technology to improve access to pharmacists’ expertise for 811 clients. They recognize the usefulness of 24/7 pharmaceutical expertise for patients.

“It is very good for drug interaction; we want to ensure that the patient’s need is met when the pharmacy is closed.” (Nurse1)

“Yes, I am satisfied. The value added is when the client’s pharmacy is not available, really especially outside business hours.” (Nurse5)

However, they had more trouble articulating the added value for their work. None of the nurses mentioned that it decreased the time spent answering drug therapy questions or that it helped them answer drug therapy questions. Several mentioned uncertainty about response time as a limitation. Some mentioned that the time required to explain to patients and then complete the referral was a challenge (decreased productivity).

“It’s taking the time to get in and fill out the form but but it’s not so long, just a matter of copy/paste. Maybe that’s what will stop some girls from doing it, (...) But we have calculated that a call takes 15 minutes but if I take 3-4 minutes afterwards to complete the form, it is our statistics at the end there.” (Nurse7)

“It’s a waste of time in terms of writing a call sheet and the government wants us to make as many calls as possible; already with COVID, we send e-mails to the testing centre and our calls are getting longer and longer.... our call volumes are going to go down and then we’re going to get hit over the head, so no, it’s really not very practical.” (Nurse8)

Overall, the low adoption of the service confirms this perception of limited usefulness for the nurses. According to the nurses interviewed, the following barriers were identified to explain the low adoption and use of the 811-AYP bridge by nurses (Table 1).

Table 1.

Barriers to the adoption of the 811-AYP bridge by nurses—selected quotes.

Barriers	Details	Quotes
Accessibility to community pharmacy is already good	Nurses prefer to refer users to their pharmacist, who already knows them, especially for multi-medicated users.	“The idea is really good but in the evening, at night it is more or less useful ... then even during the day I feel that the girls must refer to the pharmacy directly.” (Nurse4) “During the day, if I have so require, I will refer to the local pharmacist. I have not referred to the telepharmacy centre for the time being, so I don’t have much need for the project because the local pharmacists are available.” (Nurse8)
Low need for medication-related support for nurses	Nurses are able to manage most medication-related calls with their current resources.	“You have to have a relevant call to send to the pharmacist. It took me a long time to use it because I didn’t necessarily need it.” (Nurse7) “Well I am able to manage my calls without needing an assistant. No, it’s not a need for me unless it becomes an obligation instead of a pharmacy.” (Nurse8)
The response time via AYP is too long	Urgent medication-related questions are often referred to the poison control centre or the emergency room. On the other hand, patients who want an over-the-counter medication do not want to wait several hours (response time via AYP) to get an answer.	“Of course, we can’t send urgent questions because we never know the response time. ” (Nurse3) “Nurses often need a quick response to medication-related questions. The best model would be an on-call pharmacist.” (Nurse researcher)
The preferences or abilities of the patients	Patients often prefer to call their pharmacist, and some patients do not have e-mail or Internet access.	“Of course, it’s an e-mail service and we at Info-Santé have a large percentage of people who are seniors and I would say that about 50% of seniors do not have e-mail .” (Nurse6)

Discussion

This study evaluated the implementation of a pilot project on the integration of an on-line pharmacy consultation service (Ask Your Pharmacist) to a universal phone consultation service led by the universal public health system in Quebec (811/Info-Santé), funded at the beginning of the pandemic. This type of service is aligned with an evolving role of community pharmacists for minor ailment or urgent care consultations in some jurisdictions, such as the United Kingdom and Australia, especially since the pandemic onset.^24–26

Our results suggested a limited adoption of the 811-AYP bridge by 811 nurses (call handlers), in relation to a perceived low added value of the technology, both for themselves, but also for patients. While the need for pharmaceutical expertise was expressed by nurses, the AYP bridge was not clearly complementing the existing services with community pharmacies (e.g. closed during the night). Similarly, the time required to complete the referral by nurses, in addition to the call form, seemed to have hindered the adoption, in a context of high pressure on resources due to the onset of the pandemic. Moreover, potential benefits for patients was identified (with a new way of accessing pharmaceutical expertise through asynchronous on-line communication), but the access mode via a bridge with 811 was not easily realizing this potential according to nurses. Moreover, the service was already accessible on-line for patients (without requiring a referral by 811 nurses). Our results contrast with the observation on the referral by the NHS-111 line call handlers to community pharmacy services, implemented in 2017 in a pilot region of North East of England.²⁷ In their study, Nazar and colleagues observed more than 13,000 calls referred by call handlers (who are not healthcare professionals) to community pharmacists (denominator unknown to calculate referral rate) in a 1-year period. Call handlers from the NHS-111 clearly articulated the added value, for patients and for the healthcare system, of integrating a referral to community pharmacists, as an in-person visit at the pharmacy. However, no digital technology was involved for the referral service, other than a computerized decision support tool for call handlers.

Our results reinforce the importance of a global policy vision for prioritizing digital services development and implementation in primary care, for an appropriate and rationale use of the technology, as suggested by the World Health Organization (WHO), through its global digital health strategy.^28,29 Our results suggest that an on-line trajectory (from the call centre to the pharmacy) might not be a priority for the primary care universal healthcare system. Similarly, a recent study of Canadian pharmacists’ perception of telepharmacy services (such as the AYP service) suggested a low adoption of this type of service on the pharmacy side; organizational barriers were identified such as cost-related barriers and interoperability barriers (integration between systems or devices).³⁰ Further research should focus on identifying success factors for an added value of teleconsultation services in primary care, both on the pharmacy side and the patient/citizen side.

The overall pandemic environment appears to have hindered the adoption of 811-AYP bridge, given the significant pressure that this situation placed on 811 nurses. This pressure appears to be related to a constant change in the information to be integrated by nurses in relation to COVID (e.g. new training), and an increase in call volume in the context of limited resources, and productivity targets for nurses. On the other hand, enthusiasm for a technology might have fostered nurses’ interest, without necessarily translating into adoption of the 811-AYP bridge.

Some methodological limitations must be stated. The implementation analysis was conducted in only one call centre, out of three centres participating in the pilot project, which did not allow for detailed analysis of organizational factors. Likewise, the data on questions answered by AYP included only those questions that were made public, that is, about 40% (47/116) of the questions transferred via the gateway and 61% (5,517/9,023) of the questions asked directly on-line by a citizen. However, this confirms the interviews conducted with a tendency for questions to be transferred mostly at night and very focused on medications compared to the on-line questions. A comparative analysis by conducting interviews with users of another call centre would have allowed for further analysis. It is important to note that saturation was not achieved for some of the aspects studied, including usability and perceived usefulness. However, only convergent results were presented. Similarly, the patient perspective was not studied directly but through the perceptions of professionals.

In conclusion, this evaluation emphasizes the potential of a technological mode for virtual access to pharmacists’ professional expertise. However, future studies should focus on better understanding the needs of patients and professionals, to support the development of digital health services that are aligned with local needs and practices.

Supplemental Material

Supplemental Material - Evaluating the implementation of a referral system for virtual pharmacy counselling in a province-wide nurse phone line

Supplemental Material for Evaluating the implementation of a referral system for virtual pharmacy counselling in a province-wide nurse phone line by Aude Motulsky, Lara Gautier, Marie-Pierre Moreault, Janine Badr, Man Qing Liang, Amirav Davy, Arnaud Duhoux and Marie-Thérèse Lussier in Healthcare Management Forum

Footnotes

Acknowledgements

The team would like to express its sincere thanks to Alexandre Chagnon, from AYP, for his collaboration and to all the participants in the pilot project and in this project. We also wish to thank Ms. Lise Bertrand of the Ministère de la santé et des services sociaux for her support, extensive availability, and collaboration.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Ministère de l’économie et de l’innovation du Québec through the Innovation fund. Aude Motulsky is supported by a salary award from the Fonds de recherche du Québec - Santé.

Ethical approval

The study received research ethics board approval from CHUM and CISSS de la Montérégie-Centre ethical committees.

ORCID iD

Aude Motulsky

Supplemental material

Supplemental material for this article is available on-line.

References

Fagherazzi

Goetzinger

Rashid

Aguayo

Huiart

. Digital health strategies to fight COVID-19 worldwide: Challenges, recommendations, and a call for papers. J Med Internet Res. 2020;22(6):e19284.

Howard

Health JBS of P . Digital Solutions for COVID-19 Response. https://www.jhsph.edu/departments/international-health/news/johns-hopkins-researchers-publish-assessment-of-digital-solutions-for-covid-19-response-in-low-and-middle-income-countries.html. Accessed October 8, 2020.

Hollander

Carr

. Virtually perfect? Telemedicine for Covid-19. N Engl J Med. 2020;0(0):null.

Sun

Zhang

Gwizdka

Trace

. Consumer evaluation of the quality of online health information: Systematic literature review of relevant criteria and indicators. J Med Internet Res. 2019;21(5):e12522.

Nikiema

Stringer

Moreault

, et al. Assessing the quality of direct-to-consumer teleconsultation services in Canada. Stud Health Technol Inform. 2022;294:935-936. doi:10.3233/SHTI220628.

Breton

Deville-Stoetzel

Haggerty

Lussier

Kaczorowski

Motulsky

Paré

et al. Snapshot of the transition towards Teleconsultation in teaching primary healthcare clinics in Québec during COVID-19. HealthCare Policy; 17:1.

Dialogue . Dialogue - Le plus grand fournisseur de soins virtuels du Canada. https://www.dialogue.co/fr/. Accessed May 18, 2021.

Medisys . Medical innovation and experts | TELUS health care centres. TELUS. https://www.telus.com/en/health/care-centres. Published September 13, 2018. Accessed May 18, 2021.

Lai

Wittbold

Dadabhoy

, et al. Digital triage: Novel strategies for population health management in response to the COVID-19 pandemic. Healthc Amst Neth. 2020;8(4):100493.

10.

Inforoute Santé du Canada / Association médicale canadienne . Sondage national des médecins du Canada 2021. https://www.infoway-inforoute.ca/fr/component/edocman/ressources/rapports/evaluation-des-avantages/3937-sondage-national-des-medecins-du-canada-2021. Accessed August 24, 2021.

11.

Krist

Beasley

Crosson

, et al. Electronic health record functionality needed to better support primary care. J Am Med Inform Assoc JAMIA. 2014;21(5):764-771.

12.

Rahal

Mercer

Kuziemsky

Yaya

. Factors affecting the mature use of electronic medical records by primary care physicians: A systematic review. BMC Med Inform Decis Mak. 2021;21(1):67.

13.

Falk

. The state of virtual care in Canada as of wave three of the COVID-19 pandemic: an early diagnostique and policy recommendations. Health Canada; 2021. https://epe.lac-bac.gc.ca/100/201/301/weekly_acquisitions_list-ef/2021/21-38/publications.gc.ca/collections/collection_2021/sc-hc/H22-4-26-2021-fra.pdf. Accessed June 14, 2022.

14.

Gray

Mason

Loshak

. An overview of direct-to-patient virtual visits in Canada. Can J Health Technol. 2021;1(6). doi:10.51731/cjht.2021.80.

15.

Motulsky

Weir

Liang

, et al. Patient-initiated consultations in community pharmacies. Res Soc Adm Pharm. 2021;17(2):428-440.

16.

Guénette

Chagnon

Turcotte

. User experience with the ask your pharmacist teleconsultation platform. Can Pharm J Rev Pharm Can. 2021;154(3):213-221.

17.

Motulsky

Moreault

Badr

, et al. Soutien aux infirmières du 811 et autres équipes de soins par la télépharmacie - une évaluation; 2021:37p.

18.

Faucher-Paré

. Info-Social (8-1-1): description d’un service provincial d’intervention psychosociale téléphonique de première ligne. Published online 2019:8.

19.

Lau

Price

Keshavjee

. From benefits evaluation to clinical adoption: making sense of health information system success in Canada. Heal Q Tor Ont. 2011;14:39-45.

20.

Urbach

Müller

. The updated DeLone and McLean model of information systems success. In: Dwivedi

Wade

Schneberger

, eds. Information Systems Theory: Explaining and Predicting Our Digital Society, Vol. 1. Integrated Series in Information Systems. Springer; 2012:1-18.

21.

Greenhalgh

Abimbola

. The NASSS framework – A synthesis of multiple theories of technology implementation. Stud Health Technol Inf. 2019;263:193-204.

22.

Bagrat Stepanyan Armin . Using TF—IDF and Cosine Similarity to Identify Similar News Articles in R-Programming. https://arminbagrat.com/NewsSimilarityTutorial/. Published 2017. Accessed May 19, 2021.

23.

Vembunarayanan Jana . Tf-Idf and Cosine similarity. Seeking Wisdom. https://janav.wordpress.com/2013/10/27/tf-idf-and-cosine-similarity/. Published October 28, 2013. Accessed May 19, 2021.

24.

Dodds

Katusiime

Shamim

Fleming

Thomas

. An investigation into the number and nature of the urgent care consultations managed and referred by community pharmacists in South-East England. Prim Health Care Res Dev. 2020;21:e5.

25.

Dineen-Griffin

Garcia-Cardenas

Rogers

Williams

Benrimoj

. Evaluation of a Collaborative protocolized approach by community pharmacists and general medical practitioners for an Australian minor ailments scheme: Protocol for a cluster randomized controlled trial. JMIR Res Protoc. 2019;8(8): e13973 .

26.

Patterson

Cadogan

Barry

Hughes

. “It stayed there, front and centre”: perspectives on community pharmacy’s contribution to front-line healthcare services during the COVID-19 pandemic in Northern Ireland. BMJ Open. 2022;12(9): e064549 .

27.

Nazar

Evans

Kyei

, et al. A service evaluation and stakeholder perspectives of an innovative digital minor illness referral service from NHS 111 to community pharmacy. PloS One. 2020;15(3): e0230343 .

28.

WHO . Draft Global Strategy on Digital Health 2020-2024. World Health Organization. https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf?sfvrsn=f112ede5_38. Accessed May 11, 2020.

29.

Labrique

Agarwal

Tamrat

Mehl

. WHO Digital Health Guidelines: A milestone for global health. Npj Digit Med. 2020;3(1):1-3.

30.

Park

Zed

Vera

MAD

. Perspectives and experiences with telepharmacy among pharmacists in Canada: A cross-sectional survey. Pharm Pract. 2022;20(1):2609.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB