Accuracy of Inpatient Recall of Interaction With a Pharmacist: A Validation Study From 2 Acute Care Teaching Hospitals

Abstract

Background:

Research has shown that inpatients may not accurately report interacting with a pharmacist.

Objective:

To determine accuracy of patients’ recollection of meeting with a pharmacist at 2 acute care teaching hospitals in Edmonton, Alberta, Canada.

Methods:

Retrospective review of 391 surveyed patients discharged from April 2013 to March 2014. Responses to meeting a pharmacist (yes/no) were compared with 2 reference standards: pharmacist documentation in patient charts and pharmacist clinical workload data. Sensitivity, specificity, positive predictive, and negative predictive values were calculated.

Results:

One hundred ninety-five (49.9%) respondents reported meeting with a pharmacist. Of these, 71 (36.4%) had corresponding pharmacist chart documentation. Of the 196 respondents who reported not speaking with a pharmacist, 73 (37.2%) had documentation present. Compared with patient charts, sensitivity and specificity were 49.3% and 49.8%, respectively. Positive and negative predictive values were 36.4% and 62.8%, respectively. Similar results were seen in comparison with the workload data.

Conclusions:

Patients often inaccurately reported meeting with a pharmacist in the acute care setting. The results are useful for pharmacist training, patient education, and for refinement of the current survey question.

Keywords

inpatient survey patient recall pharmacist interaction communication

Introduction

Pharmacists’ interventions have achieved positive health outcomes for patients in various areas including smoking cessation, dyslipidemia, hypertension, diabetes, and asthma (1). According to the 2012 Ipsos Reid survey, pharmacists were ranked as one of the most trusted professionals in Canada (2). Patient evaluation of pharmacist services is essential to maintain a high quality of practice as well as to identify areas for improvement. The inpatient hospital experience is one important performance measure for Pharmacy Services.

In 2008, the Canadian Society of Hospital Pharmacists (CSHP) launched “CSHP 2015: Targeting Excellence in Pharmacy Practice,” a national initiative consisting of 6 goals and 36 pharmacy practice-related objectives to enhance patients’ medication-related outcomes (3). One of these objectives, objective 1.5, stated: “50% of recently hospitalized patients or their caregivers will recall speaking with a pharmacist while in the hospital” (3). According to the 2013 to 2014 Lilly Hospital Pharmacy in Canada Survey, 28 of 124 surveyed sites reported including a question about patient recall of pharmacist interaction in their satisfaction surveys; however, only 4% of the respondents reported achieving this CSHP target (4).

Within Alberta Health Services (AHS), a modified version of the Hospital-Consumer Assessment of Healthcare Providers and Systems (H-CAHPS) survey has been used to measure inpatient experience (5 –7) on a provincial basis since April 2011. This survey is conducted within 6 weeks of patient discharge from 1 of 94 acute care inpatient facilities and targets adult inpatients (≥18 years of age at time of discharge; 5 –7). The survey consists of 51 questions, which includes the 32 core H-CAHPS questions, as well as additional items deemed important from an organizational perspective. Two of these items pertain to Pharmacy Services. Patients are first asked, “During your hospital stay, did you meet with a pharmacist?” and if they answer “yes,” they are then asked “We want to know your rating of the overall contact you had with pharmacists during your hospital stay. Using a number from 0 to 10, where 0 is the worst possible contact and 10 is the best possible contact, what number would you give the contact you had with the pharmacists who treated you?” (5 –7).

Since 2011, the results from these pharmacy-specific survey questions have been quite consistent on a provincial basis. Across Alberta, 15% to 20% of patients surveyed typically recalled speaking with a pharmacist during their hospital admission. Of the respondents, 85% to 90% rated their interaction with pharmacists as 8, 9, or 10 out of 10 (8). Although the nature of the interaction was not surveyed, possibilities include medication review/history, patient assessment, medication counseling, and patient education. However, before the survey results can be interpreted, and to identify whether there is value in continuing to include the current Pharmacy Services questions in our modified H-CAHPS survey, validation is integral. Furthermore, the validated results will guide Pharmacy Services in interpreting this performance indicator information and will identify areas for potential service improvement.

The primary objective of this project was to determine the accuracy of patient responses to the survey question, when compared with pharmacist documentation in (a) patient charts and (b) pharmacist clinical workload data. A secondary objective was to determine whether the accuracy of patient recall varies according to patient characteristics such as age and length of hospital stay.

Methods

Study Design

This study was a retrospective chart review of 391 patients who were discharged from 1 of 2 acute care hospitals in Edmonton, Alberta, Canada, between April 2013 and March 2014, and who subsequently completed a telephone survey regarding their hospital experience. At the time of the study, there were approximately 1525 acute care beds between the 2 urban hospitals and each hospital deployed pharmacists to various inpatient areas including cardiology, emergency, gastroenterology, hematology, intensive care, medicine, neonatology, nephrology, neurology, orthopedics, pediatrics, psychiatry, pulmonology, surgery, and transplant (9). The study was approved by the University of Alberta Health Research Ethics Board (HREB)—Panel B (File Number Pro00053154). Given the retrospective nature of the research, a waiver of consent was provided by the HREB. All necessary data were provided from a research agreement signed between the principal investigator and AHS.

Sample Size

The sample size was derived using the CSHP 2015 prevalence goal of 50%, an ideal sensitivity of at least 85%, and a P value of significance of <.05. This resulted in a required sample size of 392 patients for a 95% precision. Patients in the final study cohort were randomly selected from each site. The number of patients sampled from each site was determined by the proportion of eligible discharges who participated in the patient experience survey.

Inclusion Criteria

Patients who answered “yes” or “no” to the survey question: “During your hospital stay, did you meet with a pharmacist?” and who were discharged from 1 of 2 selected acute care sites within Edmonton, Alberta, during the study period were eligible for inclusion. In the case of multiple discharges for the same patient during the study time period, only the first hospital discharge was included. As part of the H-CAHPS survey protocol, patients were ineligible to complete the study if one or more of the following exclusion criteria were met: less than 24 hour stay, <18 years of age, death during hospital stay, psychiatry physician or psychiatry unit, day surgery or ambulatory procedure, and individuals only receiving services from an emergency department (10). Alberta Health Services-specific survey exclusion criteria were also applied. These included any possible dilation and curettage (D&C) procedure, possible still birth, and any adult hospital stay linked to a baby who remained in hospital for greater than 6 days (eg, postbirth complication/neonatal intensive care unit stay; 11).

Data Collection

Data sources for this study included the modified patient experience (H-CAHPS) survey, the corresponding electronic inpatient record (as provided by the Discharge Abstract Database [DAD]), patient charts, and pharmacist clinical workload data extracts.

Patient Experience Survey

The results of the survey question “During your hospital stay, did you meet with a pharmacist?” were obtained for the study cohort from the 2013 to 2014 inpatient hospital experience survey. The survey was independently conducted by a team of trained health research interviewers where patients were contacted on weekdays from 10 am to 9 pm and on Saturdays from 9 am to 4 pm up to 42 days postdischarge. To ensure that quotas were met, each telephone number was dialed up to 9 times on varying days and times. Each telephone survey required approximately 15 minutes to complete using a standard script, a list of standard prompts, and responses to frequently asked questions. Ten percent of calls were monitored for quality assurance and training purposes. Over the study period, a total of 9279 completed surveys were obtained within 6 weeks of discharge from 1 of 93 hospitals across the province. Once patients were reached, the overall response rate for the survey was 72% (12).

Discharge Abstract Database

Survey results were linked with the corresponding inpatient record, as documented in the DAD, using personal health number, date of discharge, and hospital number. Data obtained from the DAD included personal health number, case number, site and unit of hospitalization, sex, age at discharge, and length of hospital stay (in days). Reports on DAD documentation and validation are provided elsewhere by the Canadian Institute for Health Information (13,14).

Chart and Workload Data Review

Patient charts and pharmacist clinical workload data were used as the 2 reference comparators for the survey results. Charts were examined to determine whether the patient was seen by a pharmacist during the hospital admission. Presence of any type of pharmacist documentation in the patient record indicating that a pharmacist interacted with the patient was noted and considered evidence of pharmacist seeing the patient (eg, Best Possible Medication History [BPMH], progress note regarding medication counseling, patient assessment, patient education, drug-related problem, etc). According to Standard 18.8 of the Alberta College of Pharmacists Standards of Practice, “a pharmacist who provides professional services in an institution pharmacy, as defined in the Pharmacy and Drug Act, or in an environment with other regulated health professionals who have a shared medical or patient record, may document the pharmacist’s activities in the institution’s medical record or the shared medical or patient record for the patient” (15). The assumption was made that any care activities done by pharmacists were documented in the patient chart according to the standards of practice.

Pharmacist clinical workload data were obtained via a self-reporting tool commonly referred to as the “Service Log” (Figure 1). Patient-specific clinical workload information reported by AHS pharmacists was obtained for the study sites and units that were in-scope for the study. The presence or absence of pharmacist documentation in the Service Log for services provided directly (“Patient Present”)/indirectly (“Patient Not Present”) to the patient was noted for each eligible case.

Figure 1.

Pharmacist Service Log.

Data Analysis

Descriptive statistics were used to describe the study population. Proportions were used for categorical variables, with medians and interquartile ranges used for continuous data. The primary study outcome was the sensitivity and specificity of the survey question, “During your hospital stay, did you meet with a pharmacist?” For each reference measure (eg, chart review and pharmacist Service Log), the percentage and corresponding 95% confidence interval (CI) were calculated. The positive predictive value (PPV) and negative predictive value (NPV) were also determined, in similar fashion, using 2 × 2 tables. To satisfy our secondary objective (whether accuracy of patient recall varies according to patient characteristics), the 2-tailed Mann-Whitney U test was used to compare length of stay and age of those who answered the survey question correctly compared to those who answered incorrectly. Analyses were performed using IBM SPSS, version 19.0.0 (IBM Corporation, Armonk, New York). In all cases, P < .05 was considered statistically significant.

Results

A total of 391 individuals were included in this study. The median age of patients was 56 years (range: 42-68 years), median length of stay was 4 days (range: 2-7 days), and 216 (55%) were females. Two-hundred fifteen patients received care at site 1, and 176 patients received care at site 2 (Table 1). Patient demographics and length of stay were similar between the 2 sites.

Table 1.

Study Sample Characteristics.^a

Characteristic	Result
Female, n (%)	216 (55.2)
Median age, in years, interquartile range	56 (42-68)
Median length of stay in days, interquartile range	4 (2-7)
Care site 1, n (%)	215 (55.0)

^an = 391.

Of the 391 patients included, 195 (49.9%) recalled speaking with a pharmacist during their hospital stay. Of these 195, 71 (36.4%) patients had corresponding pharmacist documentation in their chart (Figure 2). However, of the 196 (50.1%) patients who did not recall speaking with a pharmacist, 73 (37.2%) had pharmacist documentation in their chart (Figure 2). The majority of pharmacist chart documentation was in the form of BPMHs and progress notes. Compared with patient charts, the overall sensitivity and specificity were 49.3% (95% CI: 40.9%-57.8%) and 49.8% (95% CI: 43.4%-56.2%), respectively. The PPV and NPV were 36.4% (95% CI: 29.7%-43.6%) and 62.8% (95% CI: 55.6%-69.5%), respectively (Figure 2). The sensitivity, specificity, PPV, and NPV did not significantly differ between the 2 study sites.

Figure 2.

Patient recall of pharmacist interaction compared to pharmacist documentation in patient charts.

Of the 195 patients who recalled speaking with a pharmacist, corresponding clinical workload data indicating direct (Patient Present) interaction with the patient were found for 64 (32.8%) of these individuals (Figure 3). On the other hand, of the 196 patients who did not recall speaking with a pharmacist, clinical workload data were present for 75 (38.3%) patients (Figure 3). Compared with workload data, the overall sensitivity and specificity were 46.0% (95% CI: 37.6%-54.7%) and 48.0% (95% CI: 41.7%-54.4%), respectively. The PPV and NPV were 32.8% (95% CI: 26.3%-39.9%) and 61.7% (95% CI: 54.5%-68.6%), respectively (Figure 3). As with the chart review data, the sensitivity, specificity, PPV, and NPV did not significantly differ between study sites.

Figure 3.

Patient recall of pharmacist interaction compared to pharmacist documentation in Service Log.

With respect to secondary outcomes, recall was observed to have varied with age. Based on the chart review, patients who correctly recalled speaking with a pharmacist had a significantly lower median age, when compared to patients who incorrectly recalled interacting with a pharmacist (52 years [range: 35-66] vs 60 years [range: 48-68]; P = .03; Table 2). Accuracy of recall did not significantly differ according to median length of hospital stay (correct = 4 days [range: 2-7] vs incorrect = 4 days [range 2-8]; P = .23; Table 2).

Table 2.

Patient Recall of Pharmacist Interaction Based on Age and Length of Stay (Median and Interquartile Range).

Variable	Correct	Incorrect	P
Age (years)	52 (35-66)	60 (48-68)	.03
Length of stay (days)	4 (2-7)	4 (2-8)	.23

Discussion

Patient satisfaction is an important indicator of health-care quality. With the expanding scope of pharmacists, evaluating the quality of pharmacist services provided to patients in hospital is key in augmenting the pharmacist’s role as a valued member of the health-care team.

In our validation study, approximately half of the 391 patients reported meeting with a pharmacist during the course of their hospital stay. Of these 195 patients, only 71 (36.4%) had documentation of this interaction in their patient chart, and only 64 (32.8%) had associated documentation in the pharmacist clinical workload data. Conversely, of the 196 patients who reported never meeting with a pharmacist, chart documentation, which indicated otherwise, was found in 73 (37.2%) charts and in 75 (38.3%) of the pharmacist Service Logs. The accuracy with which patients reported meeting with a pharmacist varied according to age, with younger patients reporting more correctly. Accuracy of reporting did not vary according to length of hospital stay.

Since the launch of CSHP 2015, a number of sites across Canada, such as the former Calgary Health Region Pharmacy Program and Horizon Health Network in New Brunswick, have examined patients’ recall of pharmacist interaction through satisfaction surveys (16,17). However, patient recall of pharmacist interaction may be confounded by a number of factors, including interaction with other health-care providers, acuity of illness, age, and length of stay. Our study takes previous research one step further, by assessing the accuracy of patients’ recall of meeting with a pharmacist using 2 established reference standards.

There are several key strengths of this study. The survey methods were according to the established H-CAHPS standards. As such, a validated tool was used with a standard script, prompts, and answers to frequently asked questions, as well as a telephone dialing protocol and sampling strategy, which targets potential respondents on multiple occasions at varying times of day. This overcomes limitations of ad hoc instruments including issues pertaining to validity and reliability (18) as well as response bias. As all eligible patients are targeted until the survey quota is met, each potential respondent has an equal chance of participation. The abstracted data include up to 2 telephone numbers provided at hospital registration. Potential respondents who are not able to speak freely at the time of an initial telephone call are provided with the opportunity to book a follow-up call at their convenience. Time is set aside each day for interviewers to complete these follow-up calls to further reduce the potential for nonresponse (19,20). The quota-based sampling approach has been applied elsewhere with similar success (21). Lastly, when compared to the eligible inpatient population, the respondents have been shown to be representative in terms of age, gender, and hospital admission type (urgent vs elective; 12).

Our study is not without limitations. First, there is an assumption on our part that pharmacists are fully compliant with documentation of patient interactions in the patient chart and/or Service Log. We concede that pharmacist interaction may not have been consistently documented in patients’ charts. An attempt was made to mitigate this limitation by correlating survey results with pharmacist clinical workload data (a second reference standard). However, there were approximately 58% of patients without corresponding Service Log data. This highlights a second limitation of our study, as some patients may have been admitted to a clinical area, where clinical pharmacist services are not available or where compliance with Service Log documentation may be problematic due to time constraints, forgetfulness, or apathy. Finally, due to our telephone method, patients who are sick (eg, not able to speak on the telephone or who have been rehospitalized) may be precluded from participating in the survey. This has been shown in previous work (12), where survey respondents were shown to have shorter hospital stays and fewer comorbidities. One may also infer that this cohort of nonrespondents may also be more apt to have a pharmacist consultation due to their increasingly complex medical needs. A follow-up study may be helpful to examine any potential differences in pharmacist recall among this cohort. With that said, older age appeared to negatively impact patient recall, while length of hospital stay was similar for patients who either correctly or incorrectly recalled interacting with a pharmacist. This latter finding does suggest that these patients likely had equal opportunity to interact with a pharmacist.

Through association of patient chart and Service Log reviews with patient-reported survey responses, the overall sensitivity and specificity of the question pertaining to meeting with a pharmacist was determined to be approximately 50%. Thus, patients’ recall of pharmacist interaction was only 50% accurate over the study period at the 2 selected acute care sites. Based on our findings, this particular survey question does not accurately assess patient recall of pharmacist interaction, and there may be little value in continuing to include these questions in the survey as written.

Nonetheless, these data provide numerous opportunities for quality and service improvement. Pharmacy Services in our organization has recently modified the survey question to aim for more accurate results. The modified question is now worded: “During your hospital stay, did a pharmacist meet you at the bedside to discuss your medications?” Investigation into how patient recall pertaining to pharmacist services is assessed in other Canadian health regions, or internationally, would also be helpful. Other strategies may include implementing campaigns to increase patient recall of pharmacist interaction through improved communication between frontline pharmacists and patients. One such way to do this has been the introduction of Name, Occupation, and Duty (NOD). As part of the AHS’ Patient First Strategy, NOD stresses the importance for all staff to introduce themselves to patients and their families using their NOD that they perform (22). Given the aims of NOD, we can expect improvements in recall of pharmacist interaction. This may also provide additional benefits from a patient- and family-centered care perspective. A replication study to assess the effectiveness of NOD from a Pharmacy Services standpoint would be of benefit. Future research could also focus on expanding the present validation project to rural acute care facilities to determine whether similar results are obtained across smaller facilities within our province.

Footnotes

Authors) Note

Kyle Kemp is now affiliated to Health System Analytics, Health Quality Council of Alberta, Calgary, Alberta, Canada.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this project was provided by Pharmacy Services, Alberta Health Services.

References

Canadian Pharmacists Association. A review of pharmacy services in Canada and the health and economic evidence. 2016. https://www.pharmacists.ca/cpha-ca/assets/File/cpha-on-the-issues/Pharmacy%20Services%20Report%201.pdf. Accessed April 10, 2018.

Lynas

. Professionals you can trust: pharmacists top the list again in Ipsos Reid survey. Can Pharm J. 2012;145:55.

Canadian Society of Hospital Pharmacists. CSHP 2015 Crosswalk: Targeting excellence in pharmacy practice. 2011. https://cshp.ca/sites/default/files/Clinical%20Pharmacy/Crosswalk%202011%20Revision%20FINAL%20(2)%20-%20Nov%202011.pdf. Accessed April 11, 2018.

Hospital Pharmacy in Canada Editorial Board. Hospital pharmacy in Canada: 2013/14 Report. 2015. http://www.lillyhospitalsurvey.ca/hpc2/content/Reports3.asp. Accessed April 10, 2018.

Centers for Medicare and Medicaid Services. HCAHPS fact sheet. 2017. http://www.hcahpsonline.org/globalassets/hcahps/facts/hcahps_fact_sheet_november_2017a.pdf. Accessed April 10, 2018.

Darby

Hays

Kletke

. Development and evaluation of the CAHPS hospital survey. Health Serv Res. 2005;40:1973–6.

Goldstein

Farquhar

Crofton

Darby

Garfinkel

. Measuring hospital care from the patients’ perspective: an overview of the CAHPS hospital survey development process. Health Serv Res. 2005;40:1977–95.

Alberta Health Services. Patient Experience: Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) pharmacy results for Quarter 4 Fiscal Year 2013-2014.

Alberta Health Services Annual Report 2013-2014. 2014. https://www.albertahealthservices.ca/Publications/ahs-pub-2013-2014-annual-report.pdf. Accessed April 10, 2018.

10.

Centers for Medicare and Medicaid Services. Introduction to HCAHPS survey training. 2018. http://www.hcahpsonline.org/globalassets/hcahps/training-materials/2018_training-materials_slides_introduction.pdf. Accessed April 11, 2018.

11.

Kemp

Chan

McCormack

Douglas-England

. Drivers of inpatient hospital experience using the HCAHPS survey in a Canadian setting. Health Serv Res. 2015;50:982–97.

12.

Kemp

Chan

McCormack

. The Alberta inpatient hospital experience survey: representativeness of sample and initial findings. Survey Prac. 2015;8:1–13.

13.

Canadian Institute for Health Information. CIHI data quality study of the 2009-2010 discharge abstract database. 2012. https://secure.cihi.ca/free_products/Reabstraction_june19revised_09_10_en.pdf. Accessed April 11, 2018.

14.

Canadian Institute for Health Information. DAD data elements 2013-2014. 2014. https://www.cihi.ca/sites/default/files/document/dad_data_elements_2013_2014_en.pdf. Accessed April 11, 2018.

15.

Alberta College of Pharmacists. Standards of practice for pharmacists and pharmacy technicians. 2014. https://pharmacists.ab.ca/sites/default/files/StandardsOfPractice_May2014_v2.pdf. Accessed April 11, 2018.

16.

Slack

Ing

. Prevalence and satisfaction of discharged patients who recall interacting with a pharmacist during a hospital stay. Can J Hosp Pharm. 2009;62:204–8.

17.

Doucette

Goodine

Symes

Clarke

. Patients’ recall of interaction with a pharmacist during hospital admission. Can J Hosp Pharm. 2013;66:171–6.

18.

Waljee

McGlinn

Sears

Chung

. Patient expectations and patient-reported outcomes in surgery: a systematic review. Surgery. 2014;155:799–808.

19.

Goyder

. Face-to-face interviews and mail questionnaires: the net difference in response rate. Public Opin Q. 1985;49:234–52.

20.

Heberlein

Baumgartner

. Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. Am Sociol Rev. 1978;43:447–62.

21.

O’Cathain

Knowles

Nicholl

. Testing survey methodology to measure patients’ experiences and views of the emergency and urgent care system: telephone versus postal survey. BMC Med Res Methodol. 2010;10:52–61.

22.

Alberta Health Services. Patient First Strategy: Frequently Asked Questions. 2016. http://www.albertahealthservices.ca/assets/info/pf/first/if-pf-1-pf-faq.pdf. Accessed April 11, 2018.