Abstract
Introduction:
Remote blood pressure monitoring (RBPM) programs are increasingly utilized to improve hypertension care. Rigorous analysis of program outcomes including clinical effectiveness and participant experience can inform future initiatives.
Methods:
During year 1 of the COVID-19 pandemic, we implemented a RBPM program for patients who: (1) received primary care in a single academic medical network, (2) were part of an accountable care financial arrangement, and (3) had uncontrolled hypertension. Evaluation combined a 6-month prospective cohort observational study (assessing the program’s association with hypertension control and remote blood pressure [RBP] reporting) with surveys (assessing patient and care team experience.)
Results:
A total of 150 patients (mean age 57 years, 65% male) enrolled across 10 clinics, of whom 121/150 contributed remote blood pressure data. Among the patients who contributed data, we observed an adjusted reduction in systolic blood pressure by 1.08 mm Hg/month (95% CI = −1.24 to −0.91) and diastolic blood pressure by 0.88 mm Hg/month (95% CI = −0.99 to −0.77) associated with our intervention. The number of patients contributing to RBPM data declined from 121 to 22 from inception to the end of the 6-month study. Among the 61 patient survey respondents (40% response rate), 80% reported high program satisfaction and likelihood to recommend. Survey respondents noted improvements in weight loss (14%), medication compliance (16%), diet (29%), and exercise (35%). Qualitative survey analysis identified themes of patient convenience and increased self-efficacy in blood pressure (BP) management. Quantitative and qualitative patient and care team survey analysis showed technology linkage challenges.
Conclusion:
Overall, our primary care RPBM program was associated with improved blood pressure control among participants and favorable patient and care team experience but experienced challenges of significant decline in blood pressure reporting over time. For future institutional RBPM implementations, we aim to retain the high quality of blood pressure management guidance that participants received while increasing technology connectivity and longitudinal reporting support.
Introduction
Hypertension is a leading preventable risk factor for global cardiovascular disease and all-cause mortality, yet it is commonly undiagnosed and undertreated in outpatient settings.1 -4 Remote blood pressure monitoring (RBPM) is a method for improving longitudinal blood pressure (BP) care compared to traditional office-based management. 5 RBPM programs have improved blood pressure control which can have significant reduction on strokes, cardiac infarction, and hospital readmissions.6 -8
In late 2020, our quality improvement team at Stanford primary care implemented a novel RBPM pilot. Several institutional factors led to the pilot: (1) a decline in hypertension quality metrics due to decreased in person visits associated with the COVID-19 pandemic, (2) ongoing patient and provider preference for telemedicine in the pre-COVID vaccine era, (3) financial arrangements among accountable care insurers and employer-based clinics to incentivize hypertension management at the medical group or practice level, and (4) health system innovation scholarship funds covering the cost of remote blood pressure monitors for patients.9 -15
In this manuscript we share our mixed methods analysis of the outcomes associated with our RBPM program including several clinical effectiveness outcomes (i.e. monthly change in BP, patient-reported behavioral change, and number of patients submitting remote BP data) as well as patient and care team experience.
Methods
Study Design
Our evaluation assessed program outcomes through quantitative assessment of a prospective observational cohort and both quantitative and qualitative assessment of patient and care team experience surveys.
Setting
The pilot study was conducted at 10 primary care clinics (5 faculty employer-based clinics, 4 faculty general primary care clinics, and 1 community clinic) at Stanford Medicine, a single academic medical network in California, from July 2020 to October 2021.
Participants/Study Cohort
Patient eligibility included all of the following elements: (1) having uncontrolled hypertension, as defined by the National Committee on Quality Assurance 2019 metric, (2) (for non-employer based sites) accountable care organization (ACO) or managed care insurance, (3) (for non-employer based sites) meeting pharmacist co-management agreement eligibility criteria (see Supplemental Appendix), and (4) (for employer-based sites) primary care provider (PCP) recommendation for enrollment. 16
Intervention
Clinic staff identified patients for potential enrollment through review of electronic medical record hypertension care gap reports. From these reports, staff verified suitability for enrollment by conducting chart reviews to assess pharmacy eligibility at non-employer sites or consulting PCPs at employer sites. Staff at all clinics then contacted eligible patients for enrollment through a combination of telephone outreach and secure patient portal outreach (Figure 1).

Patient steps in remote blood pressure monitoring pilot. 17
All patients who enrolled in the pilot received a free Withings Bluetooth blood pressure cuff and written training materials. Training materials included information about device-set up including linkage to the secure patient portal, home blood pressure measurement technique, and advice to patients to monitor blood pressure for 2 weeks prior to seeing a pharmacist or PCP for follow-up (Supplemental Materials). Employer based clinics provided patients with options for on-site device set-up assistance and offered follow-up provider visits to review remote BP data. Non-employer sites varied in their approach to device set-up with some providing on-site device set-up assistance and others unable to provide this support due to insufficient personnel. Non-employer sites consistently offered follow-up pharmacist visits to review remote BP data.
Data Sources and Survey Instruments
We collected patient-level data for participants (demographics, clinical diagnoses, and remote blood pressure values and reporting dates) from the electronic medical record via the Stanford STARR data repository. 18 We used this data for quantitative assessment of program effectiveness.
To assess patient and care team experience, we developed 2 de novo surveys based on literature review from prior remote monitoring studies and consultation with institutional patient experience professionals.19 -22 While questions derived from validated surveys, our de-novo surveys did not undergo cognitive pretesting with patients or staff. The patient survey consisted of the following categories: overall experience, technology, care team support, health impact, and additional comments. The care team survey included clinic staff (e.g. medical assistants, pharmacists, and managers) and providers and focused on demographics, involvement in the project, management role, medication and lifestyle adjustments, and project evaluation (Supplemental Appendix).14 -16,18
Survey Administration
We shared an English-language survey through a secure patient portal message with participating patients at 6 to 9 months after enrollment. We performed 2 attempts at phone call outreach for phone survey administration in case of initial non-response. Interpreters aided with phone surveys issued to patients with a non-English preferred language. As an incentive for survey completion, patients were entered into a raffle for a $50 gift card. Four total cards were issued to patients over the course of 2 rounds of outreach. We invited care teams who had participated in the pilot to complete a care team survey through 3 rounds of email outreach.
We used Qualtrics software to generate individualized web links for the survey participants, and all survey questions were reported and analyzed using the Qualtrics Survey Tool. 23
Ethical Review and Participant Confidentiality
The Stanford School of Medicine IRB determined our study to be not human subjects research (protocol ID: 59076). Files tracking which patients received or completed surveys were securely stored and accessible only to select study team members (AP, SK, and MS) but not to the team statistician (YW). Survey responses did not include any protected health information.
Outcomes
The primary outcome was remote systolic and diastolic blood pressure changes over 6 months following the initial remote blood pressure measurement of each patient. Our secondary outcomes were: (1) the number of patients reporting blood pressure over 6 months (in our electronic medical record data extract), (2) patient and care team experiences with the pilot (through survey analysis), and (3) patient behavioral change associated with the pilot (through survey analysis).
Analysis
Patient Characteristics – Descriptive Analysis
We described patients’ demographics and clinical diagnoses for the entire study cohort. Frequencies and proportions were reported for categorical variables while mean and standard deviations were reported for continuous variables.
Primary Outcome of Blood Pressure Control and Secondary Outcome of Remote Blood Pressure (RBP) Reporting – Statistical Analysis
We used generalized linear mixed effect models (GLMM) to estimate the overall impact of the intervention on our primary outcome. We obtained a longitudinal dataset where the observation unit at the follow-ups were RBP readings, and we defined the baseline event as the initial RBP reading following the primary care visit during the study period. We applied mixed effect techniques by regressing each primary outcome on weeks from baseline enrollment and adjusting for other patients’ variables including age, gender, race, preferred language, Charlson Comorbidity Index, and the baseline blood pressure control status. Also included in the model is a subject-specific random effect to account for the correlation of measurements over time within a subject/patient. 24 All blood pressure measurements from RBPM were used and patients were censored at 6-month post enrollment or time lost to follow-up. For this purpose, we used a 2-sided test at the .025 level of significance.
To visualize the trend of the primary outcomes over time, trend plots including point estimates and 95% confidence intervals from the estimated marginal means of the model were produced over follow-up time. In addition, we reported the number of patients reporting RBP over time as a secondary analysis. We compared all patients who had zero BP values reported versus those who had at least 1 BP value reported. We used complete data analysis for handling missing data.
Patient and Care Team Feedback: Quantitative/Qualitative Survey Analysis
Quantitative survey questions consisting of Likert scale items were evaluated using mean and standard deviation while multiple-choice questions were evaluated using frequency distribution.
Open-ended questions were analyzed using inductive reasoning, narrative, and generic qualitative approach to identify themes. 25 Team members (HB and ST) participated in survey analysis training and used qualitative analysis methods modeled after the Braun and Clarke thematic analysis methodology. 25 Team members (JRP and AP) provided expert input in selecting pertinent quotations from the surveys and developing theme summaries. The study team performed the initial review of the participant survey comments. Four rounds of revisions were made, and 10 themes were established in conjunction with roles, values, and telehealth. Grounded theory helped the study team discover emergent themes. Codes were analyzed by team members and themes were then finalized by agreement.
Results
Patient Demographics
A total of 150 patients enrolled in the program (Table 1). Participants had a mean age of 57 years (SD = 13) and 65% were male. Racial composition was 31% White, 30% Asian, 5% Black, and 13% identified as Latino/Hispanic ethnicity. The majority had no Charlson Comorbidity Index conditions.
Patient Demographics.
Outcomes – Remote Blood Pressure Values and Reporting Over 6 Months
Over a 6-month period, remote systolic blood pressure (SBP) decreased by an average of 1.08 mm Hg/month (95% CI = 0.91-1.24, P < .001), while diastolic blood pressure (DBP) declined by 0.88 mm Hg/month (95% CI = 0.77-0.99, P < .001; Figure 2). These reductions were observed from a baseline blood pressure of 134/84 mm Hg, after adjusting for age, gender, race, preferred language, Charlson Comorbidity Index, and baseline blood pressure control status. The sharpest decline in SBP and DBP occurred in the first 2 months.

Trends in remote systolic and diastolic blood pressure among contributing patients.
There were 121 of 150 patients who provided at least 1 BP measurement at the start of the pilot; by week 24 there were 22 patients contributing BP measurements (Figure 2). The greatest decline in the number of patients reporting RBP occurred in the first month. Rate of decline in patients reporting RBP over time was more gradual thereafter.
We compared the demographic and clinical characteristics found in Table 1 for patients who reported zero BP values to those who reported at least 1 BP value and found chronic kidney disease to be the only characteristic that significantly differed between these 2 groups of patients. Chronic kidney disease prevalence was higher among patients with zero BP values reported as compared to those with 1 or more reported home BP value (P = .002.)
Secondary Outcome – Patient and Care Team Survey
Patient Surveys: Quantitative Analysis
In total, 61/150 patients (response rate 40%) responded to the patient experience survey (Figure 3). Approximately 80% of participants felt comfortable sharing BP through the new technology, found care team guidance to be helpful, were satisfied with the program, and would recommend the program. Thirty-three percent of patients expressed a negative or neutral view on the ease of BP cuff setup and connecting to the secure patient portal app, MyHealth (i.e. MyChart). Regarding lifestyle improvements among survey participants (n = 61), 14% of participants reported weight loss, 16% reported improved medication compliance, 29% reported improved diet, and 35% reported increased exercise.

Patient experiences through surveys.
Patient Surveys: Qualitative/Thematic Analysis
In thematic analysis, detailed in Table 2, patients cited several favorable components of the program including increased convenience and time-savings with remote monitoring and improved self-management. Responses to care team interaction were mixed with certain patients sharing favorable interactions and others expressing frustration regarding lack of clinic support with technology connectivity. Additionally, patients noted difficulty syncing different apps and technology failure after initial set-up.
Thematic Analysis of Patient Surveys.
Care Team Surveys: Quantitative Analysis
Twenty-three of 29 care team members across 9 sites responded to the survey (response rate 79%; 7 medical assistants, 7 pharmacists, 5 managers or assistant managers, 2 population health coordinators, and 2 physicians). The number of respondents per site ranged from 1 to 6. Among the 7 pharmacists and 2 physicians asked about ease of providing appropriate lifestyle recommendations or medication adjustments, all responded favorably. Approximately 80% of respondents found it easy to receive the patient’s remote blood pressure readings (n = 9/9 respondents) and would recommend continuing the project at their site (n = 21/29 respondents).
Discussion
Our primary care RPBM program was associated with improved blood pressure control among participants and favorable overall patient and care team experience alongside challenges of declining blood pressure reporting over time and technology linkage frustrations among patients. Over 6 months our pilot was associated with an average monthly SBP decline of 1.08 mm Hg (95% CI = 0.91-1.24) and diastolic blood pressure (DBP) decline of 0.88 mm Hg/month (95% CI = 0.77-0.99). Patient and staff survey respondents jointly reported high satisfaction and likelihood of recommending or continuing the program. The main negative finding of our program evaluation was the significant fall in blood pressure reporting over time. Throughout the study, 29 of 150 participants never reported RBP data and by study completion an additional 99 patients had stopped reporting values.
Comparison to Prior Studies
Our primary outcome of decline in remote BP could be very clinically impactful for participants if sustained. Our study found a cumulative 6 mm Hg BP decline among those reporting BP data for 6 months. Rahimi 26 report that a 5 mm Hg BP reduction correlates with a 10% cardiovascular event reduction if sustained over 3 to 5 years. Additionally, the change in BP found in our primary outcome was comparable to 2 other quality improvement (QI) studies but less than that of other studies where protocols included more frequent outreaches from the care team and increased use of automated reminders for patients to record BP values.27,28
Regarding our secondary outcomes, our BP reporting data was more rigorous then other QI comparator studies as we reported monthly rates of reporting whereas other studies only shared reporting rates at study end.27,28 Patient experience data showed similar overall program favorability to other QI studies but our respondents had less favorable views of BP machine set-up and instruction processes, likely due to more intensive onsite device set-up support in Doane et al’s study.29,30 The self-reported behavioral changes in our study are more detailed than those seen in other QI RBPM studies. 27
Qualitative themes among comparator RBPM studies included patient-perceived convenience, improved communication with the care team, and enhanced self-awareness.29 -32 Our study found similar themes: patients appreciated the convenience of RBPM, valued care team interactions, and improved their understanding of how stress and medication adherence affect BP. Challenges with device usability and digital literacy were concerns across other RBPM studies and noted in our study. Themes from the literature that were not found in our study included provider uncertainty and hesitancy with new workflows, data security and system trust among patients, and concerns with equity in RPBM participation.29 -32
Interpretation of Main Outcomes
Our patient surveys provide insights that may explain the improved BP control found in our study as well as the decline in BP reporting. As a possible explanation as to the factors supporting improved BP control, through qualitative data analysis we found patients appreciated the convenience and time saving nature of RBPM, learned from their interactions with the care team, and gained a deeper understanding of how factors such as stress and timely medication use can improve blood pressure. Quantitative patient survey data demonstrated that among the 61 survey respondents the intervention was associated with lifestyle changes that can lead to improved blood pressure control including weight loss (14%), improved medication compliance (16%), improved diet (29%), and increased exercise (35%).
When considering our secondary outcome of BP reporting, one possible explanation for decline in reporting may have been that when patients had controlled values, clinical teams instructed them to no longer report. However, an additional explanation that is supported by our patient survey results is that technology connectivity challenges were a major driver behind decreased reporting. In our patient survey, 20/61 survey respondents responded unfavorably to the question regarding ease in BP device set up and linkage to the secure patient portal. Similarly, qualitative patient surveys results demonstrate themes of frustration among patients due to insufficient technical support or when clinical teams were unable to access their data. This frustration may have prompted patients to discontinue reporting.
Strengths
Our study had several strengths. First, our outcomes offered detailed insights regarding several aspects of clinical effectiveness, patient and care team experience, and improvement opportunities. Second, our comprehensive characterization of change in blood pressure reporting over time and numerous types of behavioral change associated with an RBPM program is unique and has not been reported in other QI RBPM studies. Third, our survey analysis was detailed, including both quantitative and qualitative assessment with thematic analysis as well as patient and care team perspectives. Finally, the benefit of the real-world nature of our pilot provides a view of how benefits of RBPM gleaned from clinical trials will translate into settings where these processes do not have dedicated research-based support but instead are integrated into daily clinical operations.33,34
Limitations
Our study has multiple limitations. First, the study was based at a single academic medical center and featured patients who were predominantly white or Asian, English speaking, and had few clinical comorbidities. Second, study design limited causal inference by not controlling for other interventions outside of the RBPM program that may have impacted patient BP. 35 Third, we did not directly link survey response data to outcomes such as blood pressure change or reporting, which limited our ability to fully explain both positive and negative outcomes.27,28 Fourth, our process did not include survey validation via pilot testing. Fifth, due to limited sample size we were unable to perform subgroup analysis by clinic site and can therefore not determine if specific clinic-level differences (e.g. pharmacy follow-up vs PCP follow-up) were more associated with improved BP outcomes. Finally, our patient survey response rate of 40% is less than that of other telephone surveys which can approach 50% to 70% therefore our survey results must be interpreted in the potential context of non-response bias.36 -39
Implications and Next Steps
Through our RBPM pilot, we found that providing patients with no-cost Bluetooth enabled blood pressure cuffs and clinician home BP guidance was associated with improved home BP control and was well received by patients and staff. Long-term patient engagement and technology integration barriers provided insights for future process improvement.
For future institutional RBPM implementations, we aim to retain the high quality of blood pressure management guidance that participants receive while increasing technology connectivity and longitudinal reporting support. Our detailed program evaluation and lessons learned can help other institutions seeking to reduce their patients’ overall risk of cardiovascular morbidity and mortality through RBPM program implementation.
Supplemental Material
sj-docx-1-jpc-10.1177_21501319251358000 – Supplemental material for A Mixed-Methods Evaluation of a Primary Care Remote Blood Pressure Monitoring Quality Improvement Pilot
Supplemental material, sj-docx-1-jpc-10.1177_21501319251358000 for A Mixed-Methods Evaluation of a Primary Care Remote Blood Pressure Monitoring Quality Improvement Pilot by Hossein Bakhshandeh, Sana Tehal, Oluseyi Fayanju, Sai Priyanka Kodam, Jesse Rokicki-Parashar, Margaret Seaton, Yingjie Weng and Anuradha Phadke in Journal of Primary Care & Community Health
Footnotes
Acknowledgements
The authors would like to thank the Stanford Primary Care management, medical assistant, population health, and pharmacy teams for their partnership in conducting this pilot.
Ethical Considerations
The Stanford School of Medicine IRB determined our study to be not human subjects research (protocol ID: 59076).
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: (1) The Stanford Clinical and Translational Science Award (CTSA) to Spectrum. The CTSA program is led by the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (NIH). (2) The NIH funding source of Stanford’s Center for Clinical and Translational Education and Research award, under the Biostatistics, Epidemiology and Research Design (BERD) Program: UL1TR003142
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
All data supporting the study findings are contained within this manuscript.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
