Sage Journals: Discover world-class research

Abstract

Introduction/Objectives

We examined an initial step towards co-generation of clinic notes by inviting patients to complete a pre-visit questionnaire that could be inserted into clinic notes by providers and describe the experience in a safety-net and non-safety-net clinic.

Methods

We sent an electronic pre-visit questionnaire on visit goals and interim history to patients at a safety-net clinic and a non-safety-net clinic before clinic visits. We compared questionnaire utilization between clinics during a one-year period and performed a chart review of a sample of patients to examine demographics, content and usage of patient responses to the questionnaire.

Results

While use was low in both clinics, it was lower in the safety-net clinic (3%) compared to the non-safety-net clinic (10%). We reviewed a sample of respondents and found they were more likely to be White compared to the overall clinic populations (p < 0.05). There were no statistically significant differences in patient-typed notes (word count and number of visit goals) between the safety-net and non-safety-net samples however, patients at the safety-net clinic were less likely to have all of their goals addressed within the PCP documentation, compared to the non-safety-net clinic.

Conclusions

Given potential benefits of this questionnaire as a communication tool, addressing barriers to use of technology among vulnerable patients is needed, including access to devices and internet, and support from caregivers or culturally concordant peer navigators.

Keywords

electronic health records health information technology health equity digital divide vulnerable populations

Collaboration between patients and providers is a critical component of patient-centered care and is associated with improved health outcomes.^1–3 Open-ended questions at the start of a visit are associated with increased patient satisfaction, and in a randomized trial, pre-visit questionnaires administered via electronic portals assisted patients in preparing for visits and were viewed favorably by providers.^4,5 An innovative way to strengthen patient-provider collaboration is allowing patients to co-write their clinic visit notes.^6,7 An early study of patients and providers co-writing clinic visit notes involved patients typing their visit agendas prior to the clinic visit and inserting this text into the body of the doctor's note.⁶ Patients and providers involved in these co-written notes perceived improved communication, suggesting that co-written notes could improve patient engagement, decrease the documentation burden for providers, and improve the efficiency of clinic visits.⁴ Giving patients the opportunity to tell their story in their own words and including this in the clinic note is empowering, and may particularly appeal to patients who have experienced systemic oppression or disenfranchisement.⁸

Historically, medical records have been easily available only to medical providers. HIPAA gave patients legal access to their medical records but in practice the records could still be difficult and costly to procure.^9,10 Over the last decade the OpenNotes movement has advocated making visit notes and medical records easily accessible by patients to improve transparency and collaboration.² This movement is supported by evidence that patients who utilize health information technology to collaborate with their provider experience greater satisfaction with care^6,8,11,12 and improved quality of care.¹³ With implementation of the Final Rule of the 21st Century Cures Act, patients in the U.S.A now increasingly have ready electronic access to their health information.¹⁴

Medical record electronic patient portals may improve patient experience or outcomes,^3,15,16 however there is concern that many patients may be unable to engage with the patient portal because of limited computer and health literacy and other barriers.^17,18 Evidence suggests that non-white patients are less likely to access their health information online^13,19–21 and are less likely to be offered access to this information.²⁰ As a result, patients in some minority groups may be excluded from the potential benefits of health information technology.

Given the potential benefits of co-generated clinic notes, we wanted to understand utilization of this novel questionnaire by safety net and non-safety-net populations and had the opportunity to do so in a health system using the same technologic process and health record. We compared implementation and utilization of this novel questionnaire in two large academic clinics, one serving a safety-net population and the other a non-safety-net population. Additionally, we performed a chart review of a sample of respondents’ notes in each clinic, examining thematic content and provider use of the patient responses.

Methods

Two large academic internal medicine primary care clinics in [Seattle, WA] participated in this study between June 1, 2018 and June 30, 2019. One clinic is located at [Harborview Medical Center], a county safety-net hospital serving those with financial barriers to care and individuals from historically disenfranchised populations (the safety-net clinic). The other clinic is located at the [University of Washington Medical Center] and serves a primarily privately insured population (hereafter referred to as the non-safety-net clinic).

To be eligible, patients at these clinics had to be 18 years or older, have an active patient portal account, be able to complete the questionnaire in English, and have had three or more visits within the health system in the last 12 months. Clinic medical directors provided information about the pre-visit questionnaire to providers at clinic meetings and via email; patients did not receive any messaging beyond receiving the pre-visit questionnaire notification email. Institutional Review Board approval for the study was obtained from the University of Washington.

Eligible patients were sent an email up to seven days prior to a scheduled appointment notifying them that a new message was available on the patient portal. To answer the questionnaire, patients clicked the link in their email and logged into their patient portal account. The questionnaire asked them to type their interval histories since the most recent clinic visit, and outline their goals for the clinic visit (see Box 1 for full questionnaire). Each question had a character-limited text box for patients to type in their responses. Providers could read the responses in the patient's chart prior to and during the appointment. In the visit encounter in EpicCare (Epic Systems Corporation, Verona, WI), providers clicked a blue bar above the note labeled “Patient answers are available” to see the questions and patient-typed responses. When selected, a “Yes” button below the text titled “Do you want to add the following patient answers to your note” added the patient-entered information to the visit note at the location of the cursor in the note.
Box 1.
Two questions comprising the pre-visit questionnaire: interim history and goals.

Interim history. Please tell your doctor how have you been since last visit. (You may want to include changes in symptoms affecting your health, in the way you are taking medications, in your habits, or other changes in your life. To help guide what you write, we suggest you also read your doctor's last note, focusing in particular on previous plans or recommendations).

Goals. What issues would you like to focus on at your visit? (We suggest you limit what follows to the 2 or 3 things on your mind that are most important to you).

Responses to each question were limited to 197 words due to electronic medical record functionality.

We conducted a review of a purposive sample of patient charts and associated responses to the pre-visit questionnaire from providers who had received at least three completed pre-visit questionnaires (34 providers at the safety-net clinic and 52 at the non-safety-net clinic). Using a random number generator, we randomly selected two patients from each of these providers whose charts and responses we would review, for a total of 172 patients and their visit notes.

First, four authors (HS, EM, JJ, SJ) reviewed 10% of responses to determine the main content categories in the interval history and reached consensus on the following categories: physical symptoms, medications (prescription or over the counter medications/supplements), life changes, minimal response (eg, “fine”), mental health issues (such as depression, anxiety, or panic, or medication management of mental health diagnoses), and social issues (eg housing, financial challenges, employment, disability, violence/abuse, or paperwork related to these; Table 2). Second, two co-authors (EM and HS) each reviewed all remaining responses to categorize the interval history by content. Agreement of the content categorization was assessed using a Kappa agreement statistic; we had good agreement for physical symptoms κ = 0.75 (95% CI, 0.65 to 0.86), mental health issues κ = 0.67 (95% CI, 0.48 to 0.86), social issues κ = 0.70 (95% CI, 0.43 to 0.98), and minimal response, κ = 0.79 (95% CI, 0.68 to 0.91), and very good agreement for medications κ = 0.95 (95% CI, 0.90 to 1.00), and life changes κ = 0.82 (95% CI, 0.69 to 0.94). We reached consensus for any disagreements through a consensus meeting of co-authors.

One co-author (EM) counted the number of clinical history elements in the interval history (see Table 2),^22,23 the number of unique goals, if included, and reviewed patient charts to obtain patient demographic information, identify whether the patient attended the visit, and if the pre-visit questionnaire responses were integrated into visit notes by the primary care provider. When patients typed specific goals for the visit, the note was reviewed to identify how often the provider documented addressing the patient goals.

Differences in proportions between groups were evaluated using chi-squared tests or Fisher's Exact tests when frequencies were low. Differences in continuous measures were evaluated using two-sample t tests. All tests were two-sided and statistical significance was evaluated at p < 0.05. Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC).

Results

We sent pre-visit questionnaires before 17 844 visits at the safety-net clinic and 17 590 visits at the non-safety-net clinic; three percent were completed at the safety-net clinic (n = 473) and 10% at the non-safety-net clinic (n = 1808). We sampled 172 for inclusion in the analysis.

Among 172 sampled patient respondents, a higher percentage at the safety-net clinic were white and English speaking compared to the overall safety-net clinic population (p < 0.0001); the same was true at the non-safety-net clinic (race, p = 0.03 and English speaking, p = 0.02; Table 1). Twelve (18%) participants in the safety-net sample were black compared to five (5%) in the non-safety-net sample; and one (1%) participant was Asian in the safety-net clinic sample compared to nine (9%) at the non-safety-net clinic. The participant samples from each clinic differed in terms of age ( ≥ 45 years, 79% at safety-net and 66% non-safety-net, p = 0.001) and primary insurance provider type (private insurance 21% in safety-net, 47% non-safety-net, p = 0.0001; Table 1)

Table 1.
Demographics of Patients Attending the Safety-Net and Non-Safety-net Clinics and of Each Clinic’s Respective Sample.

Safety-Net Clinic Non-safety-net clinic

Participant sample^{^} (n = 68) Overall Clinic Population^# (n = 3887) p-value comparing sample to Overall Participant sample^{^} (n = 104) Overall Clinic Population^# (n = 8480) p-value comparing sample to Overall p-value comparing Participant samples

AGE

18–44 14 (21%) 766 (20%) 0.91 35 (34%) 2659 (31%) 0.08 0.001

45–64 33 (48%) 1824 (47%) 23 (22%) 2715 (32%)

≥ 65 21 (31%) 1295 (33%) 46 (44%) 3106 (37%)

SEX

Female 34 (50%) 1747 (45%) 0.41 46 (44%) 3896 (46%) 0.73 0.46

Male 34 (50%) 2140 (55%) 58 (56%) 4583 (54%)

PRIMARY LANGUAGE

English 68 (100%) 3011 (77%) <.0001 104 (100%) 8075 (95%) 0.02^b -

Other 0 (0%) 876 (23%) 0 (0%) 405 (5%)

RACE/ETHNICITY

White 51 (75%) 1494 (38%) <.0001^a 83 (80%) 5956 (70%) 0.03^a 0.46^a

Black or African American 12 (18%) 1176 (30%) 5 (5%) 461 (5%)

Hispanic 0 (0%) 525 (13%) 0 (0%) 307 (4%)

Asian 1 (1%) 440 (11%) 9 (9%) 924 (11%)

American Indian or Alaska Native 1 (1%) 50 (1%) 1 (1%) 41 (0.5%)

Native Hawaiian or Other Pacific Islander 1 (1%) 49 (1%) 1 (1%) 39 (0.5%)

Multiple Races 0 (0%) 31 (1%) 0 (0%) 86 (1%)

Declined to Answer/ Unavailable/Unknown 2 (3%) 122 (3%) 5 (5%) 666 (8%)

PRIMARY INSURANCE PROVIDER

Private 14 (21%) 622 (16%) 0.35 49 (47%) 4516 (53%) 0.44 0.0001^b

Medicaid 21 (31%) 1085 (28%) 13 (12%) 856 (10%)

Medicare 30 (44%) 1712 (44%) 42 (40%) 2976 (35%)

Uninsured 3 (4%) 425 (11%) 0 (0%) 52 (1%)

Other 0 (0%) 43 (1%) 0 (0%) 80 (1%)

a. Race/Ethnicity groups were collapsed into White versus Other for statistical testing due to low numbers.

b. Fisher's Exact test.

^{^}
A purposive sample of patients who completed the pre-visit questionnaire and were randomly selected from providers who had received at least 3 completed pre-visit questionnaires.

* Age, sex, language, race/ethnicity, insurance from administrative data. Percentages for each demographic category may not add to 100% due to rounding.

^#
Clinic visit data for all patients with one or more clinic visits during the one-year study period.

We reviewed 172 patient responses to the pre-visit questionnaire (68 safety-net and 104 non-safety-net; Table 2). Of these, 10 patients answered only the history item, 11 answered only the goals question, and 151 answered both. The average length of the interval history responses was 34 words at the safety-net clinic (median: 18.50, standard deviation: 41.96) and 29 words at the non-safety-net clinic (median: 14, standard deviation: 37.71). Responses to the interval history in both clinics were most often about physical symptoms, followed by a medical history update, life changes, mental health issues, and social issues. At least half of the interim history responses included one or more history elements. For the visit goals, patients from both clinics wrote approximately 20 words in their responses (safety-net clinic: median: 6, standard deviation: 36.2; non-safety-net clinic: median: 10, standard deviation: 29.1) and had an average of two goals for the clinic visit (safety-net clinic: median: 2, standard deviation: 1.5; non-safety-net clinic: median: 1, standard deviation: 1.1). There were no statistically significant differences in the patient-typed content between the two clinics (p > 0.05). Tables 3 and 4 show example patient responses to the pre-visit questionnaire.

Table 2.
Characteristics of Sample of Patient Responses to the Pre-Visit Questionnaire in a Safety-net and a Non-Safety-net Clinic.

Safety-net Clinic Non-safety-net Clinic

N (min, max) % N (min, max) % p-value

SAMPLE QUESTIONNAIRE DATA

Questionnaires reviewed for sample* 68 104

Questionnaires with both questions completed 61 90% 90 87% 0.55

Total # primary care providers involved in the care of the respondents 34 52

QUESTION 1 - INTERIM HISTORY

Patient wrote interim history^† 66 97% 95 91% 0.20

Average word count 34 (0, 177) 29 (0, 197) 0.44

Themed content of interval history^‡

Report symptoms 42 64% 59 62% 0.84

Medication related 22 33% 23 24% 0.20

Life changes/habits 11 17% 18 19% 0.71

Minimal response 16 24% 19 20% 0.52

Mental health issues 8 12% 9 9% 0.59

Social issues 3 5% 4 4% 1.00^a

Number of OLD CARTS history elements in interim history^§

Zero 33 50% 45 47% 0.18^a

1–3 28 42% 48 51%

4 + 5 8% 2 2%

QUESTION 2 - VISIT GOALS

Patient wrote one or more visit goals^‖ 63 93% 99 95%

Average word count 20 (0, 181) 19 (0, 197) 0.97

Average number of visit goals in Q2 2.03 (0, 6) 1.75 (0, 5) 0.20

Primary Care Provider Use of Patient responses (Interim History and Goals)

Patient completed questionnaire and attended the visit ^# 56 97

PCP added patient's text to the visit note^¶

Yes 39 70% 60 62% 0.33

No 17 30% 37 38%

PCP documentation addressed all of patient's goals for the visit (Q2)

Yes 38 68% 81 84% 0.0025

No 18 32% 16 16%

Number of interim histories (Q1) left blank = 2 (safety-net clinic), 9 (non-safety-net clinic). Number of visit goals (Q2) left blank = 5 (safety-net clinic), 5 (non-safety-net clinic).

†Some responses were left blank by patients.

^‡
Could be more than one per questionnaire.

^§
OLD CARTS stands for onset, location, duration, characteristics, aggravating/alleviating factors, radiation, timing, and severity and represent descriptions that patients could provide in their questionnaire to better characterize their symptoms or illness. There are a total of 8 OLD CARTS criteria and a patient could have provided no OLD CARTS descriptions in their questionnaire responses or up to 8. Patients answers were categorized as zero, 1–3, or 4 + , indicating no OLD CARTS descriptors were included in questionnaire responses, at least one or up to three OLD CARTS descriptors were included in questionnaire responses, and 4 or greater (up to 8) OLD CARTS descriptors were included in questionnaire responses.

^‖
Some responses were left blank by patients.

^¶
Both interim history (Q1) and visit goals (Q2) included.

^#
Twelve patients at the safety-net-clinic and 7 at the non-safety-net clinic who completed a pre-visit questionnaire did not attend their appointment; thus, the denominators are 56 at the safety-net and 97 at the non-safety-net clinics for day of visit metrics.

^a.
Fisher's Exact test.

Table 3.
Interim History Examples of Each Theme.

Theme Example quote from interim history

Report symptoms Lower back pain has gotten better, middle back feels out of alignment quite often. New symptoms of rib pain in past week. (safety-net)*

coughing and lots and I mean lots of flim to the point I throw up and getting worse as each day goes by also it is causing me to have short breaths hard to breath with the flim (non-safety-net)

Good energy but blood pressure fluctuations (non-safety-net)

Medication related I’m Nauseated from the meds. (safety-net)

I stopped taking the victoza and am taking my Levemir, 30 units in the morning as well as 30 units in the evening. (safety-net)

I am taking Daily Essential Nutrients vitamins from Canada and am reducing my regular psychiatric medications as prescribed by XX. (non-safety-net)

Life changes/habits I've had more fright so that makes me more sedentary. (safety-net)

I need to lose more weight and I'm having difficulties eating balanced meals and losing weight while trying to get in enough food to take the vitamins I am taking. (non-safety-net)

Minimal response About the same as before (safety-net)

I've been feeling fine overall. (non-safety-net)

Mental health issues Just the chance of having cancer has me not sleeping well and feeling down. (safety-net)

Suggestion to cease illicit chemical use. Current use replaces lonliness and lack of companionship. Relationship self esteem suggestions. (safety-net)

I have my good days and bad days regarding depression and anxiety, but overall I'm stable. Anxiety is more pervasive and is unsurprisingly associated with stress. (non-safety-net)

Social issues I continue to be about as isolated (safety-net)

I've been working 60–70 hour weeks for the last couple of months--a regular job and a side gig. (non-safety-net)

I have not been back to work. (non-safety-net)

Table 4.
Visit Goal Examples.

Example goal(s) Goal(s) addressed in associated visit note

lymphedema - arm and legs/ core strength excessive sleep attacks - probably medicine related (safety-net) All goal(s) addressed

1. Right shoulder pain 2. Apparent weakness in right leg (gluts, quadricep) that limits the distance I can walk (especially uphill). 3. I would like to get a flu shot. (non-safety-net) All goal(s) addressed

follow up on: heart monitor, MRI, ankle x-ray, possibly rheum visit. (non-safety-net) Some goal(s) addressed

1.I would like to focus on the following issues and concerns: overall medical issues mentioned on the above question and to some extent on my back pain. I do have a follow up with my surgeon in February. 2.I would be also discussing some social concerns or needs that might help me ease in my living standards in relation to my health issues. (safety-net) Some goal(s) addressed

is my hip Pain normal. - Sometime back XX prescribed Metforman. I have run out of tablets and wondered should I continue to take it? (safety-net) Goal(s) not addressed

Dizziness (non-safety-net) Goal(s) not addressed

Of those who both completed a pre-visit questionnaire and attended their appointment (n = 56 safety-net, n = 97 non-safety-net), the majority of patient responses were added by the provider to the visit note (70% at safety-net clinic, 62% at the non-safety-net clinic) with no statically significant difference between the two clinics (p > 0.05). Patients at the safety-net clinic were less likely to have all of their goals addressed within the PCP documentation, compared to the non-safety-net clinic: 68% safety-net clinic, 84% non-safety net, a statistically significant difference (p < 0.01; Table 2).

Discussion

Electronic health portals are increasingly utilized for communication between patients and care teams,²⁴ including early trials of co-generation of visit notes.^6,25 In this study comparing use of a novel pre-visit questionnaire asking patients to write an interim history and their goals for the upcoming visit, we found that questionnaires returned in a safety-net clinic and a non-safety net clinic had similar word counts, number of clinical history elements, and number of goals for the clinic visit, and that insurance type was not associated with whether or not a patient returned the questionnaire. However, use of the pre-visit questionnaire was very low at the safety-net clinic. Patients in the safety-net clinic also had fewer of their stated goals addressed in the provider's documentation of the visit, compared to the non-safety-net clinic; there were slightly more goals on average at the safety-net clinic compared to the non-safety-net which may have made it more difficult for all of the goals to be addressed.

Patients crafting their stories in their own words could enable them to feel “heard” by the provider, leave a permanent record of their story in the chart as they perceive it, and could improve clinic note accuracy.²⁶ For patients who are disenfranchised from doctors and the medical system, this could be particularly empowering.⁸ More work will need to be done to determine if this tool improves trust with providers, or if non-digital engagement methods are more effective. Providers may also benefit from the pre-visit information for agenda setting and potentially more streamlined history taking and efficient visits. Our chart review of historical elements in the patient interval histories found that about half of patients submitted one or more historical elements, thus future revision of the questionnaire may be needed to gather more complete historical data. With recent changes to Medicare documentation requirements for billing to reflect medical complexity and time,²⁷ counting elements of history are less important for billing, but patients documenting them could still save provider time and improve documentation quality and related provider burnout.

The COVID-19 pandemic required social distancing for patients with co-morbidities and moved telehealth and remote care via technology to the forefront, with a resulting gap in access for patients from vulnerable populations.²⁸ Studies suggest that underserved populations are highly motivated to adopt new technology, but that access alone is not sufficient and tailoring technologies for users is needed.²⁹ Prior work suggests that accessing health information is more likely if patients from vulnerable populations own smartphones, and that language influences preferences related to accessing electronic data.³⁰ As technology becomes an increasingly important for patient engagement in health care due to the 21st Century Cares Act and the pandemic initiated telehealth options, policies will need to specifically work with vulnerable populations and address barriers to benefiting from these health care advances.

Provider access to patient responses also required technical skills in order to find the questionnaire and add them to the visit note; additional work is needed on the part of the electronic medical record system to improve providers’ ability to merge patient data into clinic notes to ensure ease of use for providers, and to create a work flow that routes the data to providers even if the patient misses the visit. Designing an interface that improves the visibility of the patient responses and ease of adding to the note may require physician training and improved user centered design. This study examined utilization at two different clinics, however, these clinics are associated with a single academic system meaning that our participants represent a small subset of patients and may not be widely generalizable. Similarly, the participant sample may not be representative of all respondents in the larger study. We did not notify patients about the study ahead of time; we simply activated it in the electronic portal and chart; this required patients to have numerous technical skills to participate, including being signed up for the portal, accessing the link from email, and knowing their password to login. We do not have information about patients who did not respond to the questionnaire. The pre-visit questionnaire was also only available in English; future electronic tools should include other languages and advances such as dictation options on smart phones to increase engagement for all populations. The patients included in this study had at least three visits to the healthcare system in the last year, and were greater utilizers of clinic services; however, even within this more engaged cohort, use of this novel communication tool was limited. Further research involving a broader spectrum of patients in varied clinical settings is needed in order to fully understand the optimal roles for co-generation of notes.

In conclusion, this novel questionnaire was utilized less by patients at the safety-net than the non-safety-net clinic. Respondents at the two clinics were more similar than different, with White patients more likely to use the tool. Developing electronic patient engagement tools that are accessible to non-English speaking patients, and investing in human resources, such as patient care partners or cultural and/or linguistic peer navigators, as well as improving provider training and access to patient responses like these, are opportunities that should be studied. As health care increasingly depends upon remote access to clinical teams and health care information via electronic portals, it is imperative that we invest in mitigating digital disparities.

	Safety-Net Clinic	Non-safety-net clinic
AGE
18–44	14 (21%)	766 (20%)	0.91	35 (34%)	2659 (31%)	0.08	0.001
45–64	33 (48%)	1824 (47%)		23 (22%)	2715 (32%)
≥ 65	21 (31%)	1295 (33%)		46 (44%)	3106 (37%)
SEX
Female	34 (50%)	1747 (45%)	0.41	46 (44%)	3896 (46%)	0.73	0.46
Male	34 (50%)	2140 (55%)		58 (56%)	4583 (54%)
PRIMARY LANGUAGE
English	68 (100%)	3011 (77%)	<.0001	104 (100%)	8075 (95%)	0.02^b	-
Other	0 (0%)	876 (23%)		0 (0%)	405 (5%)
RACE/ETHNICITY
White	51 (75%)	1494 (38%)	<.0001^a	83 (80%)	5956 (70%)	0.03^a	0.46^a
Black or African American	12 (18%)	1176 (30%)		5 (5%)	461 (5%)
Hispanic	0 (0%)	525 (13%)		0 (0%)	307 (4%)
Asian	1 (1%)	440 (11%)		9 (9%)	924 (11%)
American Indian or Alaska Native	1 (1%)	50 (1%)		1 (1%)	41 (0.5%)
Native Hawaiian or Other Pacific Islander	1 (1%)	49 (1%)		1 (1%)	39 (0.5%)
Multiple Races	0 (0%)	31 (1%)		0 (0%)	86 (1%)
Declined to Answer/ Unavailable/Unknown	2 (3%)	122 (3%)		5 (5%)	666 (8%)
PRIMARY INSURANCE PROVIDER
Private	14 (21%)	622 (16%)	0.35	49 (47%)	4516 (53%)	0.44	0.0001^b
Medicaid	21 (31%)	1085 (28%)		13 (12%)	856 (10%)
Medicare	30 (44%)	1712 (44%)		42 (40%)	2976 (35%)
Uninsured	3 (4%)	425 (11%)		0 (0%)	52 (1%)
Other	0 (0%)	43 (1%)		0 (0%)	80 (1%)

	Safety-net Clinic	Non-safety-net Clinic
	N (min, max)	%	N (min, max)	%	p-value
SAMPLE QUESTIONNAIRE DATA
Questionnaires reviewed for sample*	68		104
Questionnaires with both questions completed	61	90%	90	87%	0.55
Total # primary care providers involved in the care of the respondents	34		52
QUESTION 1 - INTERIM HISTORY
Patient wrote interim history^†	66	97%	95	91%	0.20
Average word count	34 (0, 177)		29 (0, 197)		0.44
Themed content of interval history^‡
Report symptoms	42	64%	59	62%	0.84
Medication related	22	33%	23	24%	0.20
Life changes/habits	11	17%	18	19%	0.71
Minimal response	16	24%	19	20%	0.52
Mental health issues	8	12%	9	9%	0.59
Social issues	3	5%	4	4%	1.00^a
Number of OLD CARTS history elements in interim history^§
Zero	33	50%	45	47%	0.18^a
1–3	28	42%	48	51%
4 +	5	8%	2	2%
QUESTION 2 - VISIT GOALS
Patient wrote one or more visit goals^‖	63	93%	99	95%
Average word count	20 (0, 181)		19 (0, 197)		0.97
Average number of visit goals in Q2	2.03 (0, 6)		1.75 (0, 5)		0.20
Primary Care Provider Use of Patient responses (Interim History and Goals)
Patient completed questionnaire and attended the visit ^#	56		97
PCP added patient's text to the visit note^¶
Yes	39	70%	60	62%	0.33
No	17	30%	37	38%
PCP documentation addressed all of patient's goals for the visit (Q2)
Yes	38	68%	81	84%	0.0025
No	18	32%	16	16%

Theme	Example quote from interim history
Report symptoms	Lower back pain has gotten better, middle back feels out of alignment quite often. New symptoms of rib pain in past week. (safety-net)
coughing and lots and I mean lots of flim to the point I throw up and getting worse as each day goes by also it is causing me to have short breaths hard to breath with the flim (non-safety-net)
Good energy but blood pressure fluctuations (non-safety-net)
Medication related	I’m Nauseated from the meds. (safety-net)
I stopped taking the victoza and am taking my Levemir, 30 units in the morning as well as 30 units in the evening. (safety-net)
I am taking Daily Essential Nutrients vitamins from Canada and am reducing my regular psychiatric medications as prescribed by XX. (non-safety-net)
Life changes/habits	I've had more fright so that makes me more sedentary. (safety-net)
I need to lose more weight and I'm having difficulties eating balanced meals and losing weight while trying to get in enough food to take the vitamins I am taking. (non-safety-net)
Minimal response	About the same as before (safety-net)
I've been feeling fine overall. (non-safety-net)
Mental health issues	Just the chance of having cancer has me not sleeping well and feeling down. (safety-net)
Suggestion to cease illicit chemical use. Current use replaces lonliness and lack of companionship. Relationship self esteem suggestions. (safety-net)
I have my good days and bad days regarding depression and anxiety, but overall I'm stable. Anxiety is more pervasive and is unsurprisingly associated with stress. (non-safety-net)
Social issues	I continue to be about as isolated (safety-net)
I've been working 60–70 hour weeks for the last couple of months--a regular job and a side gig. (non-safety-net)
I have not been back to work. (non-safety-net)

Example goal(s)	Goal(s) addressed in associated visit note
lymphedema - arm and legs/ core strength excessive sleep attacks - probably medicine related (safety-net)	All goal(s) addressed
1. Right shoulder pain 2. Apparent weakness in right leg (gluts, quadricep) that limits the distance I can walk (especially uphill). 3. I would like to get a flu shot. (non-safety-net)	All goal(s) addressed
follow up on: heart monitor, MRI, ankle x-ray, possibly rheum visit. (non-safety-net)	Some goal(s) addressed
1.I would like to focus on the following issues and concerns: overall medical issues mentioned on the above question and to some extent on my back pain. I do have a follow up with my surgeon in February. 2.I would be also discussing some social concerns or needs that might help me ease in my living standards in relation to my health issues. (safety-net)	Some goal(s) addressed
is my hip Pain normal. - Sometime back XX prescribed Metforman. I have run out of tablets and wondered should I continue to take it? (safety-net)	Goal(s) not addressed
Dizziness (non-safety-net)	Goal(s) not addressed

Footnotes

Acknowledgments

The authors do not report any financial conflicts of interest. This study was supported by The Commonwealth Fund and the Gordon and Betty Moore Foundation. The authors wish to thank Megan Eguchi for her assistance with statistical testing and all of the patients and providers who participated in this study for their time and effort.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Commonwealth Fund, Gordon and Betty Moore Foundation,

ORCID iD

Sara L. Jackson

References

Stewart

Brown

Donner

, et al. The impact of patient-centered care on outcomes. J Fam Pract. 2000;49(9):796–804.

Delbanco

Walker

Darer

, et al. Open notes: doctors and patients signing on. Ann Intern Med. 2010;153(2):121-125.

Finkelstein

Knight

Marinopoulos

, et al. Enabling patient-centered care through health information technology. Evid Rep Technol Assess (Full Rep). 2012;206:1-1531.

Robinson

Heritage

. Physicians’ opening questions and patients’ satisfaction. Patient Educ Couns. 2006;60(3):279-285.

Wald

Businger

Gandhi

, et al. Implementing practice-linked pre-visit electronic journals in primary care: patient and physician use and satisfaction. J Am Med Inform Assoc. 2010;17(5):502-506.

Anderson

Jackson

Oster

, et al. Patients typing their own visit agendas into an electronic medical record: pilot in a safety-net clinic. Ann Fam Med. 2017.

Mafi

Gerard

Chimowitz

Anselmo

Delbanco

Walker

. Patients contributing to their doctors’ notes: insights from expert interviews. Ann Intern Med. 2018;168(4):302-305.

Walker

Leveille

Bell

, et al. Opennotes after 7 years: patient experiences with ongoing access to their clinicians’ outpatient visit notes. J Med Internet Res. 2019;21(5):e13876.

Jaspers

Cox

Krumholz

. Copy fees and limitation of patients’ access to their own medical records. JAMA Intern Med. 2017;177(4):457-458.

10.

Lye

Forman

Gao

, et al. Assessment of US hospital compliance with regulations for patients’ requests for medical records. JAMA Netw Open. 2018;1(6):e183014.

11.

Mishra

Hoyt

Wolver

Yoshihashi

Banas

. Qualitative and quantitative analysis of patients’ perceptions of the patient portal experience with OpenNotes. Appl Clin Inform. 2019;10(1):10-18.

12.

O'Neill

Chimowitz

Leveille

Walker

. Embracing the new age of transparency: mental health patients reading their psychotherapy notes online. J Ment Health. 2019;28(5):527-535.

13.

Garrido

Kanter

Meng

, et al. Race/ethnicity, personal health record access, and quality of care. Am J Manag Care. 2015;21(2):e103-e113.

14.

IT OotNCfH. 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program. https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification. Published 2020. Accessed January 25, 2021.

15.

Avdagovska

Menon

Stafinski

. Capturing the impact of patient portals based on the quadruple aim and benefits evaluation frameworks: scoping review. J Med Internet Res. 2020;22(12):e24568.

16.

Irizarry

DeVito Dabbs

Curran

. Patient portals and patient engagement: a state of the science review. J Med Internet Res. 2015;17(6):e148.

17.

Chakkalakal

Kripalani

Schlundt

Elasy

Osborn

. Disparities in using technology to access health information: race versus health literacy. Diabetes Care. 2014;37(3):e53-e54.

18.

Han

Sunderland

Kendall

Gudes

Henniker

. Professional practice and innovation: chronic disease, geographic location and socioeconomic disadvantage as obstacles to equitable access to e-health. Health Inf Manag. 2010;39(2):30-36.

19.

Goel

Brown

Williams

Hasnain-Wynia

Thompson

Baker

. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med. 2011;26(10):1112-1116.

20.

Peacock

Reddy

Leveille

, et al. Patient portals and personal health information online: perception, access, and use by US adults. J Am Med Inform Assoc. 2017;24(e1):e173-e177.

21.

Yamin

Emani

Williams

, et al. The digital divide in adoption and use of a personal health record. Arch Intern Med. 2011;171(6):568-574.

22.

Bickley

Hoekelman

Bates

. Bates’ Pocket Guide to Physical Examination and History Taking. Lippincott, Williams & Wilkins; 2000.

23.

Seidel

. Mosby's Guide to Physical Examination. Mosby/Elsevier; 2011.

24.

HHS Finalizes Historic Rules to Provide Patients More Control of Their Health Data [press release]. March 9, 2020 2020.

25.

Walker

Leveille

Kriegel

, et al. Patients contributing to visit notes: mixed methods evaluation of OurNotes. J Med Internet Res. 2021;23(11):e29951.

26.

Bell

Delbanco

Elmore

, et al. Frequency and types of patient-reported errors in electronic health record ambulatory care notes. JAMA Netw Open. 2020;3(6):e205867.

27.

Evaluation and Management Services. In: Services CfMaM, ed2021.

28.

Thronson

Jackson

Chew

. The pandemic of health care inequity. JAMA Netw Open. 2020;3(10):e2021767.

29.

Huh

Koola

Contreras

, et al. Consumer health informatics adoption among underserved populations: thinking beyond the digital divide. Yearb Med Inform. 2018;27(1):146-155.

30.

Khoong

Hoskote

Rivadeneira

Hiatt

Sarkar

. Health information-seeking behaviors and preferences of a diverse, multilingual urban cohort. Med Care. 2019;57(Suppl 6 2):S176-s183.

	Safety-Net Clinic			Non-safety-net clinic
	Participant sample^{^} (n = 68)	Overall Clinic Population^# (n = 3887)	p-value comparing sample to Overall	Participant sample^{^} (n = 104)	Overall Clinic Population^# (n = 8480)	p-value comparing sample to Overall	p-value comparing Participant samples
AGE
18–44	14 (21%)	766 (20%)	0.91	35 (34%)	2659 (31%)	0.08	0.001
45–64	33 (48%)	1824 (47%)		23 (22%)	2715 (32%)
≥ 65	21 (31%)	1295 (33%)		46 (44%)	3106 (37%)
SEX
Female	34 (50%)	1747 (45%)	0.41	46 (44%)	3896 (46%)	0.73	0.46
Male	34 (50%)	2140 (55%)		58 (56%)	4583 (54%)
PRIMARY LANGUAGE
English	68 (100%)	3011 (77%)	<.0001	104 (100%)	8075 (95%)	0.02^b	-
Other	0 (0%)	876 (23%)		0 (0%)	405 (5%)	0.02^b	-
RACE/ETHNICITY
White	51 (75%)	1494 (38%)	<.0001^a	83 (80%)	5956 (70%)	0.03^a	0.46^a
Black or African American	12 (18%)	1176 (30%)		5 (5%)	461 (5%)
Hispanic	0 (0%)	525 (13%)		0 (0%)	307 (4%)
Asian	1 (1%)	440 (11%)		9 (9%)	924 (11%)
American Indian or Alaska Native	1 (1%)	50 (1%)		1 (1%)	41 (0.5%)
Native Hawaiian or Other Pacific Islander	1 (1%)	49 (1%)		1 (1%)	39 (0.5%)
Multiple Races	0 (0%)	31 (1%)		0 (0%)	86 (1%)
Declined to Answer/ Unavailable/Unknown	2 (3%)	122 (3%)		5 (5%)	666 (8%)
PRIMARY INSURANCE PROVIDER
Private	14 (21%)	622 (16%)	0.35	49 (47%)	4516 (53%)	0.44	0.0001^b
Medicaid	21 (31%)	1085 (28%)		13 (12%)	856 (10%)
Medicare	30 (44%)	1712 (44%)		42 (40%)	2976 (35%)
Uninsured	3 (4%)	425 (11%)		0 (0%)	52 (1%)
Other	0 (0%)	43 (1%)		0 (0%)	80 (1%)

Clinical Use of an Electronic Pre-Visit Questionnaire Soliciting Patient Visit Goals and Interim History: A Retrospective Comparison Between Safety-net and Non-Safety-net Clinics

Abstract

Introduction/Objectives

Methods

Results

Conclusions

Keywords

Methods

Results

Discussion

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iD

References