Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting

Abstract

French

Objective

This study aims to understand whether higher use of a patient portal can have an impact on mental health functioning and recovery.

Method

A mixed methods approach was used for this study. In 2019–2021, patients with mental health diagnoses at outpatient clinics in an academic centre were invited to complete World Health Organization Disability Assessment Scale 12 (WHODAS-12) and Mental Health Recovery Measure surveys at baseline, 3 months, and 6 months after signing up for the portal. At the 3-month time point, patients were invited to a semistructured interview with a member of the team to contextualize the findings obtained from the surveys. Analytics data was also collected from the platform to understand usage patterns on the portal.

Results

Overall, 113 participants were included in the analysis. There was no significant change in mental health functioning and recovery scores over the 6-month period. However, suboptimal usage was observed as 46% of participants did not complete any tasks within the portal. Thirty-five participants had low use of the portal (1–9 interactions) and 18 participants had high usage (10+ interactions). There were also no differences in mental health functioning and recovery scores between low and high users of the portal. Qualitative interviews highlighted many opportunities where the portal can support overall functioning and mental health recovery.

Conclusions

Collectively, this study suggests that higher use of a portal had no impact, either positive or negative, on mental health outcomes. While it may offer convenience and improved patient satisfaction, adequate support is needed to fully enable these opportunities for patient care. As the type of interaction with the portal was not specifically addressed, future work should focus on looking at ways to support patient engagement and portal usage throughout their care journey.

Keywords

patient portals clinical informatics nursing informatics evaluation mental health

Introduction

Since 2010, there has been a proliferation of health information shared with patients through a patient portal.^1,2 Many platforms allow patients and families to perform various tasks including scheduling appointments, viewing lab results and reviewing clinical documentation.^3,4 Since the COVID-19 pandemic, patient portals have become an instrumental tool for communicating polymerase chain reaction (PCR) test results to confirm diagnoses for patients and families.^5-7 This has led to exponential adoption and use of portals in primary and acute care settings.⁸ Particularly, in the US, the 21st Century Cures Act has mandated the sharing of clinical documentation with patients and families.^9-12

While many studies have yielded important lessons for successful portal implementations in primary and acute care settings (e.g., the importance of engaging patients and families and clinical workflows),^8,13-15 in the mental health domain, it is still relatively uncommon to share health information with patients and families due to stigma and impact on therapy.¹⁶ Despite growth in this area,^17-20 there has been significant concern and apprehension around sharing notes with patients. In particular, there are perceptions among healthcare professionals that information sharing may ruin therapy and negatively impact recovery.^{12,16,19,21-23} While Canada does not have a national policy such as the “Cures Act” in the United States to proactively share clinical notes with patients, the clinician concerns noted above may further impact the adoption of patient portals.^23-25

Despite ongoing efforts to evaluate the impact of patient portals on mental health care, the current literature lacks evidence on whether the patient portal is detrimental to therapy and/or has a negative impact on mental healthcare.¹⁷ More importantly, little is known about the impact of the portal on patients’ mental health recovery.^18,26 A recent literature review on the impact and benefits of patient portals has found insufficient evidence to understand their impact on clinical care.²⁷ Providing quantitative and qualitative data from real-world environments can help validate some of these concerns and inform future efforts to support the sustainable scale-up of this tool. Thus, the present work focuses on examining the impact the use of a patient portal has on mental healthcare and the drawbacks that clinicians fear. The specific aims are as follows:

Aim 1: Determine if higher use of mental health patient portals is associated with changes in mental health functioning and recovery over a 6-month period.

Aim 2: Determine whether the higher usage of the mental health patient portal affected the overall trajectory of mental health functioning and recovery.

Aim 3: Understand the perceptions and experiences of patients on the use of a mental health patient portal for supporting mental health functioning and recovery.

Methods

A convergent, concurrent mixed methods approach (QUAN + qual)²⁸ was used to examine the relationship between a patient portal and mental health functioning and recovery. Further details can be found in the protocol paper published in BMJ Open.²⁹ This work is informed by the Canadian Institute of Health Research Strategy for Patient-Oriented Research Framework,³⁰ which outlines key principles for engaging patients to improve care delivery through research. The study was approved by the Research Ethics Board at the Centre for Addiction and Mental Health (REB 044/2018) and the University of Toronto (REB No. 40342). A patient and family advisory committee was engaged throughout the study.

Setting

This study was conducted at a large academic and teaching mental health hospital in Toronto, Ontario. The organization implemented a comprehensive electronic health record in 2014 and achieved Stage 7 on the HIMSS Electronic Medical Record Adoption Model.³¹ In 2018, the hospital launched its mental health patient portal to several outpatient clinics. The mental health patient portal allows patients to (a) view parts of their health record (e.g., discharge notes) and lab tests; (b) complete self-assessments; (c) attend virtual care appointments; and (d) message their care provider. Patients are offered portal access by their clinician or administrative staff and are free to choose whether or not they want to use the patient portal for their care.

Participants

Patients receiving care at 1 or more outpatient clinics offering the patient portal were invited to the study from June 2019 to April 2021. When patients signed up for the portal, they received a pamphlet about the portal including a flyer to the study. Patients could also sign-up for the study on the registration page and patient portal home page. To be eligible for the study, the patient must have self-reported signing up for the portal within the last 2 weeks of enrolling in the study. There were no exclusion criteria based on diagnosis or demographics. Based on the sample size calculation outlined in the protocol, about 100 patients (accounting for ∼30% attrition) were required for the study.²⁹ A sample size of N = 68 completers achieved 80% power (alpha = 5%) for detecting a 30% drop in World Health Organization Disability Assessment Scale 2.0 (WHODAS 2.0) from baseline to T2 (equivalent to a small to medium effect size of Cohen's d = 0.27). All participants provided informed consent in written form as outlined in the REDCap eConsent recommendations.^32,33

Data Collection

After enrolling in the study, the quantitative strand of the study began. The quantitative strand involved completing survey instruments online using REDCap at baseline, 3-month and 6-month mark. After consent, they were asked to complete a baseline package asking them about their demographics, which include sex, primary diagnosis, and ethnicity, among others. After completing the demographics section, participants were asked to complete 2 survey instruments at each of the 3 time points. The WHODAS 2.0—Short Version³⁴ and the Mental Health Recovery Measure (MHRM)³⁵ were used to assess mental health functioning and recovery respectively. Both of these instruments are scales frequently used in both clinical care and mental health research.^34,36-38 Details about the domains and reliability of these instruments can be found in the protocol.²⁹

In addition to collecting survey data for the quantitative strand, usage data^39-41 on the mental health patient portal were collected from the back-end platform throughout the 6 months of the study. This includes the duration, frequency and usage of the different functionalities of the portal.^42,43 The usage data was validated internally as part of the implementation process and was matched based on the patient's name reported in the study. As part of the analysis, participants who have used the portal were stratified into low (1–9 interactions) and high usage (10+ interactions) for the analysis.

For the qualitative strand, at the 3-month mark, participants were invited to complete a 1-hour semistructured virtual interview⁴⁴ with a graduate student in health informatics (BL, JK or MM) using a qualitative descriptive approach.⁴⁵ Participants were asked to share their experience using the portal, their perceptions on how the portal has impacted their mental health functioning and recovery journey, and challenges that hindered the value and benefits of the patient portal, among other outcomes. The question guide (Supplemental Appendix A) builds on the existing literature^17,18,46,47 and the needs suggested by our Patient and Family Advisory Council. Participants were invited to the qualitative strand until data saturation⁴⁸ was achieved.

Data Analysis

The quantitative analysis was carried out using SAS Enterprise Guide v7.1 (SAS Institute, Cary, NC, USA). Descriptive statistics (mean, standard deviation, frequency, and proportion) were used to summarize participant characteristics. For each study outcome, linear mixed effect models with random participant effects were performed to model the change in outcome over the 3 time points.⁴⁹ Linear contrasts were conducted to compare across the 3 time points. While the main analysis was not adjusted, an additional exploratory analysis adjusting for sex, age and diagnosis was conducted. Missing data was imputed using SAS PROC MI at the item level. A total of 5 imputed datasets were generated and SAS PROC MIANALYZE was used to aggregate the findings. Two-sided p-values < .05 were considered statistically significant.

For the analytics data, a doctoral student (BL) familiar with analytics and user engagement led the data cleaning and analysis. Given that usage data is considered approximate and dirty data (flawed, uncertain, proximate and sparse [FUPS]),⁵⁰ any redundant data entries were removed. In order to segregate the users across low and high users, the total number of interactions related to the main use cases of the portal was summated. This includes the: (a) number of appointments viewed, (b) clinical notes downloaded/viewed, (c) number of lab results viewed, (d) self-assessments completed, and (e) number of messages reviewed and sent. The trajectory in functioning and mental health recovery between low and high users was then compared by stratifying it based on the number of interactions with the portal.

The qualitative data was analyzed by JK and HDS using the content analysis protocol outlined by Elo and Kyngäs⁵¹ using a deductive approach. JK and HDS deductively analysed the same constructs asked in the MHRM and developed new codes inductively when appropriate. In order to ensure trustworthiness⁵² of the analysis, the analysis was done in duplicate independently (JK and DS) and an audit trail of the discussion and decision points was maintained.

Results

A total of 113 participants were enrolled and consented to participate in the study. Overall, most participants identified as male sex at birth (68%), between 25 and 64 years old (34%), never married (55%), and Caucasian or European (68%) (Table 1). With regard to the most responsible psychiatric diagnoses, 38% of participants indicated they had a mood disorder (e.g., depression and bipolar disorder); 31% had a diagnosis other than mood disorders, anxiety or psychotic disorders.

Table 1.

Baseline Characteristics for Participants (n = 113).

Characteristic	Response	Frequency (%)
Sex	Female	33 (29%)
	Male	77 (68%)
	Prefer not to answer	3 (3%)
Age	Less than 25 years old	39 (34%)
	25–64 years old	70 (62%)
	Over 65 years old	4 (3%)
Marital status	Divorced	11 (10%)
	Married/domestic partnership/common law	35 (31%)
	Never married	62 (55%)
	Widowed	3 (3%)
	Prefer not to answer	2 (2%)
Ethnicity	Black or African American	8 (7%)
	Caucasian or European	76 (68%)
	East Asian	10 (9%)
	Hispanic or Latino	3 (3%)
	Indigenous/First Nations	2 (2%)
	Mixed Heritage	4 (4%)
	South Asian	6 (5%)
	Other	4 (4%)
Most responsible psychiatric diagnosis	Anxiety	21 (19%)
	Mood disorders (e.g., depression)	43 (38%)
	Schizophrenia	5 (4%)
	Other	35 (31%)
	Prefer not to answer	9 (8%)
Access to internet	Daily	112 (99%)
Access to internet	Weekly	1 (1%)
Level of support for condition from friends and family (mean, standard deviation)		57 (31)

Quantitative Analysis

Mental Health Functioning

In the unadjusted model of WHODAS over time (Table 2 and Figure 1A), there was a decreasing trend in WHODAS score from baseline to 3 months (mean difference = −1.26; 95% confidence interval [CI], −2.70 to 0.19; p-value = .09). There were nonsignificant decreases in WHODAS from baseline to 6 months (mean difference = −1.07; 95% CI, −2.56 to 0.42; p-value = .16). The model adjusted for sex, age and diagnosis showed similar results.

Figure 1.

Estimated marginal means in WHODAS (left) and MHRM (right) scores over the 3 time periods, based on the imputed dataset. The confidence intervals denote the 95% confidence interval. Note. WHODAS = World Health Organization Disability Assessment Scale; MHRM = Mental Health Recovery Measure.

Table 2.

Overall Models for Mental Health Functioning and Recovery.

Variable	Estimate (SD)	Difference from baseline (T0)
Variable	Estimate (SD)	Estimated difference from baseline (95% CI)	P value
Primary outcome: Mental health functioning (WHODAS)
Imputed model:
T0 (baseline)	19.97 (1.01)	—	—
T1	18.72 (1.07)	−1.26 (−2.70 to 0.19)	0.09
T2	18.91 (1.08)	−1.07 (−2.56 to 0.42)	0.16
Imputed model adjusting for sex, age and diagnosis:
T0 (baseline)	17.68 (2.92)	—	—
T1	16.45 (2.94)	−1.23 (−2.68 to 0.22)	0.10
T2	16.63 (2.94)	−1.06 (−2.55 to 0.43)	0.16
Secondary outcome: MHRM
Imputed model:
T0 (baseline)	57.30 (2.01)	—	—
T1	59.10 (2.12)	1.81 (−1.15 to 4.76)	0.23
T2	56.81 (2.19)	−0.49 (−3.59 to 2.60)	0.75
Imputed model adjusting for sex, age and diagnosis:
T0 (baseline)	63.50 (5.69)	—	—
T1	65.26 (5.71)	1.76 (−1.19 to 4.71)	0.24
T2	62.96 (5.72)	−0.54 (−3.63 to 2.56)	0.73

Note. WHODAS = World Health Organization Disability Assessment Scale; MHRM = Mental Health Recovery Measure; SD = standard deviation; CI = confidence interval.

Mental Health Recovery

A total of 110 participants were included in the analysis of MHRM scores (3 participants did not complete the baseline and/or subsequent MHRM surveys). Based on the model with time as an independent variable (Figure 1B), there was no significant difference in MHRM scores between baseline and 3 months (mean difference = 1.20; 95% CI, 1.77 to 1.86; p-value = .23). There were also no differences between baseline and 6-months (mean difference = −0.31; 95% CI, −0.86 to −0.23; p-value = .76). Similar findings were obtained after adjusting for sex, age and diagnosis.

Impact of Portal Usage on Functioning and Recovery

Usage data was retrieved for 99 participants (87%). Some participants’ usage data could not be retrieved due to the use of a different name in the patient portal account. In terms of usage, about 46 participants (46%) did not complete a task in the portal (e.g., viewed a note and completion of a self-assessment). Thirty-five participants (35%) had low usage of the portal (1–9 interactions) and 18 participants (18%) had high usage, which is defined as more than 10 interactions. In terms of interactions, the majority comprised self-assessments (311 interactions from 16 users), followed by viewing upcoming appointments (133 interactions from 33 users), messages (58 interactions from 18 users), notes (56 interactions from 22 users) and labs (45 from 5 users). In the model with time adjusted for sex, age and diagnosis, there were no significant differences between participants with low usage and those with high usage on mental health functioning score, F(2,85) = 0.36, p = .70, and mental health recovery score, F(2,83) = 1.37, p = .26.

Qualitative Analysis

Of the participants who completed the survey, 10 participants also completed the interview portion of the study. Overall, the majority of participants were male, 25–64 years old with a mood disorder and had interacted with the portal (see Supplemental Appendix B). A summary of the themes and representative quotes is listed in Table 3 and in detail below.

Table 3.

Main Themes and Example Quotes from the Qualitative Interviews.

Theme	Participant quote
Overcoming stuckness through engagement with the portal	Quote S1: “Improved efficiency can improve the client care interaction because speedy information is always, the faster the better.”—Participant 3 Quote S2: “It would have been nice to see the notes of the doctors. Because I know they’re so busy there, so I don’t feel like I always get my questions answered. It would be nice to have that to look back and say, oh, okay, so that's what he was thinking about.”—Participant 6 Quote S3: “[It was] almost like they didn’t care as much just because they're not using [the portal]… Because I was filling out these mood charts and then [the healthcare provider] would just file them away and I questioned if he was reading them, I questioned if I was wasting my time.”—Participant 1
Basic functioning and self-empowerment for care planning	Quote B1: “If I find out my blood sugar is 11. Then, I say, oh, well, that explains a lot. That could explain why I feel so down and crappy. That explains why I’m confused and forgetful. That's something I need to get addressed… Or if for some reason my pH was off, and I was acidotic. That can really affect you mentally… So, sure, stuff like that could be helpful.”—Participant 5 Quote B2: “It was very descriptive. Because for this specific medication it was very detailed of how it needed to be taken, the dosage. There was a lot of information for it, and both my doctor and I actually talked about how it was so important to get that information to know exactly what to do.”—Participant 11 Quote B3: “So, it was super useful when I needed to know when my appointments were. Now that I have a regular 9:00 a.m. phone call every so often, every couple of weeks, I just kind of rely on it being a regular occurrence. It was very helpful when my mind wasn’t working very well. It was a great aid. The portal was a great aid.”—Participant 2 Quote B4: “For me, I’m always really on top of my blood work. So, to be able to go in, even though results would be normal, I can see where they’re going which I’ve always liked to see.”—Participant 6
Learning and self-redefinition	Quote L1: “I believe as I continue to develop and change and grow as a person it will be useful to reread the consultation notes that's about the interview report when I first came in. It's like a good milestone.”—Participant 10 Quote L2: “[…] there were things that I read with my first of the two appointments within the doctor's notes that I didn’t even fully realize was being said in the appointment. So, that's the main function I used it for, and I found that incredibly helpful.”—Participant 11 Quote L3: “And having a space for the patients to be like well, I disagree and this is how I saw it. So if you do have more than one doctor or if I were to be sharing this information with my family doctor or even friends and family, just being able to have both sides there.”—Participant 12
Impact on overall well-being	Quote W1: “Because of all the intimate details that has been recorded, right? So that's the part that's a bit [stressful], in a sense. Even though in rehab I have gone through these things, reviewing it and revisiting it in my mind feels like replaying the entire negative aspect of my previous life.”—Participant 10 Quote W2: “When you go [onto the portal] and read that they’re writing about how you look, some people might think that doesn’t have anything to do with my mental health, and I think it might make you leery of the [health] professional.” -Participant 1
New potential, advocacy and enrichment	Quote P1: “[…] if I have a space to read [my notes] I can take the time to calm down and I can come back and I have that kind of layer of protection where I don’t have to worry about what [the healthcare provider's] reaction is going to be in the moment. Because one, if it's written down [the healthcare providers] are going to have to be professional either way and it's less intimidating than being face to face.”—Participant 12 Quote P2: “For me, the most interesting conversations were with my family about [the information in the portal]. It helped us come to terms with things and we could talk about it if I were like, this [information in the portal] doesn’t represent what I went through.”—Participant 2

Overcoming Stuckness Through Engagement With the Portal

Individuals described that the portal frequently served as an adjunct tool to support their understanding and treatment process. As outlined by 1 participant (Quote S1), providing information can enhance the efficiency of patients in understanding their care, and serve as another tool to enhance the recovery process. In fact, with the limited time during face-to-face encounters, the ability to review their notes at any time can provide another opportunity to improve the therapeutic relationship and process (Quote S2). However, when the clinician did not use the tool, interested patients may perceive that their clinicians do not support their involvement and engagement in the care. As mentioned by another participant (Quote S3), the lack of interest from clinicians in the use of the tool led to doubt about the clinician's compassion towards the patient. Thus, the patient portal may be seen to connect with the clinician outside of their appointment time, serving as a tool to help patients move forward with recovery.

Basic Functioning and Self-Empowerment for Care Planning

After participants reviewed their health information, they felt that the portal provided them with the ability to have empowerment over their own health data. In Quote B1, understanding their own health can help encourage them to self-reflect on their progress to date and provide possible rationales as to why they may feel a certain way. For some participants, this can encourage them to be in control of their own self-management and recovery. Moreover, the information about medications and appointments can help them understand their care plan. Participants also appreciated having a copy of the medications they need to take on a regular basis, which affirmed the plan they had discussed with their clinician (Quote B2). Similarly, the portal can help them keep track of their appointments and documents they need to read beforehand (Quote B3/B4). Thus, the portal acts as an aid to enhance their independence in adhering to the treatment plan while in the community.

Learning and Self-Redefinition

As part of reading notes, participants outlined the potential of using the portal to learn in terms of who they are and gain knowledge about their mental health. For example, 1 participant (Quote L1) recounted that the ability to review their health information within the portal helped them understand who they are as a person living with a mental illness. Moreover, sometimes, during the visit, it can be hard to interpret all the recommendations and discussions made with the clinician, even with assistance from others (Quote L2). By reviewing the exact language and verbiage that is written in the documentation, it reduces the chance of misinterpreting recommendations from the clinician. Lastly, the ability to review information after the encounter provides patients with the opportunity to reflect on their progress and the space to have disagreements with clinicians about their care. One participant (Quote L3) recounted how being able to review their notes allowed them to identify different perspectives on this issue and devise a plan to ensure that both perspectives can be visible in future conversations. Collectively, viewing notes in the portal has improved the ability for patients to learn and grow about their own mental health journey and care.

Impact on Overall Well-Being

While many individuals had positive experiences with the patient portal, some participants were also able to identify situations where the lack of appropriate support can hinder their well-being. For example, 1 participant (Quote W1) recounted that depending on the journey when the note was documented, there could be the risk of “replaying” parts of their life that were challenging. These documents may not be in alignment with their perceptions, and, in some cases, such as Quote W2, have language that may be considered offensive (Quote W2). Thus, reading health information can have an impact on their overall wellness.

New Potentials, Advocacy and Enrichment

In addition to reading their own health information for care planning, participants also spoke of how the portal can enhance engagement and greater awareness of their own rights in achieving their care goals. One participant (Quote P1) highlighted that being able to view their notes in a comfortable space at home allowed them to review the milestones and overall progress they have achieved. Moreover, access to their own health information via a portal can offer them the possibility to share their health information more easily and openly with other family members (Quote P2). Collectively, this allows patients to become more empowered and to identify any discrepancies they encounter throughout their care.

Integration of Quantitative and Qualitative Findings

The research team integrated the findings to contextualize the impact of patient portals on mental health functioning and recovery (Table 4). Overall, there was no significant impact of portal use on the mental health functioning and mental health recovery scales. However, there were mixed perceptions identified from patient interviews. A clear promise to enhance mental health functioning and recovery using the portal has been expressed, but patients also highlighted limitations of the portal that hindered their ability to realize the promise and impacted their care. Mixed experiences identified in the qualitative strand partially explain nonsignificant results in the quantitative strand and that the portal does not have a unanimous positive impact.

Table 4.

Integration of the Quantitative and Qualitative Findings.

Outcome	Quantitative findings	Qualitative findings
Mental health functioning	No significant differences over the period of 6 months, except for a slight decrease at the 3-month time point	• Depends on the care journey and support provided for the use of the portal • Providing patients access to their information can enable them to be more empowered in their care, thereby promoting treatment outcomes • However, in some cases, reading notes about their past mental health challenges can be triggering and hinder their overall mental health well-being
Mental health recovery	No significant differences over the period of 6 months	• The portal allows patients to review their care and their progression to date, which can help them identify opportunities to enhance their recovery process • However, the lack of patient-centric language in the notes and support from clinicians can limit the utility of the portal in enhancing mental health recovery

Discussion

To our knowledge, this is one of the first studies outlining concrete evidence of the impact a patient portal implemented in a Canadian mental health hospital has on the functioning and recovery of patients suffering from mental health challenges. Contrary to the concerns outlined in numerous articles,^{12,16,19,21-23} this work has provided the foundation for us to reconsider the potential concerns around portals in mental health settings and think about how we can best position the delivery of patient portals to enhance mental health clinical care outcomes.⁴⁶

There has been significant interest in the impact of a patient portal on mental health functioning. In our study, we found no significant differences over the period of 6 months in mental health functioning, which suggests that patient portals do not negatively affect therapy and mental health care. This work complements the review by Zhang et al.,²⁷ who found the use of patient portals was associated with increased participation in their care (patient activation). Our findings from the qualitative interview with patients may shed some light on the mixed results. While patient portals can support adherence to appointments and medications, there can be difficulties in fully interpreting their health information.²⁷ Thus, delivery of compassionate care through the portal requires exploring ways to ensure that clinical practices are supportive and conducive to the use of the tool in practice.⁵³ Moreover, as outlined in Quote W2, the findings also raise highlight potential communication issues through the portal. While many assessments are considered standard (e.g., mental health status exam), the approach to which the results are written and summarized by clinicians may be perceived as offensive to patients and families.^54,55 While studies on the concept of sharing notes (OpenNotes) have highlighted some strategies to mitigate this including reviewing with patients and setting expectations, further work is necessary to provide feasible approaches to address these challenges within the mental health context.^56,57 The authors are currently conducting a study that focuses on addressing these challenges within the mental health context.⁵⁸ Nevertheless, this work highlights that the portal may offer conveniences and improve satisfaction while not necessarily impacting the quality of care.

In addition, our quantitative strand found that there were no significant changes in recovery scores over time. These findings are in contrast to the findings from a previous study from another Canadian mental health care centre, where they found a significant increase in recovery scores.⁵⁹ Moreover, many interview participants in our current study highlighted the important role of viewing their health information for learning and self-understanding of themselves and their condition. However, the mixed findings in our current study and the suboptimal usage of portals suggest that many of the opportunities to enhance outcomes could be limited if there is insufficient support to enable the benefits. Implementation frameworks such as the Consolidated Framework for Implementation Research⁶⁰ can be useful to support the identification of key facilitators and challenges in supporting the adoption and use of these innovations.

Lastly, it is worth highlighting the suboptimal level of user engagement observed within the portal. While there is no gold standard measuring user engagement,^41,61 the majority of participants had <2 interactions with the portal and only a small fraction of patients used the portal repeatedly. In this context, the low usage by participants may be suggestive of high levels of disengagement with the technologies, thereby leading to a futile impact on change in health behaviour and outcomes.⁶² Unfortunately, disengagement with a digital tool is complex and Perski et al.⁶³ suggest numerous factors related to the tool and environment that can hinder engagement with the tool. Moreover, there has been discussion about the importance of health equity and inclusivity consideration in supporting the use of digital tools.⁶⁴ Thus, future work should uncover factors that influence how individuals remain engaged with portals in mental health settings.

There are also broad policy implications from this work. In Ontario, improving patient access to health information has become a provincial priority and Patient Portal Standards have been developed to ensure that these tools have the appropriate functionalities (e.g., single digital identity) for patients to effectively embed it as part of their care.⁶⁵ These guidelines align closely with the findings from the qualitative findings, in that there is a need to look at how clinic workflow and clinician engagement should be involved in supporting patient access to health information for improved efficiency in care planning. It would also be useful to look at how patients and families can be better supported to bridge the digital divide by using the portal. In the current study, due to the early stages of implementation, there was limited support (other than a technical hotline) being provided to support the challenges (e.g., education) related to digital health equity. As digital health equity^66,67 continues to be a growing concern for usability and scale, looking at interventions that focus on empowering patients and families with the right knowledge and competencies is critical.

Limitations

From a methodological perspective, given that this is an observational study without a comparator group in a single academic Canadian site, it is unclear whether the trajectory of these patients would be different if they had not used the portal. In particular, due to the imbalanced mix of mental health disorders in the sample, limited clinical anchors at baseline and the nature that each patient has a different mental health trajectory, it is likely that we were underpowered to detect any difference among some of the subpopulations. Moreover, as patient portals function within a complex health-care environment, it is unclear how the specific patient portal software and the evolving workflows within the clinical environment contributed to the observed findings. As such, the findings should be considered with studies on clinician perspectives. It is important to note that the study was conducted during the COVID-19 pandemic, which may have influenced how the portal was used by participants. Due to the limitations in sample size and rapidly changing environment, we were unable to add targeted questions specific to the pandemic and compare use before and during the pandemic. For the qualitative interviews, the imbalanced and small sample size also hindered the ability to compare the perceptions and experiences between high and low users and generate useful insights for optimizing the platform.

Conclusion

Over a 6-month time period, there were no significant differences in mental health functioning and recovery, suggesting that the portal does not have a detrimental effect on the treatment and recovery of patients. However, usage of the portal remains suboptimal. From the qualitative findings, numerous opportunities to enhance patients’ mental health function and recovery through portal use are highlighted, but appropriate support for patients and clinicians is required to achieve the portals’ fullest potential and engagement.

Supplemental Material

sj-docx-1-cpa-10.1177_07067437231197060 - Supplemental material for Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting

Supplemental material, sj-docx-1-cpa-10.1177_07067437231197060 for Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting by Brian Lo, Hwayeon Danielle Shin, Jessica Kemp, Mikayla Munnery, Sheng Chen, Clement Ma, Damian Jankowicz, Rohan Mehta, Alexandra Harris, Moshe Sakal, Ryan Pundit, Kevin Chung, Craig Kuziemsky, Sarah Rossetti and Gillian Strudwick in The Canadian Journal of Psychiatry

Supplemental Material

sj-docx-2-cpa-10.1177_07067437231197060 - Supplemental material for Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting

Supplemental material, sj-docx-2-cpa-10.1177_07067437231197060 for Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting by Brian Lo, Hwayeon Danielle Shin, Jessica Kemp, Mikayla Munnery, Sheng Chen, Clement Ma, Damian Jankowicz, Rohan Mehta, Alexandra Harris, Moshe Sakal, Ryan Pundit, Kevin Chung, Craig Kuziemsky, Sarah Rossetti and Gillian Strudwick in The Canadian Journal of Psychiatry

Footnotes

Acknowledgements

The authors of this paper would like to acknowledge the Centre for Addiction and Mental Health for their overall support of this work. In addition, we would like to acknowledge Anna Chyjek, Emily Winters and Uzma Haider for their support in this project.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is supported by a Strategy for Patient-Oriented Research (SPOR) Grant from the Canadian Institutes for Health Research (Funding Reference Number: PEH-157079) as well as a Sigma/Canadian Nurses Foundation Grant from the Sigma Foundation for Nursing. In-Kind contributions were also provided by the Centre for Addiction and Mental Health (CAMH).

ORCID iD

Brian Lo

Supplemental Material

Supplemental material for this article is available online.

References

Lye

Forman

Daniel

Krumholz

. The 21st Century Cures Act and electronic health records one year later: will patients see the benefits? J Am Med Inform Assoc. 2018;25(9):1218-1220.

Mehan

Jr Brink

Hirsch

. 21st Century Cures Act: patient-facing implications of information blocking. J Am Coll Radiol. 2021;18(7):1012-1016.

Alturkistani

Greenfield

Greaves

Aliabadi

Jenkins

Costelloe

. Patient portal functionalities and uptake: systematic review protocol. JMIR Res Protoc. 2020;9(7):e14975.

Ammenwerth

Neyer

Horbst

Mueller

Siebert

Schnell-Inderst

. Adult patient access to electronic health records. Cochrane Database Syst Rev. 2021;2:CD012707.

Turer

DesRoches

Salmi

Helmer

Rosenbloom

. Patient perceptions of receiving COVID-19 test results via an online patient portal: an open results survey. Appl Clin Inform. 2021;12(4):954-959.

Margolius

Hennekes

Yao

, et al. On the front (phone) lines: results of a COVID-19 hotline. J Am Board Fam Med. 2021;34(Suppl):S95-S102.

Imborek

Krasowski

Natvig

Merrill

Diekema

Ford

. Experience with pretravel testing for SARS-CoV-2 at an Academic Medical Center. Acad Pathol. 2021;8:23742895211010247.

Charow

Laberge

Bakas

Williams

Wiljer

. Why are patient portals important in the age of COVID-19? Reflecting on patient and team experiences from a Toronto Hospital Network. J Patient Exp. 2022;9:23743735221112216.

Waseem

Kircher

Feliciano

. Information blocking and oncology: implications of the 21st Century Cures Act and open notes. JAMA Oncol. 2021;7(11):1609-1610.

10.

Arvisais-Anhalt

Lau

Lehmann

, et al. The 21st Century Cures Act and multiuser electronic health record access: potential pitfalls of information release. J Med Internet Res. 2022;24(2):e34085.

11.

Genes

Darrow

Vijayaraghavan

Rucker

. The 21st Century Cures Act and emergency medicine – part 1: digitally sharing notes and results. Ann Emerg Med. 2022;79(1):7-12.

12.

Magid

Cohen

Katzovitz

. 21(st) Century Cures Act, an information technology-led organizational initiative. HSS J. 2022;18(1):42-47.

13.

Fujioka

Bickford

Gritke

, et al. Implementation strategies to improve engagement with a multi-institutional patient portal: multimethod study. J Med Internet Res. 2021;23(10):e28924.

14.

Skrzypcak

McFadden

Bauer

. Using electronic health record modifications to increase use of adolescent sensitive notes. Hosp Pediatr. 2023;13(4):e81-ee7.

15.

Wang

Shen

Barbaro

, et al. A multi-level analysis of individual and neighborhood factors associated with patient portal use among adult emergency department patients with multimorbidity. Int J Environ Res Public Health. 2023;20(2):1231.

16.

Hoffman

. What the therapist thinks about you. New York, NY: New York Times; 2014. https://archive.nytimes.com/well.blogs.nytimes.com/2014/07/07/what-the-therapist-thinks-about-you/.

17.

Strudwick

Booth

McLean

, et al. Identifying indicators of meaningful patient portal use by psychiatric populations. Inform Health Soc Care. 2020;45(4):396-409.

18.

Leung

Clark

Sakal

Friesen

Strudwick

. Patient and family member readiness, needs, and perceptions of a mental health patient portal: a mixed methods study. Stud Health Technol Inform. 2019;257:266-270.

19.

Strudwick

Clark

Sanches

Strauss

. Predictors of mental health professionals’ perceptions of patient portals. AMIA Annu Symp Proc. 2018;2018:989-997.

20.

Strudwick

Yeung

Gratzer

. Easy access, difficult consequences? Providing psychiatric patients with access to their health records electronically. Front Psychiatry. 2019;10:917.

21.

Dobscha

Denneson

Jacobson

Williams

Cromer

Woods

. VA Mental health clinician experiences and attitudes toward OpenNotes. Gen Hosp Psychiatry. 2016;38:89-93.

22.

O’Neill

Blease

Delbanco

. Open notes become law: a challenge for mental health practice. Psychiatr Serv. 2021;72(7):750-751.

23.

Blease

Kharko

Hagglund

, et al. The benefits and harms of open notes in mental health: a Delphi survey of international experts. PLoS One. 2021;16(10):e0258056.

24.

Blease

Torous

Hagglund

. Does patient access to clinical notes change documentation? Front Public Health. 2020;8:577896.

25.

Blease

Torous

Kharko

, et al. Preparing patients and clinicians for open notes in mental health: qualitative inquiry of international experts. JMIR Ment Health. 2021;8(4):e27397.

26.

Walker

Leveille

Bell

, et al. Opennotes after 7 years: patient experiences with ongoing access to their clinicians’ outpatient visit notes. J Med Internet Res. 2019;21(5):e13876.

27.

Zhang

Shen

Booth

LaChance

Jackson

Strudwick

. Supporting the use of patient portals in mental health settings: a scoping review. Inform Health Soc Care. 2022;47(1):62-79.

28.

Creswell

Plano Clark

. Designing and Conducting Mixed Methods Research. 2nd ed. Los Angeles: Sage Publications; 2013.

29.

Strudwick

Kuziemsky

Booth

, et al. Engaging patients and family members in the evaluation of a mental health patient portal: protocol for a mixed-methods study. BMJ Open. 2018;8(8):e025508.

30.

Holmes

Bryan

McGavin

. Engaging patients as partners in health research: lessons from BC, Canada. Healthc Manage Forum. 2018;31(2):41-44.

31.

Healthcare Information and Management Systems Society. Electronic Medical Record Adoption Model (EMRAM) [Web]. 2022 [cited 2022 Aug 15]. https://www.himss.org/what-we-do-solutions/digital-health-transformation/maturity-models/electronic-medical-record-adoption-model-emram.

32.

Lawrence

Dunkel

McEver

, et al. A REDCap-based model for electronic consent (eConsent): moving toward a more personalized consent. J Clin Transl Sci. 2020;4(4):345-353.

33.

Chen

Turner

Sholle

, et al. Evaluation of a REDCap-based workflow for supporting federal guidance for electronic informed consent. AMIA Jt Summits Transl Sci Proc. 2019;2019:163-172.

34.

Üstün

Kostanjsek

Chatterji

Rehm

. Measuring health and disability: manual for WHO disability assessment schedule WHODAS 2.0. Geneva, Switzerland: World Health Organization; 2010.

35.

Young

Bullock

. The Mental Health Recovery Measure. 2003.

36.

Konecky

Meyer

Marx

Kimbrel

Morissette

. Using the WHODAS 2.0 to assess functional disability associated with DSM-5 mental disorders. Am J Psychiatry. 2014;171(8):818-820.

37.

Roosenschoon

B-J

Kamperman

Deen

Weeghel

Mulder

. Determinants of clinical, functional and personal recovery for people with schizophrenia and other severe mental illnesses: a cross-sectional analysis. PLoS One. 2019;14(9):e0222378.

38.

Bitter

Roeg

van Assen

van Nieuwenhuizen

van Weeghel

. How effective is the comprehensive approach to rehabilitation (CARe) methodology? A cluster randomized controlled trial. BMC Psychiatry. 2017;17(1):396.

39.

Baumel

Muench

Edan

Kane

. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9):e14567.

40.

Matthews

Topham

Caleb-Solly

. Interaction and engagement with an anxiety management app: analysis using large-scale behavioral data. JMIR Ment Health. 2018;5(4):e58.

41.

Miller

Ainsworth

Yardley

, et al. A framework for analyzing and measuring usage and engagement data (AMUsED) in digital interventions: viewpoint. J Med Internet Res. 2019;21(2):e10966.

42.

Pham

Graham

Carrion

, et al. A library of analytic indicators to evaluate effective engagement with consumer mHealth apps for chronic conditions: scoping review. JMIR Mhealth Uhealth. 2019;7(1):e11941.

43.

Pham

Graham

Lalloo

, et al. An analytics platform to evaluate effective engagement with pediatric mobile health apps: design, development, and formative evaluation. JMIR Mhealth Uhealth. 2018;6(12):e11447.

44.

Patton

. Qualitative research & evaluation methods: Integrating theory and practice. Saint Paul, MN: Sage Publications; 2002.

45.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334-340.

46.

Kemp

Zhang

Inglis

, et al. Delivery of compassionate mental health care in a digital technology-driven age: scoping review. J Med Internet Res. 2020;22(3):e16263.

47.

Shen

. The eHealth trust model: Understanding the patient privacy perspective in a digital health environment. Toronto, ON: University of Toronto; 2019.

48.

Saunders

Sim

Kingstone

, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893-1907.

49.

Cnaan

Laird

Slasor

. Using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data. Stat Med. 1997;16(20):2349-2380.

50.

Wolpert

Rutter

. Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health. BMC Med. 2018;16(1):82.

51.

Elo

Kyngas

. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107-115.

52.

Lincoln

Guba

. But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Dir Program Eval. 1986;1986(30):73-84.

53.

van Rijt

Hulter

Weggelaar-Jansen

Ahaus

Pluut

. Mental health care professionals’ appraisal of patients’ use of web-based access to their electronic health record: qualitative study. J Med Internet Res. 2021;23(8):e28045.

54.

Walker

Leveille

Bell

, et al. Opennotes after 7 years: patient experiences with ongoing access to their clinicians’ outpatient visit notes. J Med Internet Res. 2019;21(5):e13876.

55.

Dobscha

Denneson

Jacobson

Williams

Cromer

Woods

. VA Mental health clinician experiences and attitudes toward OpenNotes. Gen Hosp Psychiatry. 2016;38:89-93.

56.

Murugan

Gooding

Greenbaum

, et al. Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system. Appl Clin Inform. 2022;13(01):113-122.

57.

Kassam

Shin

Durocher

, et al. “I think it’s something that we should lean in to”: the use of OpenNotes in Canadian psychiatric care contexts by clinicians. Digit Health. 2022;8:20552076221144106.

58.

Kassam

Durocher

Shen

Strudwick

. Identifying implementation strategies to enhance the uptake of OpenNotes in mental health settings: a study protocol. Stud Health Technol Inform. 2022;290:1044-1045.

59.

Kipping

Stuckey

Hernandez

Nguyen

Riahi

. A web-based patient portal for mental health care: benefits evaluation. J Med Internet Res. 2016;18(11):e294.

60.

Kirk

Kelley

Yankey

Birken

Abadie

Damschroder

. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2015;11(1):1-13.

61.

O’Brien

Toms

. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol. 2008;59(6):938-955.

62.

Price

Lau

. The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health information systems. BMC Med Inform Decis Mak. 2014;14:43.

63.

Perski

Blandford

West

Michie

. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017;7(2):254-267.

64.

Crawford

Serhal

. Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health. J Med Internet Res. 2020;22(6):e19361.

65.

Ontario Health. Patient Portal (PP) Provincial Service Standards. 2021.

66.

Richardson

Lawrence

Schoenthaler

Mann

. A framework for digital health equity. NPJ Digit Med. 2022;5(1):119.

67.

Crawford

Serhal

. Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health. J Med Internet Res. 2020;22(6):e19361.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB