Abstract
Background
The objectives of this study were (1) to describe the symptoms noted by family physicians during help-seeking visits for early psychosis, relative to a validated screening tool for early psychosis in primary care, and (2) to examine the referral disposition of patients meeting the screening tool cut-off.
Methods
We constructed a retrospective cohort of Ontario residents aged 14–35 years with an incident diagnosis of non-affective psychotic disorder between 2005–2015 in health administrative data, and at least one visit in the Electronic Medical Record Primary Care database during the 6 months prior to the date of psychotic disorder diagnosis (n = 572). We abstracted symptoms of psychosis noted by the family physician in the electronic medical records and compared these to the Primary Care Checklist (PCCL) for early psychosis.
Results
The most frequent PCCL items noted were “tension or nervousness” (13.3%), “depressive mood” (12.5%), “increased stress or deterioration in functioning” (7.5%), and “sleep difficulties” (6.6%). The PCCL cut-off was met by 187 patients (33%) across 327 visits (8%). A greater proportion of visits meeting the PCCL cut-off had psychosis noted as the main presenting issue (55.4% vs. 6.8%) and resulted in referral to mental health services (33.3% vs. 6.0%) than those not meeting the cut-off. However, two in three visits where the screening cut-off for early psychosis was met did not result in a referral to mental health services.
Discussion
The findings of this study suggest that family physicians may benefit from a screening tool when early psychosis is suspected to improve identification and guide referral practices.
Introduction
Background
Early psychosis intervention services provide rapid, coordinated and comprehensive care to young people experiencing a first episode of psychosis. The evidence indicates that these services are effective in attenuating symptoms, preventing relapse and enhancing overall recovery.1,2 A core component of early psychosis intervention is reducing the duration of untreated psychosis via early detection and prompt treatment,3,4 which is associated with improved clinical and functional outcomes in both the short and long term.5,6 Family physicians have an opportunity to facilitate this process due to their important role on the pathway to care. 7
In Ontario, the majority of young people make help-seeking contacts in primary care in the months prior to a first diagnosis of psychotic disorder, 8 and primary care is often the first contact during help-seeking. 9 Family physician involvement during help-seeking is associated with a lower likelihood of aversive pathways to care10,11 and a greater likelihood of follow-up with psychiatric care. 12 Similarly, young people who initiate their own help-seeking are more likely to do so in primary care than in emergency or psychiatric services. 13 As such, family physicians can help improve timely access to care by reducing the referral delay – the period between initiating help-seeking and receiving appropriate care – thereby shortening the duration of untreated psychosis and enhancing the potential for positive outcomes. 14
Despite their importance on the pathway to care, family physicians often face challenges in recognizing and responding to early psychosis. The involvement of family physicians in help-seeking is associated with a greater number of help-seeking contacts and a longer period before psychiatric follow-up,9,10 and half of young people who make help-seeking contacts in primary care are ultimately diagnosed in secondary/tertiary care.12,15 The clinical presentation of early psychosis is complex, comprised of broad and non-specific symptoms that overlap with common mental health conditions.16,17 International surveys of family physicians reveal an overall lack of knowledge and confidence in diagnosing and treating psychotic disorders, which are generally outside their scope of training and practice.18–20 For instance, family physicians have noted that they may not be asking the correct questions to identify early psychosis, and that their patients experiencing early psychosis may not report their symptoms unless they are asked specific questions about them. 21
Structured diagnostic interviews are the gold standard for identifying psychotic disorder, but they are time and resource intensive and require extensive specialized training. Clinician-administered and self-reported screening tools can address these barriers, and there are currently several tools designed for psychotic disorders.22,23 The evidence suggests that screening tools are sufficiently sensitive and specific to identify people who warrant a more comprehensive evaluation for psychotic disorder, particularly among help-seeking populations.22–24 The Primary Care Checklist (PCCL) was developed and validated by French and colleagues as a selective screening tool for symptoms of early psychosis in a primary care setting. 25 The PCCL was designed to be a brief, accessible and practical clinical aid for family physicians to guide them in identifying patients who warrant further assessment or referral. Compared to other screening tools, the PCCL is focused on observable changes in affect, behaviour and functioning rather than specific diagnostic criteria. Although the PCCL has been used in some international settings,26–28 it has not been widely adopted by family physicians.
Rationale and Objectives
In the absence of a commonly used screening tool for early psychosis in primary care, it is not known what symptoms are identified by family physicians during help-seeking visits as part of usual care, or how these physicians manage patients when these symptoms are identified. This information can help us understand whether family physicians would benefit from a screening tool in practice, which could increase detection and appropriate referrals to specialist care, such as early psychosis intervention services. 29 Prior studies of screening tools for early psychosis in primary care have largely focused on development, validation and implementation,22–24 but their application to the real-world clinical documentation and referral practices of family physicians has yet to be examined. Therefore, the objectives of this study were: (1) to describe the early psychosis symptoms noted organically by family physicians in an electronic medical record (EMR) during help-seeking visits relative to the PCCL items; and (2) to examine the referral disposition of patients meeting the PCCL cut-off.
Methods
This study follows the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) guidelines for observational studies. 30
Study Design and Data Sources
We obtained data from ICES (formerly the Institute for Clinical Evaluative Sciences) in Ontario, Canada. ICES is an independent, non-profit research institute whose legal status under the province's health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement.
We used linked health administrative data to construct a retrospective cohort of incident cases of non-affective psychotic disorder aged 14 to 35 years from 2005 to 2015 within Ontario. 7 This time frame corresponds to the rollout of early psychosis intervention programs across the province, and this age range aligns with the inclusion criteria of most programs.31,32 Cases were identified using a modified version of a validated algorithm, defined based on either one hospitalization or at least two emergency department or outpatient visits within a 12-month period with a diagnostic code for a non-affective psychotic disorder. 33 We excluded prevalent cases with a diagnosis of non-affective psychosis in the data holdings prior to 2005. The date of first diagnosis was considered the index date.
We linked this cohort with the Electronic Medical Records Primary Care (EMRPC) database using unique encoded identifiers. The EMRPC database is comprised of clinical data from approximately 400,000 patients in Ontario, contributed by over 450 family physicians using PS Suite EMR software for a minimum duration of 2 years. The data in EMRPC include the cumulative patient profile, complete progress notes by the family physicians, as well as summaries from consultant physicians, emergency department visits and inpatient hospitalizations. The quality and completeness of EMRPC database have been formally evaluated, which found reasonable capture of information within the EMR data when compared with administrative data.34,35 The generalizability of the EMRPC database to the entire province has also been formally evaluated, which identified some notable differences in the characteristics of patients (e.g., rurality, income, comorbidity) and physicians (e.g., age, sex, enrolment model). 36
Data Abstraction
We identified all visits to family physicians in EMRPC in the 6 months prior to and including the index date, which was the first diagnosis of psychotic disorder; this period was selected to reflect the early phases of psychotic illness. 37 We abstracted information based on key words and phrases from all visits recorded in the EMR during the 6-month period. The data abstraction was performed manually by a family physician using a standardized abstraction platform, which was developed with input from clinicians and based on prior research. 17 Intra-rater and inter-rater reliability of the EMR abstraction were assessed through re-abstraction of 5% and 15% of charts, respectively. Intra-rater agreement ranged from 75% to 100%, while inter-rater agreement ranged from 92% to 100%, indicating good consistency within and across abstractors. 38
Abstraction was guided by the PCCL, which contains 20 items relating to general, psychological and social functioning, in addition to psychotic symptoms (e.g., hallucinations, delusions, disorganized speech). Each item has a numerical value depending on its relevance to the overall risk of psychosis (ranging from 1 to 5), and a total score is calculated by summing the endorsed items (ranging from 0 to 55). The PCCL cut-off is a total score of ≥20 or endorsement of ≥1 key items (i.e., attenuated positive symptoms, state/trait vulnerability features), which indicates the need for further specialist assessment. Among a sample of young people referred to early psychosis intervention services, the PCCL showed 89% sensitivity and 60% specificity for identifying the early stages of psychosis, as validated against the Comprehensive Assessment of At-Risk Mental States (CAARMS). 25 These psychometric properties are comparable to those of other screening tools for psychotic disorders, such as the Prodromal Questionnaire and PRIME Screen.22,23
Study Variables
For each visit, we recorded the PCCL items noted organically by the family physician in the EMR during routine clinical practice. We used this information to calculate a PCCL score and determine whether the PCCL cut-off was met (i.e., total score of ≥20 or endorsement of ≥1 key items). We also recorded whether the patient was referred to a mental health service (i.e., early psychosis intervention, psychiatry, psychology, psychotherapy, or counselling) or an acute mental health service (i.e., emergency department or crisis centre) at each visit. If available, we recorded the main presenting issue noted by the family physician at the visit.
We collected information on patient characteristics that may be associated with the clinical presentation of early psychosis. Sociodemographic characteristics included age, sex, rural residence, migrant status, and neighbourhood-level income quintile. Clinical characteristics included the index diagnosis and the number of comorbid chronic conditions derived from the Aggregated Diagnosis Groups (ADGs) of the Johns Hopkins Adjusted Clinical Group® System Version 10. 39 Service-related characteristics included the number of visits with the family physician over the 6 months prior to the index date, the amount of time on the family physician's EMR, and whether they were rostered to the family physician.
Statistical Analysis
We described the cohort using frequencies and proportions, as well as medians and interquartile ranges, at the visit level. Standardized differences (std. diff.) were calculated to compare those who did and did not meet the PCCL cut-off, with differences greater than 0.10 considered meaningful between-group differences. 40 All analyses were conducted using SAS (Version 9.4).
Results
We identified 719 patients with a first diagnosis of non-affective psychotic disorder with a record in the EMRPC database. Among these patients, 572 patients had at least one visit with a family physician in the 6-month period prior to and including the index date, for a total of 4,023 visits. The characteristics of the sample are summarized at the visit level in Table 1, the PCCL items noted at each visit are summarized in Table 2, and the distribution of visits by PCCL cut-off and mental health referrals is illustrated in Figure 1.
Distribution of visits by Primary Care Checklist (PCCL) cut-off and mental health referrals.
Characteristics of Sample Overall and by Primary Care Checklist (PCCL) Cut-Off, at the Visit Level.
EMR = electronic medical records; IQR = interquartile range; PCCL = Primary Care Checklist; PNOS = psychosis not otherwise specified; SD = standard deviation; SSD = schizophrenia spectrum disorders; Std. Diff. = standardized difference.
Primary Care Checklist (PCCL) Items and Mental Health Referrals Overall and by PCCL Cut-Off, at the Visit Level.
PCCL = Primary Care Checklist; NR = not reported; Std. Diff. = standardized difference.
Across the entire sample, the most frequent PCCL items noted organically by family physicians during visits were “tension or nervousness” (13.3%), “depressive mood” (12.5%), “increased stress or deterioration in functioning” (7.5%) and “sleep difficulties” (6.6%). Overall, 7.9% and 0.4% of visits resulted in a referral to a mental health service and acute mental health services, respectively.
The PCCL cut-off was met by 187 patients (33%) in a total of 327 visits (8%). The items most frequently noted across visits meeting the PCCL cut-off were “feeling people are watching you or giving you a hard time for no reason” (55.4%), “feeling, hearing, or seeing things that others cannot” (50.5%), “tension or nervousness” (47.7%) and “increased stress or deterioration in functioning” (45.9%). Referral to mental health services or acute mental health services were made at 102 (31.2%) and 7 (2.1%) of these visits, respectively, whereas no referral was made at 218 visits (66.7%) where the PCCL cut-off was met.
Patients who met the PCCL cut-off across visits were predominantly male (54.4% vs. 45.3%, std. diff. = 0.18), had fewer ADGs (6.02 ± 3.44 vs. 7.49 ± 4.07, std. diff. = 0.42) and had fewer visits in primary care prior to the index date (5.82 ± 4.73 vs. 7.01 ± 5.69, std. diff. = 0.23) compared to those who did not meet the cut-off. In addition, a greater proportion of visits meeting the PCCL cut-off had psychosis symptoms or psychotic disorder noted as the main presenting issue (55.4% vs. 6.8%, std. diff. = 1.23) and resulted in referral to any mental health service (33.3% vs. 6.0%, std. diff. = 0.73) than visits where the cut-off was not met. In general, all PCCL items were more frequently noted at visits where the PCCL cut-off was met.
Discussion
Interpretation
We found that signs and symptoms of early psychosis, as outlined by the PCCL, were infrequently noted by family physicians during patient visits over the 6 months prior to a first diagnosis of psychotic disorder. Family physicians see only a few incident cases of psychosis each year,18,19 which may limit their experience with and confidence in detecting and treating psychotic disorders. 20 Limited time during patient visits, competing clinical demands and lack of specialized mental health training have also been cited by family physicians as major barriers to detection and treatment of psychosis in their practices.41,42 As well, patients may not disclose issues related to psychosis during their primary care visit, or they may be experiencing other issues that are more salient and less stigmatized.20,43
Young people making help-seeking contacts in primary care for psychosis are likely at an earlier stage of illness with a less acute presentation, 44 although prodromal symptoms are most often present during this period.16,37 Prior research suggests that family physicians are more adept at identifying overt symptoms of psychosis, such as delusions, hallucinations and bizarre behaviour, but may have difficulties identifying more insidious psychotic symptoms, such as low mood, social withdrawal and functional decline.18,19,44 Interestingly, we found that family physicians most often noted less overt symptoms during visits. These symptoms may have been more apparent to physicians, as they are more common in primary care practice, 45 and often occur during help-seeking for psychosis.16,17 However, the noted symptoms were not necessarily considered within the context of early psychosis, which was not often recorded as the main presenting issue by the family physician.
Psychosis symptoms or psychotic disorder were noted by family physicians as the main presenting issue in over half of visits where the PCCL cut-off was met, which was considerably more than when the cut-off was not met. The items most often noted during these visits were positive symptoms (i.e., delusions and hallucinations), which family physicians more strongly associate with psychotic disorders.18,19 A much greater proportion of visits meeting the PCCL cut-off resulted in a mental health referral than when the cut-off was not met. Most family physicians prefer referral to specialized mental health services for making an initial diagnosis and establishing a treatment plan when psychosis is suspected. 46 Yet only a third of visits meeting the PCCL cut-off resulted in a referral to a mental health service. Our findings indicate that family physicians can indeed identify psychosis and make necessary referrals, but also suggest that they may benefit from a screening tool to assist them with these processes.
Family physicians have shown interest in receiving further training and support in identifying early psychosis, including resources for screening.20,21,46 Screening tools can address barriers such as providers’ lack of time and experience, and they are generally well received by providers, patients and family members. 47 There are various strategies to improve the acceptability and feasibility of screening tools, which can facilitate their implementation in primary care and encourage uptake among family physicians. These strategies include integrating the tool into the EMR system with automatic score calculation, having allied health staff administer the tool during intake or waiting periods and providing physicians with practical training for using the tool. 47 An important future direction for research would be exploring the acceptability and feasibility of using the PCCL in a primary care setting for patients presenting with mental health symptoms. Understanding how family physicians perceive the relevance, usability and practicality of the PCCL is critical to informing implementation strategies.
While screening tools may assist with detecting early psychosis in primary care, family physicians often report difficulties with the mental health care system following detection, including long waitlists, rejection of referrals and overall poor communication from specialists. 41 These difficulties are relevant to Ontario, especially with respect to extensive wait times.46,48,49 Family physicians may therefore decide to delay or avoid referring patients to mental health services and instead choose to oversee the care of these patients themselves. This situation may partially explain the low proportion of patients referred to mental health services in our study, including those where psychosis was recognized and/or the PCCL cut-off was met. In a future study, it would be worthwhile to explore how family physicians manage the care of patients who are not referred to mental health services. Moreover, family physicians have noted that specialized consulting and referral services with low thresholds and short waitlists would be the most helpful resource in managing psychotic disorders.19,46 Shared care models may offer a promising approach to addressing these challenges by fostering collaboration between family physicians and mental health specialists, which can facilitate timely and coordinated care for patients with early psychosis.50–52
Limitations
This study has some methodological limitations. First, the PCCL was not prospectively administered during patient visits, and thus what family physicians discussed and recorded for each visit was not standardized. We were unable to determine whether symptoms of psychosis were not present or if they were simply not recorded in the EMR, which could underestimate the prevalence of symptoms and the proportion of patients meeting the PCCL cut-off. There may also be differences across physicians in terms of the level of detail recorded in the EMR. Second, screening tool cut-offs assume that all items have been considered and evaluated, which may explain the low frequencies of PCCL items noted and the small proportion of visits meeting the PCCL cut-off. We urge caution in generalizing the symptom frequencies reported here to purposive screening, which may result in differences in the frequency of noted symptoms among those referred for further evaluation. Third, we did not limit the visits to mental health-related help-seeking, which may also explain the low frequencies of PCCL items noted. Fourth, the PCCL does not capture all symptoms that may warrant or prompt a referral such as self-harm and suicidality, which could underestimate the proportion of patients who warrant a referral. Fifth, the algorithm used to identify cases within the health administrative data was validated for chronic, non-affective psychotic disorders, which may impact the reliability and generalizability of our findings. Finally, the EMRPC database is limited to a small subset of family physicians, and our findings may not be generalizable to all family physicians and patients across the province. We may have also missed relevant information on symptoms and referrals from other primary care providers not directly captured in the database, such as paediatricians and nurse practitioners.
Conclusions
In this study using EMR data, we found that family physicians infrequently recorded items from a validated screening tool for psychosis during visits occurring 6 months prior to a first diagnosis of psychotic disorder. In visits where family physicians recorded sufficient information such that the screening tool cut-off was met, only a third of patients were referred to mental health services. These findings suggest that family physicians may benefit from a screening tool for early psychosis to improve identification and guide referral practices.
Footnotes
Acknowledgments
This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). The study was completed at ICES Western, where core funding is provided by the Academic Medical Organization of Southwestern Ontario (AMOSO), Lawson Health Research Institute, Schulich School of Medicine & Dentistry, and Western University. This document used data adapted from the Statistics Canada Postal Code OM Conversion File, which is based on data licensed from Canada Post Corporation and/or data adapted from the Ontario MOH Postal Code Conversion File, which contains data copied under license from Canada Post Corporation and Statistics Canada. Parts or whole of this material are based on data and/or information compiled and provided by: Ontario MOH; Ontario Health (OH); Immigration, Refugees and Citizenship Canada (IRCC), current to 2015; and Canadian Institute for Health Information (CIHI). The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. The project co-investigators include: Kelly K. Anderson (Western University); Suzanne Archie (McMaster University); Richard G. Booth (Western University); Elisa Candido (ICES); Chiachen Chang (Northern Ontario School of Medicine); Saadia Hameed Jan (Western University); Liisa Jaakkimainen (ICES); Paul Kurdyak (Centre for Addiction and Mental Health); Daniel Lizotte (Western University); Arlene G. MacDougall (Western University); Ross M.G. Norman (Western University); Lena Palaniyappan (McGill University); Rebecca Rodrigues (Western University); Bridget L. Ryan (Western University); Amanda L. Terry (Western University); Aristotle Voineskos (Centre for Addiction and Mental Health).
Author Contributions
Joshua C. Wiener was involved in the composition of the manuscript, which entailed review of the literature, interpretation of the data, contextualization of the findings and coordination of revisions. Rebecca Rodrigues was involved in the conception and design of the study, in validation of abstraction, and in critical revision of the manuscript. Jennifer NS Reid was involved in management/analysis of the data and in critical revision of the manuscript. Dr. Suzanne Archie, Dr. Lena Palaniyappan and Dr. Liisa Jaakimainen were involved in the abstraction process and in critical revision of the manuscript. Dr. Saadia Hameed Jan, Dr. Arlene G MacDougall, Dr. Neo Sawh and Branson Chen were involved in the abstraction process. Dr. Kelly K. Anderson was involved in the conception and design of the study and in critical revision of the manuscript.
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: LP reports personal fees from Janssen Canada, Otsuka Canada, SPMM Course Limited UK and Canadian Psychiatric Association; investigator-initiated educational grants from Janssen Canada, Otsuka Canada and Sunovion; and book royalties from Oxford University Press outside the submitted work. All other authors report no conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by an Early Researcher Award from the Ontario Ministry of Research, Innovation, and Science, and by a Project Grant from the Canadian Institutes of Health Research (PJT 153022). Dr. Kelly K. Anderson is supported by a Canada Research Chair in Public Mental Health Research.
Ethics
ICES is an independent, non-profit research institute that is a prescribed entity under Section 45 of Ontario's Personal Health Information Protection Act, which enables compilation and analysis of personal health information related to the management, evaluation and monitoring of the health system without consent for such purposes. The portions of this study that involve health administrative data do not require review by a Research Ethics Board.
Data availability
The dataset from this study will be held securely in coded form at ICES. While data sharing agreements between ICES and data providers prohibit ICES from making the dataset publicly available, access can be granted to those who meet pre-specified criteria for confidential access. The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES, and thus may require modification or be inaccessible.