Abstract
Introduction
Patient experience, or the range of interactions patients have within a health care system, is a fundamental aspect of health care quality (1,2). Indeed, understanding and managing patient experience are critical in transforming organizational cultures into highly reliable and high-quality patient-centered care systems (3–5). Further, evidence of an association between positive patient experience and improved health care outcomes exists (4,6). Improving the quality of mental health care treatment, therefore, depends on robust measurement, evaluation, and systematic implementation of patient experience tools (7,8). Despite the widespread agreement of its importance, the structures and processes that guide and support patient experience are underdeveloped for inpatient psychiatry (9,10). This lack of development is likely due to the nature and context of inpatient psychiatric care, which has unique challenges in terms of how patient experience is defined, measured, and ultimately understood by patients. The resulting insufficiency has left psychiatric patients and families underserved within this critical quality domain.
Patient experience measurement should ideally entail a framework for understanding the cumulative sum of all a patient's interactions and perceptions across the continuum of care (11). It is the preferred approach for measuring the quality of patient care and services because it overcomes biases associated with satisfaction surveys by including objective components into its scope of evaluation (8). To obtain accurate measurement and actionable feedback, patient experience measurement must be centered on the elements of care that matter most to patients, while also corresponding to the care process relevant to the health service in question (7). Such measurements can be achieved by operationalizing concepts of dignity, recovery, and service-user experience. Measuring patient experience, therefore, involves both universal and context-specific interactions that are valued by patients and organizations in pursuit of continuous improvement.
Inpatient psychiatry involves the care of individuals who are experiencing acute and severe mental health problems and who cannot be safely treated in a less restrictive environment. The objectives of hospitalization are to provide a secure and safe physical environment, attend to basic personal needs, perform continuous monitoring and treatment by an interdisciplinary team, and resolve primary reasons for hospitalization (eg, stabilize acute symptoms, reduce risk of harm to self/others, and re-establish community supports) (12,13). In addition, a subset of the population may be involuntarily hospitalized in response to immediate safety concerns (14). Considering these conditions, defining and reliably measuring patient experience for this unique setting requires a specific approach.
Previous empirical work on psychiatric inpatient experience has characterized patient-centered aspects of care as being focused on collaborative treatment team relationships, planning effective treatments, and promoting a healing environment (5,15,16). Notably, these concepts are not mutually exclusive, rather they form an interdependent system aimed at social, psychological, spiritual, physical, and behavioral support (17). Indeed, the relational nature of care involves the entire care staff as well as fellow patients. Regardless of role or discipline, interactions involving respect, empathy, and a sense of trust are understood as foundational to a positive experience (18).
A significant subset of the extant literature has detailed the importance of the therapeutic relationships between patients and primary treatment providers (19,20). This collaborative relationship is predicated on mutual respect and trust, while jointly establishing treatment goals along as well as coordination with the family and community supports (16,18). Agreed upon treatments that patients find beneficial may include medications, group therapy, education, skill-based techniques, and other alternative therapies (21). The quality of the therapeutic relationships, in turn, is linked to improved perceptions of care and positive treatment outcomes (22,23).
While the quality of the treatment team relationship is closely connected to treatment outcomes, nurses and direct care staff form an essential nexus between relationships and the healing environment (24). Patients have reported that the physical and emotional presence of staff is beneficial due to their role as pragmatic caregivers and monitors for safety (25,26). Just as these relationships impact the social and psychological milieu of the unit, the physical space and unit organization similarly contribute to the overall sense of safety and healing (17). Whereas the locked physical environment and coercive interventions can result in feelings of powerlessness and humiliation, (27) these may be countervailed by an appropriate focus on a calm and relaxing physical space, organized leisure time, frequent access to outdoor spaces, and connection with external supports (17).
Patient-centered care is integral to modern psychiatric therapies, yet there are relatively few psychometrically validated inpatient psychiatric experience surveys that are widely used (9,28). Although a nationally standardized, publicly reported patient experience survey has been adopted across primary and tertiary medical settings, inpatient psychiatric facilities are not currently mandated or incentivized to measure patient experience (9,29). Even as the Centers for Medicare & Medicaid Services (CMS) has sought public comment on the prospect of measuring patient experience as part of its Inpatient Psychiatric Facility Quality Reporting (IPFQR) program, several concerns have been raised. Primarily, existing measurement tools exhibit varying degrees of psychometric validation and generalized utility (8,30). Additionally, there are unresolved programmatic issues related to mode, timing, setting, and burden of administration. Despite the significant work to understand the psychiatric inpatient experience, these aspects of care have not been reliably synthesized into a framework that supports meaningful, reliable, and actionable measurement for quality improvement. Importantly, such instruments should be grounded in existing theoretical models of the patient experience while also reflecting the unique interdisciplinary and operational nature of inpatient psychiatric care.
The purpose of the current study was to develop a psychometrically valid psychiatric inpatient patient experience measure via a rigorous development and validation process. Based on extant theories found in the literature, including multidimensional and conceptual frameworks of patient experience, we developed a novel measure with a hypothesized five-factor model, including treatment team relationships, effective treatment, healing environment, unit organization, and basic needs (31–33).
Methods
Participants
Participants were 4245 individuals receiving treatment at acute inpatient psychiatric hospitals in a single health system located in the northeastern United States between January 2020 and June 2021 were included in this study. Hospital units are organized by specialty and age, including adolescent, transitional age youth, adult mood disorders, general adult, and geriatric. Among the individuals discharged during this period, the primary diagnoses include depressive disorders (35%), schizophrenia and other psychotic disorders (26%), bipolar disorders (18%), anxiety disorders (5%), substance use disorders (5%), and other diagnoses (10.0%). Inclusion requirements were individuals preparing for hospital discharge with the ability to read and respond to questions in one of the multiple languages on a tablet-based computer. The intent was to capture all patients using convenience sampling procedures, but extenuating operational circumstances prevented some patients from receiving the survey; we estimate a 5% miss rate. Some participants declined to complete the survey, which left a total of 2486 individual responses. At the beginning of the survey, optional self-reported demographic information was collected including age, gender, and race/ethnicity given in Table 1.
Participant Demographic Information.
Procedure
Institutional review board approval was obtained from the university where the research team is based to analyze the data from the quality improvement project. Initial development and data collection efforts occurred at 2 primary psychiatric hospitals within a single urban city. The initiative eventually expanded to multiple hospitals across 3 cities within the same regional health system. In total, 8 inpatient units from 4 system-based hospitals participated in this quality improvement initiative.
Unit administrative staff distributed each survey after notification of a pending discharge (typically within 24 h of planned discharge date). These staff received brief training on survey administration and a script to minimize potential response bias. After consent was obtained, participants were requested to complete the survey using a tablet computer. An estimated 10% of respondents required assistance completing the survey. The survey was available in English and Spanish, and data were captured using Qualtrics XM software.
Survey Development
Survey items were formulated by an interdisciplinary team of researchers and clinicians from psychiatry, psychology, nursing, social work, and occupational therapy. A comprehensive review of the literature was first conducted to understand the theoretical underpinnings of a patient experience construct for inpatient psychiatry (31–33). A series of patient focus groups were then facilitated over 4 weeks to identify important aspects of being in the hospital within an inpatient psychiatric setting. Patients (n = 40) from each of the inpatient units were invited to participate in a focus group during regularly scheduled group times. Experienced clinicians guided focus group discussions with 7 probing questions that examined the patient's feelings on important aspects of being in the hospital (ie, important components of treatment, relationship with the treatment team and staff, unit safety and security, physical space, sense of community, staff responsiveness, and treatment effectiveness). Individual responses were transcribed on a flip chart for all participants to view during the focus group and subsequently integrated using thematic analysis. Further input was solicited from a patient and family advisory council using a similar methodology. Finally, content validity was established by both internal and external experts from various psychiatry and related disciplines and a measurement design team.
An initial 28-item, five-domain survey with 2 additional open-ended prompts for narrative feedback was developed and piloted using exploratory factor analysis (n = 462). A 5-point Likert agreement scale was used which included both face and text components. Each item was carefully worded to minimize subjectivity and ensure reading accessibility. Items were randomized to minimize item-order effects. A thematic analysis was subsequently performed based on the open-ended items to elucidate primary and secondary themes of patients’ narrative feedback. Results were used to further inform a conceptual framework, semantic taxonomy, and item design.
Results
Data Analyses
Data acquired from the Psychiatric Inpatient Experience (PIX) survey was submitted to R, an open-source statistical environment, for the purposes of descriptive and basic psychometric analysis. Exploratory and confirmatory factor analyses required statistical packages (ie, mice, psych, lavaan, and SemPlot) that could perform structural equation modeling. To address missing data, maximum likelihood estimation was employed for all confirmatory factor analyses.
Thematic Analysis
An inductive thematic analysis was performed and resulted in 4 superordinate themes that were important to patients, including relationships, presence, treatment, and healing environment (34). Each of the patterns consisted of multiple themes and subthemes. Relationships consisted of respect, character, and communication; Treatment entailed mental health therapies, recovery, and medical care; Presence included attentiveness, teamwork, and professionalism; and Healing Environment consisted of safety, physical space, wellness, and unit structure.
Exploratory Factor Analysis
Exploratory factor analysis (n = 462) was conducted to determine the factor structure of the PIX survey. Oblique rotation was used due to expected correlations among individual- and factor-level items. Due to the ordinal nature of the variables, weighted least squares mean- and variance-adjusted χ2 test of model fit estimator was used. The scree plot of eigenvalues was examined to determine the No. of factors to retain in the model with a point of inflection on the curve observed above the fourth factor. Individual items were removed if they had a factor loading of less than the absolute value of 0.40, except for 3 items that had sufficiently close factor loadings and were important questions of interest to the hospital system (35). Based on exploratory factor analysis, 5 of the original 28 items were removed. A total of 23 items were retained in the final four-factor solution, easily meeting the guidelines for the minimum ratio of participants to items (36).
The four-factor model resulted in a root mean square error of approximation (RMSEA) fit index of below 0.05, suggesting a good fit and explained 53% of the variance in the set of predictors (35). The Kaiser–Meyer–Olkin Measure of Sampling Adequacy suggested that the data sample was adequate and seemed appropriate for factor analysis (KMO = 0.91) (37). The final factor solution is presented in Table 2 with the item loadings for each of the PIX factors. The identified 4 factors were treatment team relationships, treatment effectiveness, nursing team presence, and healing environment. Descriptive statistics for each of the PIX factors and items are presented in Table 2.
Factor Item Composition and Loadings for the Four-Factor Model.
Abbreviations: HE, healing environment; NTP, nursing team presence; TTR, treatment team relationships; TE, treatment effectiveness.
*Items were retained due to clinical significance.
Confirmatory Factor Analysis
Confirmatory factor analysis (n = 2204) was completed using a separate sample. Our posited four-factor model was tested against a single factor model as well as theoretically plausible two and three-factor models. The single factor model, in which all item parcels load onto 1 patient experience factor, was first tested. As expected, the fit was poor as shown in Table 3. Two- and three-factor models in which all items loaded on to 2 factors and 3 factors also evinced poor fit as shown in Table 3. The posited four-factor model was tested. All indices demonstrated good or adequate fit properties. The results countenance a four-factor hierarchical patient experience model for a sample of inpatient psychiatric patients as shown in Table 3 and Figure 1.

Graphical illustration with the four-factor model.
Results of all Confirmatory Factor Analyses.
* = P < .001.
Abbreviations: RMSEA: root mean square error of approximation.
Discussion
The purpose of this research was threefold: (i) to determine the best factor structure for a novel psychiatric inpatient experience measure; (ii) to assess the reliability of this instrument in a clinical sample; and (iii) to elucidate the role of a psychometrically validated measure of patient experience in a large hospital system. Results indicated that patient experience, as measured by the PIX survey, is best represented as a statistically significant valid and reliable construct that subsumes a four factor model (ie, treatment team relationships, treatment effectiveness, nursing team presence, and the healing environment) in a large sample of inpatient psychiatric patients. These 4 factors are accompanied by a corresponding four-domain taxonomic framework for understanding narrative feedback.
Our novel PIX survey integrates established patient experience theory as well as the aspects of care that are important to psychiatric inpatients, while concurrently providing organization and discipline-specific feedback. Previous empirical work has characterized patient-centered aspects of care as collaborative, relationship-oriented, and promoting a healing care environment (24,38). Accordingly, our survey highlights the role of patient and provider relationships that entail mutual respect, careful listening, and shared responsibility for working toward treatment goals. It further establishes the importance of deliberate, meaningful interactions oriented around patient-centered recovery with attention to acute needs while prioritizing reliable teamwork. These concepts are enhanced by individually tailored treatment interventions in a physically clean and safe hospital environment that promotes patient healing. The PIX survey synthesizes these concepts across domains that align with the core functions of the interdisciplinary team and hospital programming. The resulting model provides an integrated framework for understanding psychiatric inpatient experience in a way that is both meaningful and actionable for continuous quality improvement.
Limitations
This study had a few limitations. First, opportunities to fully assess construct validity, including discriminant and convergent validity, were limited due to the burdensome nature of administering multiple surveys to an inpatient psychiatric population. Additional research may be useful to further establish internal and construct validity. Second, because data were collected for quality improvement purposes, a test–retest procedure was not performed, therefore limiting the conclusiveness of internal validity. Although participants reflected a demographically diverse population across multiple cities and hospitals, sampling procedures could feasibly have introduced concerns about generalizability (ie, the definition of patient experience may vary across regions or unrepresented subpopulations). However, in accordance with established theory, broad concerns about generalizability were likely mitigated (31–33). Finally, results relied on self-report, which is vulnerable to social response bias. Nonetheless, this was partially mitigated by administering the survey approximately 24 h prior to discharge.
Conclusion
The PIX survey is a valid and reliable tool for measuring the psychiatric inpatient experience. It was tested within a diverse, multisite hospital system. Although no standardized patient experience tool has been widely implemented, PIX importantly contributes to the patient experience measurement infrastructure. Future work is needed to further examine and refine the psychometric properties of this survey. Although this tool is focused on the inpatient setting, a further suite of related surveys may also contribute to improved measurement of psychiatric services across the continuum of care.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
