Sage Journals: Discover world-class research

Abstract

The importance of effective doctor-patient communication is well established; however, rigorous evaluations of its association with patient-reported outcomes among seriously ill patients are needed. We analyzed 2 waves of survey data from adults with serious illnesses in primary care clinics. We administered the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) communication scale, Patient-Reported Outcomes Measurement and Information System (PROMIS^®) global health items, and the PROMIS-29 depression and anxiety scales. We evaluated a structural equation model to assess relationships between patient experience and self-reported global physical and mental health in 779 patients at baseline and 12 months later. Average age was 69, with 52% male, 18% Hispanic, 9% Asian, and 7% Black; 24% had a high school education or less. Better global health (PROMIS) at baseline was associated with better doctor-patient communication (CAHPS) at 12 months (β = 0.09, P = .005), and better doctor-patient communication at baseline was related to better mental health at follow-up (β = 0.07, P = .0105). The results suggest that patients’ overall health may influence doctor-patient communication, and this communication may impact patients’ mental health over time.

Keywords

clinician-patient relationship communication patient-reported experience measures patient-reported outcome measures quality of life patient/relationship-centered

Introduction

Effective communication with patients is considered a cornerstone of high-quality healthcare.¹ Patient-reported experience measures (PREMs) are essential for evaluating the effectiveness of provider-patient communication.² They also identify opportunities for improving the physician-patient relationship.³ Patient reports of provider communication are strongly associated with their overall perceptions of physicians^4,5 and are a key target for quality improvement efforts.^6,7

Robust improvements in patient-provider communication and a trend toward a positive association between provider use of patient-reported outcome measures (PROMs) and subsequent health outcomes were found in a systematic review of the use of PROMs in oncology clinical trials.⁸ Positive associations between PREMs and patient adherence to medical recommendations, health-promoting behaviors, and PROMs have been found.^9-13 Patient experience with communication is especially critical for sicker patients due to the increased treatment demands. Effective communication between doctors and patients with serious illnesses is crucial to ensure goal-concordant care, particularly at the end of life.¹⁴

In this longitudinal study, we examined doctor-patient communication and self-reported physical and mental health over time among patients with serious illnesses. Given the positive associations reported in the literature between doctor-patient communication and patient adherence to treatment, we hypothesized that patients who reported more positive communication with their doctor at baseline would report better health 12 months later.

Methods

The UCLA Institutional Review Board approved this study (18-001612).

Sample

The UC Health Care Planning Study is a multisite cluster randomized trial in 3 academic primary care clinics (41 Los Angeles, 6 Irvine, and 3 San Francisco clinics) that tested advance care planning (ACP) interventions with seriously ill patients 18 years or older who had a primary care clinician and had attended at least 2 primary care office visits in the prior 12 months. ACP is a process of understanding and sharing personal values, life goals, and preferences regarding future medical care.¹⁵ It includes clarifying and discussing values and goals and, if appropriate, completing written documents and medical orders.

An advance directive is a legal document specifying medical care preferences if one cannot make decisions due to illness or injury. It may also designate someone to make medical decisions for the person. It may also designate someone to make medical decisions on their behalf. Of the 6154 patients with serious illness who lacked an advance directive in the prior 3 years and were eligible to receive an ACP intervention as part of the UC Health Care Planning Study, 5382 were sent the baseline survey via mail. Serious illness was assessed using a validated automatic electronic health record phenotype algorithm using administrative billing codes, encounter data, and clinical information. Serious illness was defined by having an at-risk medical diagnosis (cancer, heart failure, chronic obstructive pulmonary disease, end-stage liver disease, end-stage renal disease, or amyotrophic lateral sclerosis) linked with advanced age or a level of severity such that ACP would be a priority: (1) poor short-term survival prognosis or (2) developing incapacity or (3) worsening functional status or (4) high burden of disease, causing significant suffering potentially leading to increased care utilization.¹⁶ This study's 24-month mortality rate for seriously ill patients was 15%.

Measures

Before the ACP interventions, patients were mailed study information and a survey in English or Spanish. Twelve months after baseline, the follow-up survey was administered by mail or phone.¹⁷

Patient experience. We included Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) items with a recall period of the last 12 months. Instructions preceding the items were as follows: “Your personal doctor could be a doctor, a nurse practitioner, or anyone else you see for regular medical care. Your medical provider will not know any of your answers, or even that you took this survey.” The 5 items were (1) How often did this doctor explain things in a way that was easy to understand? (2) How often did this doctor listen carefully to you? (3) How often did this doctor show respect for what you had to say? (4) How often did this doctor spend enough time with you?) and (5) How often did this doctor seem to know the important information about your medical history? All items were administered using Never, Sometimes, Usually, or Always response choices. The communication scale was scored to have a possible range of 1 to 4, with a higher score representing better communication. The reliability and validity of the communication scale are reported elsewhere.¹⁸

Self-reported health. The survey also included a Patient-Reported Outcomes Measurement and Information System (PROMIS^®) global physical health item (“In general, how would you rate your physical health?”) and a global mental health item (“In general, would you say your quality of life is”) both asked with Poor, Fair, Good, Very Good, or Excellent response options.¹⁹ Also included were the PROMIS-29 v2.1 4-item depression and 4-item anxiety scales.²⁰ The PROMIS measures are scored using a T-score metric (the mean in the US general population is 50, and the standard deviation is 10). A higher score represents better communication, global physical health, and global mental health, but more depression and anxiety.

Analysis plan. First, we estimated internal consistency reliability for the CAHPS communication scale and the PROMIS anxiety and depression scales.²¹ Then, we computed the means and standard deviation for the CAHPS communication scale and the PROMIS depression, anxiety, global physical health, and global mental health measures. Next, we estimated Pearson's product-moment correlations among the 5 measures at baseline and 12 months later. We hypothesized that more positive assessments of the doctor's communication would be associated with better self-reported health. Finally, we estimated a cross-lagged structural equation model to assess longitudinal relationships between patient experience and self-reported health.²² We began by specifying a model with latent variables (ie, not directly observed but inferred from observed variables) of global health (global physical health and global mental health) and mental health (anxiety and depression), as well as the communication scale, which is an observed variable. The initial model included correlations among the latent and observed variables at baseline, direct effects from baseline to 12 months later to represent the stability of the latent variables and the communication scale, and residual correlations among the latent variables and communication scale at 12 months. Then, we added cross-lagged paths between baseline and 12-month variables based on Lagrange multiplier tests.²³ Next, we estimated covariances between residuals of the same measured variables at different time points. These correlated uniqueness estimates help account for associations caused by unmeasured variables. We report on results for a model that includes only significant parameter estimates. Excluding nonsignificant parameter estimates (fixing them to zero) is a standard procedure for obtaining a parsimonious model.

Because the likelihood of rejecting a model based on the chi-square test increases with sample size, the practical measures provide essential information about model fit to the data. We evaluated model fit using the comparative fit index (CFI) and the root mean square error of approximation (RMSEA). A good model fit is indicated by a CFI of 0.95 or higher and an RMSEA of 0.06 or lower.²⁴

The conventional P < .05 level of significance was used for statistical tests. Analyses were conducted using SAS 9.4 (TS1M7) software (SAS Institute Inc.).

Results

Sample

Of the 5382 seriously ill patients invited, 1100 (20%) completed the baseline survey, which was administered via mail (n = 1000), phone (n = 70), and email (n = 30). The enrolled participants were similar to the population of seriously ill patients in gender, age, race/ethnicity, and primary language.¹⁷ Seventy-one percent of the baseline sample (n = 779) also completed the 12-month survey and were included in the analyses. The average age of the analytic sample was 69 years (median, 72; range, 22-102 years). The majority were male (52%); 16% were Hispanic, 11% Asian, and 6% Black; 7% had less than a high school education, 13% were high school graduates, 30% had some college or a 2-year degree, 21% a 4-year college degree, and 29% more than a 4-year degree. In addition, 60% were in a committed relationship. The primary language spoken at home by 15% of the analytic sample was a language other than English.

Patient Experience and Self-Reported Health

Internal consistency reliability at baseline was 0.88 for the CAHPS communication scale, 0.89 for the PROMIS depression scale, and 0.87 for the PROMIS anxiety scale. Descriptive statistics for the CAHPS and PROMIS measures are provided in Table 1. Mean scores were similar for each measure at baseline and the 12-month survey. The CAHPS communication scale was skewed, with mean scores near the positive end of the scale (3.60 and 3.56 at baseline and 12 months, respectively). The sample had slightly higher (worse) PROMIS depression and anxiety scores and lower (worse) global physical and mental health scores than the general population mean of 50 (ie, 1-2 points at baseline and 2-3 points at the 12-month follow-up).

Table 1.

Mean (Standard Deviation) for CAHPS and PROMIS Measures (n = 779).

Measure	Baseline survey	12-month survey
CAHPS communication	3.60 (0.52)	3.56 (0.58)
PROMIS depression	51 (8)	52 (8)
PROMIS anxiety	52 (9)	53 (8)
PROMIS global physical	46 (7)	46 (7)
PROMIS global mental	49 (8)	48 (8)

Note: The communication scale has a possible range of 1 to 4. PROMIS measures are scored using a T-score metric (the mean in the US general population is 50, and the standard deviation is 10). A higher score represents better communication, PROMIS global physical health, and PROMIS global mental health but worse PROMIS depression and anxiety. CAHPS, consumer assessment of healthcare providers and systems; PROMIS = patient-reported outcomes measurement and information system.

Pearson's product-moment correlations among the CAHPS and PROMIS measures are shown in Table 2. Correlations indicating the stability from baseline to 12 months later ranged from 0.51 (doctor communication) to 0.67 (global physical health). Correlations of doctor-patient communication at baseline with health 12 months later ranged from 0.12 (global physical health) to −0.22 (anxiety). Correlations of the health measures at baseline with doctor communication 12 months later ranged from 0.15 (global physical health) to 0.19 (global mental health) and −0.19 (anxiety).

Table 2.

Pearson Product-Moment Correlations of CAHPS^® and PROMIS^® Measures (n = 779).

	Com_b	Dep_b	Anx_b	Phy_b	Men_b	Com_12	Dep_12	Anx_12	Phy_12	Ment_12
Dep_b	−.19	1.00
Anx_b	−.22	0.71	1.00
Phy_b	0.17	−.38	−.28	1.00
Men_b	0.21	−.43	−.29	0.73	1.00
Com_12	0.51	−.17	−.19	0.15	0.19	1.00
Dep_12	−.20	0.63	0.52	−.31	−.32	−.21	1.00
Anx_12	−.22	0.52	0.62	−.24	−.23	−.24	0.70	1.00
Phy_12	0.12	−.32	−.23	0.67	0.58	0.18	−.36	−.30	1.00
Men_12	0.17	−.37	−.27	0.58	0.66	0.25	−.43	−.33	0.74	1.00

Note: Com, communication; Dep, depression; Anx, anxiety; Phy, physical health; Men, mental health; _b, baseline; _12, 12-months postbaseline; underlined correlations indicate stability estimates; CAHPS, consumer assessment of healthcare providers and systems; PROMIS = patient-reported outcomes measurement and information system.

All P's < .001.

Structural Equation Model

The model was not statistically rejectable (χ² = 32.24, df = 22, P = .0734) and provided a good fit to the data: RMSEA = 0.0243 and CFI = 0.9975. All estimates were statistically significant (P < .05). Standardized loadings on the mental health latent variable at baseline (12 months later) were 0.76 (0.76) for anxiety and 0.94 (0.91) for depression. Standardized loadings on the global health latent variable at baseline (12 months later) were 0.80 (0.80) for physical health and 0.90 (0.92) for mental health. The correlated uniqueness estimates for the anxiety and global physical health observed variables over time were 0.19 and 0.16, respectively. The correlated errors for the depression and the global mental health observed variables were not significant and fixed to zero.

Figure 1 presents the structural equation model estimates for the latent and observed doctor-patient communication variables. The global health latent variable correlated 0.50 (P < .0001) with the mental health factor at baseline. Doctor communication at baseline correlated 0.23 with the global health latent variable and 0.22 with the mental health latent variable.

Figure 1.

Standardized estimates for structural equation model of patient experience and health.

The global health latent variable had a stability coefficient of 0.79, and the mental health latent variable had a standardized direct effect (stability) of 0.72 between baseline and follow-up. Significant residual correlations, indicating unique associations beyond those estimated by the latent variables, were found between anxiety at baseline and follow-up (r = .19, P < .0001) and global physical health at baseline and follow-up (r = .16, P < .0001). Better global health at baseline was significantly associated with doctor-patient communication at follow-up (standardized β = 0.09, P = .005). Additionally, doctor-patient communication at baseline was significantly associated with better mental health at follow-up (standardized β = 0.07, P = .0105).

Discussion

This longitudinal study provides evidence of the potential effects of patient experience on patient-reported outcomes in a relatively large sample of seriously ill adults. Direct effects estimated between the same concepts measured at baseline and 12 months later represent stability over time. The stability coefficient for doctor communication (0.49) indicates substantial individual change, consistent with data from the Medical Expenditure Panel Survey.²⁵ There was more stability (less change) in mental health (0.77 stability coefficient) and global health (0.79 stability coefficient) over time.

Better provider-patient communication at baseline had a positive direct effect on the 12-month mental health. This finding is consistent with a study of veterans receiving care in general mental health clinics, which found that better patient-reported communication with providers was associated with higher mental health summary scores on the short-form health survey (SF-12).¹³ Street²⁶ provided a conceptual model about how doctor-patient communication can affect outcomes. Proximally, successful provider-patient communication may improve patients’ understanding of their health and the recommended treatment plan, engender trust in their physician, increase their involvement in shared decision-making, and provide validation of their concerns. Proximal outcomes may improve intermediate outcomes, such as patients’ adherence to treatment. Intermediate outcomes may, in turn, impact longer-term health outcomes.

The positive association between doctor-patient communication and subsequent health outcomes was limited to mental health in our study. However, one study found that better technical quality of care was associated with significantly more favorable changes in the SF-12 physical component summary score.²⁷ Moreover, a systematic review suggested that more positive CAHPS communication scale reports were associated with better self-reported physical and mental health.²⁸

We found that global health at baseline had a small, significant association with reports of doctor communication 12 months later. In another study, patients whose orthopedic surgeons used PROMIS survey information about them during presurgery visits were likelier to report that the provider spent enough time with them, rate the provider positively, and recommend the provider's office to other patients on the CAHPS Clinician and Group Survey.²⁹

This research suggests an interplay between patients’ care experiences and their perceptions of health. Specifically, patients’ perceptions of their provider's communication may influence their subsequent mental health, and perceptions of global health may impact future communication. Thus, the study supports improving patient-centered care and communication between providers and patients.

Limitations

There are study limitations. The data are self-reported and do not include “objective” communication or clinical health status measures. While the response rate for this study was low (20%), the sample size was large and representative of the target population's demographic characteristics.¹⁷ Because the study participants were seriously ill patients of physicians at 3 academic medical centers in the same state, it is unknown whether the findings apply to other settings and patients with other serious illnesses. Further, this study does not address which aspects of doctor-patient communication have the most significant impact on patients' overall ratings of their experiences with their physicians.³⁰ Moreover, unmeasured variables such as health literacy and prior healthcare experiences may impact perceptions of communication and health outcomes. Finally, while the results may be suggestive, inferring causality from this observational study is problematic.

Conclusion

In this study of seriously ill older adults receiving primary care and ACP interventions at large tertiary health systems, we found associations between doctor-patient communication and patient physical and mental health. Understanding the mechanisms between communication and health could inform interventions to improve patient outcomes. Further research is needed to investigate the longitudinal associations between patient-reported experiences and patient-reported outcomes.

Footnotes

Acknowledgments

The authors appreciate Victor Gonzalez's and Katherine Santos’ administrative assistance. Dr Hays was supported by a cooperative agreement from the Agency for Healthcare Research and Quality (5U18HS029321). Dr Walling is funded by the Cambia Health Foundation's Sojourns Scholars Leadership Program. Dr Sudore is partly funded by the National Institute on Aging, National Institutes of Health (K24AG054415).

Authors’ Contributions

Sarah F. D'Ambruoso: first draft and revisions; Ron D. Hays: conceptualization, methodology, analysis, reviewing, and editing. Anne M. Walling: data curation, reviewing and editing, and supervision. Drs Wenger, Sudore, Gibbs, and Rahimi: reviewing and editing.

Data Availability Statement

Deidentified data can be made available upon reasonable request when overall study analyses are completed.

Declaration of Conflicting Interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

The study was approved by the UCLA Institutional Review Board (18-001612).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported in this report was funded through a Patient-Centered Outcomes Research Institute^® (PCORI^®) Award (PLC-1609-36291). The views presented in this publication are solely the authors’ responsibility and do not necessarily represent the views of PCORI^®, its Board of Governors, or the Methodology Committee. The contents do not represent the views of the US Department of Veterans Affairs or the United States Government.

Statement of Human and Animal Rights

All procedures in this study were conducted in accordance with the UCLA Institutional Review Board (18-001612).

Statement of Informed Consent

Written informed consent was obtained from the patient(s) for their anonymized information to be published in this article. Primary care providers were sent a secure e-mail to obtain permission to contact their eligible patients. Eligible patients were mailed a letter introducing the study that included a written consent form.

ORCID iDs

Sarah F. D’Ambruoso

Ron D. Hays

References

Jenerette

Mayer

. Patient-provider communication: the rise of patient engagement. Semin Oncol Nurs. 2016;32(2):134-143. doi: https://doi.org/10.1016/j.soncn.2016.02.007

Bull

Teede

Watson

Callander

EJ.

Selecting and implementing patient-reported outcome and experience measures to assess health system performance. JAMA Health Forum. 2022;3(4):e220326. doi:https://doi.org/10.1001/jamahealthforum.2022.0326

Drossman

Chang

Schneck

, et al. A review of the evidence and recommendations on communication skills and the patient–provider relationship: a Rome foundation working team report. Gastroenterology. 2021;161(5):1670-1688.e7. doi:https://doi.org/10.1053/j.gastro.2021.07.035

Hays

Skootsky

. Patient experience with in-person and telehealth visits before and during the COVID-19 pandemic at a large integrated health system in the United States. J Gen Intern Med. 2022;37(4):847-852. doi:https://doi.org/10.1007/s/11606-021-07196-4

Parast

Hays

, et al. Development and validation of a patient experience of care survey for emergency departments. Health Serv Res. 2022;57(1):102-112. doi: https://doi.org/10.1111/1475-6773.13853

Quigley

Qureshi

Palimaru

Pham

Hays

. Content and actionability of peer shadow coaching recommendations to providers for improving provider-patient interactions. Qual Manag Health Care. 2022;31(4):199-209. doi:https://doi.org/10.1097/QMH.0000000000000354

Quigley

Elliott

Slaughter

, et al. Shadow coaching improves patient experience with care, but gains erode later. Med Care. 2021;59(11):950-960. doi:https://doi.org/10.1097/MLR.0000000000001629

Chen

Hollis

SJ.

A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13(211). doi:https://doi.org/10.1186/1472-6963-13-211

Haskard Zolnierek

DiMatteo

. Physician communication and patient adherence to treatment: a meta-analysis. Med Care. 2009;47(8):826-834. doi:https://doi.org/10.1097/MLR.0b013e31819a5acc

10.

Swain

Hariharan

Rana

Chivukula

Thomas

. Doctor-patient communication: impact on adherence and prognosis among patients with primary hypertension. Psychol Stud. 2015;60(1):25-32. doi:https://doi.org/10.1007/s12646-014-0291-5

11.

Auerbach

. The impact on patient health outcomes of interventions targeting the patient-physician relationship. Patient. 2009;2(2):77-84. doi:https://doi.org/10.2165/01312067-200902020-00003

12.

Sharkiya

. Quality communication can improve patient-centred health outcomes among older patients: a rapid review. BMC Health Serv Res. 2023;23(1):886. doi:https://doi.org/10.1186/s12913-023-09869-8

13.

Katz

Resnick

Hoff

. Associations between patient experience and clinical outcomes in general mental health clinics: findings from the veterans outcomes assessment survey. Psychiatry Res. 2021;295:113554. doi: https://doi.org/10.1016/j.psychres.2020.113554

14.

McMahan

Tellez

Sudore

. Deconstructing the complexities of advance care planning outcomes: what do we know and where do we go? A scoping review. J Am Geriatr Soc. 2021;69(1):234-244. doi:https://doi.org/10.1111/jgs.16801

15.

Sudore

Lum

You

, et al. Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel. J Pain Symptom Manage. 2017;53(5):821-832. doi: 10.1016/j.jpainsymman.2016.12.331

16.

Walling

Sudore

Bell

, et al. Population-based, pragmatic trial of advance care planning in primary care in the University of California health system. J Palliat Med. 2019;22(S1):72-81. doi:https://doi.org/10.1089/jpm.2019.0142

17.

Chau

Sudore

Hays

, et al. Telephone outreach enhances recruitment of underrepresented seriously ill patients for an advance care planning pragmatic trial. J Gen Intern Med. 2023;38(8):1848-1853. doi:https://doi.org/10.1007/s11606-022-08000-7

18.

Hays

Walling

Sudore

Chau

Wenger

. Support for the use of consumer assessment of healthcare providers and systems communication items among seriously ill patients. J Palliat Med. 2023;26(9):1234-1239. doi:https://doi.org/10.1089/jpm.2022.0572

19.

Hays

Bjorner

Revicki

, et al. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res. 2009;18(7):873-880. doi:https://doi.org/10.1007/s11136-009-9496-9

20.

Cella

Choi

Condon

, et al. PROMIS® adult health profiles: efficient short form measures of seven health domains. Value Health. 2019;22(5):537-544.

21.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297-334. doi:https://doi.org/10.1007/BF02310555

22.

Kenny

. Cross-lagged panel correlation: a test for spuriousness. Psychol Bull. 1975;82(6):887-903. https://doi.org/10.1037/0033-2909.82.6.887

23.

Lee

. On testing functional constraints in structural equation models. Biometrika. 1985;72(1):125-131.

24.

Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1-55. doi:https://doi.org/10.1080/10705519909540118

25.

Buta

Anhang Price

Elliott

Hays

Cleary

. Methodological considerations when studying the association between patient-reported care experiences and mortality. Health Serv Res. 2014;50(4):1146-1161.

26.

Kahn

Tisnado

Adams

, et al.

Does ambulatory process of care predict health-related quality of life outcomes?

Health Serv Res. 2007;42(1p1):63-83. doi:10.1111/j.1475-6773.2006.00604.x

27.

Navarro

Ochoa

Chan

, et al. Will improvements in patient experience with care impact clinical and quality of care outcomes? A systematic review. Med Care. 2021;59(9):843-856. doi: https://doi.org/10.1097/ mlr.000000000000159862

28.

Street

. How clinician-patient communication contributes to health improvement: modeling pathways from talk to outcome. Patient Educ Couns. 2013;92(3):286-291. doi:https://doi.org/10.1016/j.pec.2013.05.004

29.

Bernstein

Fear

Mesfin

, et al. Patient-reported outcomes use during orthopaedic surgery clinic visits improves the patient experience. Musculoskeletal Care. 2019;17(1):120-125. doi: https://doi.org/10. 1002/msc.1379

30.

Quigley

Elliott

Farley

Burkhart

Skootsky

Hays

RD.

Specialties differ in which aspects of doctor communication predict overall physician ratings. J Gen Intern Med. 2014;29(3):447-454. doi:https://doi.org/10.1007/s11606-013-2663-2.

Associations of Patient Experience With Doctor-Patient Communication and Patient-Reported Physical and Mental Health in Seriously Ill Adults

Abstract

Keywords

Introduction

Methods

Sample

Measures

Results

Sample

Patient Experience and Self-Reported Health

Structural Equation Model

Discussion

Limitations

Conclusion

Footnotes

Acknowledgments

Authors’ Contributions

Data Availability Statement

Declaration of Conflicting Interest

Ethical Approval

Funding

Statement of Human and Animal Rights

Statement of Informed Consent

ORCID iDs

References