Abstract
Growing recognition of the importance of patient perspectives has changed health-care and qualitative research landscapes. This has contributed to growth in patient-reported outcomes (PRO) research studies. Development of a new PRO measure or adapting an existing PRO measure are significant milestones that can assist patient outcome assessment (e.g., disease burden, symptoms, and quality of life) and improve health-care decision-making. PRO measures, in general, are comprised of several elements. These include concept of interest, items, instructions, anchors and response options, recall period, and subscale and overall scores. These elements collectively capture the experiences and dimensions of health status of particular importance to patients.
While qualitative research can potentially play a role in PRO measurement development and adaptation, qualitative research has not been used widely to inform item generation of new PRO measures or the conceptual equivalence of content of adapted PRO measures. Here, we discuss why qualitative research should be used to bring patient perspectives to the forefront of PRO measures.
Embracing Content Validity and Bringing Patient Perspectives to the Forefront
The development of a robust and meaningful PRO measure requires combining both inductive and deductive research processes. Establishing content validity through qualitative methods is the first and critical step to ensure the items of a new PRO measure have meaning and relevancy to the population of interest (Lasch et al., 2010) and, where appropriate, ensure conceptual equivalence with the original and the adapted PRO measure (Wild et al., 2005). This process is iterative and encompasses two types of interviews: (1) concept elicitation interviews to generate PRO items and inform the domain structure and (2) cognitive debriefing interviews to confirm the items and domains that measure the intended concepts and to ensure the appropriateness, comprehensiveness, and interpretable of PRO measures by patients (Brod, Tesler, & Christensen, 2009; Hay et al., 2014; Lasch et al., 2010).
Direct patient input and grounding in patient data are necessary to develop PRO measures. To ensure a PRO measure has essential content validity, phenomenology (to provide a lens to understand the lived experience of individual) and grounded theory (to codify experience into a meaningful conceptual framework) can be used to develop a framework of PRO items and domains (Brod et al., 2009; Lasch et al., 2010). This framework is vital importance because it articulates concepts and represents the relationships between and across items and domains that undergird the PRO measure from patient perspectives. This is a precursor to forming the latent factor structure of a PRO measure with significant covariance and high inter-item correlation coefficients. This content validation of instrument development is key because this distinguishes successful from unsuccessful instruments (Haynes, Richard, & Kubany, 1995). No amount of psychometric analyses can recover a poorly conceived questionnaire.
Making Best PRO Items, Making a Successful PRO Measure
Sound PRO items are essential to a successful PRO measure. Concept elicitation and cognitive debriefing are key dimensions of qualitative interview methods. The phenomenological inquiry of understanding patient experience of a health/medical condition can be used to provide the open-ended approach necessary for concept elicitation. Individual interviews and focus groups are useful and valid methods for this elicitation (Brod et al., 2009; Lasch et al., 2010). As the goal of concept elicitation interviews is to generate PRO items that can capture the perspectives and aspects judged to be of great importance by and to patients, rich and in-depth data should be used to reach conceptual saturation.
Sampling and qualitative analysis is also important. Sampling for concept elicitation interviews should be purposive and diverse to adequately represent experiences of a health/medical condition of target population, for example, across different levels of severity of the condition, so as to capture the entire range of patient experience (Patrick et al., 2011). For some health/medical conditions (e.g., mucositis), it may help understand the impact of eating/talking difficulties and thus enrich the data (e.g., social interaction and social well-being) if concept elicitation interviews also involve key others, such as family caregivers (Cheng et al., 2007). Some aspects of importance of a health/medical condition would be shared by a majority of patients—and are termed “universal PRO items.” Other aspects may be experienced or considered to be important by only some patients: These are “rare PRO items.” Both should be included in items to maximize the content validity of the PRO measure.
The inductive process of moving from specific (items) to general PRO concepts (domains) and the iterative and interpretive process of constant comparison constitute a distinctive feature of the grounded theory approach of textual data analysis for PRO development. The core tenet for inductive analysis is to identify conceptual patterns and categories around an overarching PRO concept from interview data (Brod et al., 2009; Lasch et al., 2010). Constant comparison enriches data whether comparison of similarities and differences is made among patients with different severities of a health/medical condition and/or treatment regimens, and so on. The coding process and derived conceptual framework and patient quotations can document and archive systematically to support content validity. Although literature and clinical experts would not contribute as data for PRO measure, researchers can discuss the extent of the findings’ resonance with the published literature and clinical experts. The items, wordings, anchors and response options, recall period, and format and structure of the first draft of PRO should be fully supported by the data derived from the concept elicitation interviews prior to cognitive debriefing interviews.
Cognitive debriefing interviews evaluate patients’ responses to PRO measures in terms of retrieval of information, response judgment, and mapping (Willis, 2005). This provides assurance of content validity and accuracy of measurement, that is, that the PRO items and domains measure the intended concepts. This also ensures that the questions, format and structure of the PRO questions, are appropriate, comprehensive, and understandable to the target population (Patrick et al., 2011). Think aloud and verbal probing are two common methods for cognitive debriefing interview. With the think-aloud approach of cognitive debriefing interview, the interviewer reads each item or question of the PRO measure to the participant. Upon completion of each question by the participant, he or she is encouraged to think aloud or verbalize his or her thoughts about the question and to explain how he or she arrives at his or her response to each question (Brod et al., 2009; Willis, 2005). Alternatively, verbal probing approach of cognitive debriefing interview uses additional probing questions to elicit further information after the participant has answered each question. The probing questions may include: how did you interpret the question, how easy or difficult it was to understand the question, how easy or difficult it was to answer the question, and how did you arrive your response and issue on other elements of PRO measure (e.g., instruction, recall period, etc.). Verbal probing approach of cognitive debriefing interview can be used for self-administered PRO measure. In addition, it can either be performed after the participant has responded to each question (concurrent verbal probing) or at the completion of the whole PRO measure (retrospective verbal probing) (Brod et al., 2009; Willis, 2005). Verbal probing may have a distinctive advantage over think-aloud method to help identify specific issues and inform revisions of PRO measure, such as low relevancy or high ambiguity of items or instructions. Nevertheless, the decision to modify or add/delete an item and change instructions or format is not easy. In general, changes can be made when three or more participants reported similar problems with some aspects of the PRO measure. Decisions around revisions also require taking the problems and characteristics of the participants (e.g., education and literacy levels) into account (Hay et al., 2014). Sufficiently varied samples (e.g., including patients of lower educational attainment) should be used to inform revisions.
For an Adapted PRO Measure, Don’t Lose the Original Content Validity
The inclusion of qualitative methods during the translation and adaptation of an existing PRO measure is necessary to ensure cross-cultural content validity of measure. High-quality translation without rigorous evaluation and confirmation of conceptual equivalence of content with the original versus the adapted PRO measure may lose the original content validity of the PRO measure. Although considerable international efforts have been made to develop approaches or guidelines to support the process and research practice in translation and adaptation of PRO measures from original English language to target languages (Brod et al., 2009; World Health Organization [WHO], 2016; Wild et al., 2005), methodologies remain inconsistent. In general, independent and blinded forward and back translations, reconciliation and harmonization, and cognitive debriefing interviews and finalization are the major steps of translation and establishing content validity of an adapted PRO measure (WHO, 2016; Wild et al., 2005). As in new PRO development, in adaptation of existing PRO, rigorous cognitive debriefing interviews are needed to ensure the conceptual equivalence of the translated content as well as the translation is comprehensible to the target population in particular with participants of lower educational attainment.