Abstract
Recent changes to the US Federal Policy for the Protection of Human Subjects (the “Common Rule”) require that Informed consent must begin with a concise and focused presentation of the key information that is most likely to assist a prospective subject or legally authorized representative in understanding the reasons why one might or might not want to participate in the research.
1
This policy revision draws special attention to the need for informed consent documents to consider not only what information must be included but also on how to present that information. The translation of this policy into practice requires informed consent documents to be restructured, placing the information most relevant to clinical trial enrollment decisions front and center.
Various guidelines indicate the kinds of information that informed consent documents need. For example, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Good Clinical Practice (ICH-GCP) lists 20 categories of information. 2 However, these guidelines are relatively silent about how to set priorities among and communicate such information, as required by the revisions in the Common Rule. Absent such priorities, potential trial participants may allocate their attention inefficiently or be frustrated by the difficulty of navigating complex consent documents. 3
Recognizing these threats, researchers have examined many aspects of informed consent document design. Systematic reviews exist for methods that include reducing text, using multimedia, and simplifying language.4,5 Most tests of actual documents have done everything possible to make them useful. As a result, there are relatively few controlled tests of how individual design elements affect their usability. Here, we provide such a test, examining several theoretically plausible ways of satisfying the revised Common Rule. We evaluate them in terms of how much information potential trial participants remember, for the most relevant information and overall, as defined by individuals drawn from a similar population.
Restructured Informed Consent Documents
The present study enlisted individuals with self-reported asthma in evaluating an informed consent document for an injectable drug for treating severe asthma, with the identity of the study sponsor and investigators anonymized. In order to determine enrollment-relevance, we conducted a preliminary study asking participants to rate how each statement in an abridged version of the document would affect their willingness to enroll, using a visual analogue scale, anchored at 0 = no impact and 100 = great impact. Further details appear in Supplement 1.
We developed five versions of the informed consent document, each reflecting a different, seemingly plausible design strategy:
As a baseline comparator, we used the original full-length consent document drawn from the clinical trial, provided to the authors by ICON, PLC. Excluding the signature page, it had approximately 6,000 words, across 16 pages and 13 sections.
The concise version of the document, developed and evaluated in a previous study, 6 contains all statements from the original version that two thirds of the participants in that study indicated as useful, along with additional information recommended by ICH-GCP. It followed the order and organization of the full-length version, but had only about 2,000 words, across 5 pages and 8 sections. In that earlier study, 6 when compared with the full-length version, the concise version had similar levels of comprehension, with reduced reported effort. The three other versions of the document used the material in the concise version, restructured in ways that might improve recall and understanding.
The interactive version presented each section of the concise document on a separate web page to which participants could navigate, from an initial landing page with the introductory section. The section tabs followed the order of the concise document, as did the information within each section. This design allows participants to follow their natural sense of what seems relevant. The risk of such a design is that participants might not navigate to information whose importance they would realize only if they saw it.
The reordered version had the same introductory information as the concise version, but its remaining sections were reordered according to the enrollment-decision relevance ratings from the preliminary study. This structure might improve understanding by focusing users on the most relevant issues, as defined by people like themselves who had seen the entire concise version.
The highlights version added a summary box at the top of the concise version, with the heading, “Information identified as important by other patients.” It operationalized this requirement of the revised Common Rule 1 by including the 10 statements with the highest enrollment-decision relevance ratings in the preliminary study. Although such advanced organizers can aid learning, they also add length, redundancy, and complexity to the document. 7 This box lengthened the concise version by 188 words.
Full texts of the five versions are in Supplement 2.
Measures
The knowledge test had 20 questions designed to capture the ICH-GCP’s 20 required components of informed consent. 3 Every question had 4 items, each of which could be true or false. Weighting these true-false items equally meant that the test could be viewed as an 80-item true-false test with a potential range of 0 to 80. Each item could be categorized into one of three groups. Nine items related to statements with mean ratings of at least 50 in the preliminary study were categorized as high-relevance items. Forty-two items related to statements with mean ratings less than 50 were categorized as low-relevance items. Twenty-nine items were unrated because they neither appeared in the full-length nor the concise version. For example, one asked whether nausea was noted as a side effect. As it was not, it did not appear in the consent document and was not rated in the preliminary study. The full knowledge test appears in Supplement 3.
Methods
Participants
Participants with self-reported asthma were recruited on Amazon Mechanical Turk on December 14 and 15, 2017. They received $5 for completing the study, which took about 25 minutes.
Procedures
Participants were told that results of the research might be used to inform the design of informed consent documents and that their responses could help the decision making of future potential clinical trial participants. Participants were randomly assigned to read one of the five formats of the informed consent document. Given the goals of the research and to encourage more naturalistic reading, participants were told, “We are interested in how such a form might be understood in a real context. We ask that you read the consent form as you would if it were presented to you to learn more about a clinical trial.” Participants could then interact with the informed consent document in whatever manner they preferred (e.g., the time spent reading each section). After reading the informed consent document, participants answered the knowledge test.
Prior to reviewing the consent documents, participants reported their asthma on the PROMIS Asthma Impact Scale. 8 After reviewing their version of the consent document and prior to completing the knowledge test, participants answered secondary outcome measure questions, including their likelihood of enrollment, confidence in their enrollment decision, satisfaction with the informed consent document, trust in the trial investigators, probability of treatment effectiveness, and perception of key information. Discussion of these secondary measures are included in Supplement 5. Finally, participants answered demographic questions.
Carnegie Mellon University’s Institutional Review Board approved the study. Participants read a description of the study and its purpose before agreeing electronically to continue.
Analysis
The primary study outcome is participants’ knowledge score for the nine high-relevance items. To test for differences across versions, we used a mixed-effects logistic regression, with version as a fixed effect and the original full-length version as the baseline. Participants and items were included as random effects. For the primary outcome measure, we use a Bonferroni correction to account for multiple comparisons, thus using an α = 0.0125 as a threshold for significance.
Secondary outcomes include knowledge scores on all items, and on items with low and no enrollment-relevance ratings. Additional analysis on items linked to ICH-GCP categories, as well as qualitative assessments of the informed consent documents can be found in Supplements 4 and 5.
The analysis plan can be located on ClinicalTrials.gov with the identifier NCT03416907.
Results
Participants
Five hundred and fifteen individuals began the study. Of those, 495 provided complete responses, and are included here. Participants were prompted to indicate whether they were currently taking any asthma medications (73% were). These self-reports were screened to identify participants who appeared to have misrepresented their asthma status. No obvious cases were found. Table 1 reports participants’ demographics.
Participant Demographics
Test Scores of High-Relevance Items
Our primary outcome measure was participants’ scores on the high-relevance items in the knowledge test. As seen in Table 2, participants who read the full-length version performed worse than did participants who read each variant of the concise version. This difference was statistically significant for the highlights version, P = 0.0105.
Knowledge Scores by Version and Item Ratings a
Shaded cells indicate the highest value across versions; Number of Items indicates the maximum possible score.
Test Scores for Other Item Sets
Looking at test scores for all 80 items, the 42 low-relevance items, and the 29 unrated items, there were no significant differences between groups (see Table 2). For the full test, participants who read any variant of the concise version scored better than participants who read the full-length version. Again, those reading the highlights version had the highest average score. For the low-relevance items, those who read the reordered and interactive versions had the lowest scores. For the unrated items, those who read the full-length version had the lowest scores.
Discussion
Informed consent procedures must not only provide participants with information that might be relevant to their enrollment decisions but must also help them understand that information and evaluate its relevance. In actual trials, investigators and clinicians can provide valuable support. However, the revised Common Rule places some of that responsibility on the informed consent document itself. It also provides guidelines on how that should be done, requiring those documents to highlight the information most relevant to enrollment decisions. In previous work, we offered and evaluated a procedure for making consent documents more concise, finding that it improved comprehension. 6 Here, we examine ways of making that information more accessible, comparing them with the full-length version, focusing on mastery of the items most relevant to enrollment decisions.
We find that the highlights version, which aligns most closely with the revised Common Rule, performs the best. Participants who read it scored significantly higher in the knowledge test on high-relevance items than those who read the original full-length consent document. One possible explanation for that difference is that the highlights box draws attention to the information. A second is that it conveyed social information, as “Information identified as important by other patients,” or as what the investigators deemed as relevant. The finding that all concise versions were at least as good as the full-length version is consistent with our prior results 6 and could reflect some combination of reduced distraction, fatigue, cognitive load, or simply a more readable document. Further research is needed to disentangle these effects.
Our study has several limitations. First, as participants considered hypothetical enrollment in a clinical trial, they may have read the documents differently than they would for an actual trial. We have no specific hypotheses on how those differences might have affected our results. Second, participants were recruited from the Amazon Mechanical Turk online panel. While the decision-making processes of panel members have been found to be similar to the general population on other tasks,9,10 they should be more familiar with research procedures than the usual lay population and perhaps better able to master informed consent documents. Third, participants in our sample are more likely to be college-educated than the general US population, also facilitating that mastery. Fourth, there is also growing concern that Amazon Mechanical Turk tasks may elicit low effort responses. As a planned exclusion criterion, we asked participants to indicate any asthma medication they were currently using. We planned to exclude those who provided incoherent responses, but identified none.* For the purposes of our study, some low effort (but not random) responses are a desirable feature of the sample population, as we might expect changes to the structure of the consent document to be particularly beneficial for those who may be unable or unwilling to read the full consent document carefully. Last, we relied on judgments of individuals with asthma both to select items for the concise version 6 and to set priorities among them (Supplement 1). Doing so is consistent with the desire to include potential participants in designing informed consent procedures, but it is not a substitute for also including input from clinicians and ethicists.
The new Common Rule addresses the widely recognized need to provide potential participants in clinical trials with information that is sufficient to make decisions about their own care, in a comprehensible format. It offers one specific recommendation that is consistent with cognitive research and demonstrated its efficacy here—concisely highlighting relevant material. We offer an approach to identifying that material, using professional guidelines to identify sufficient information, and ways to format material that are steps toward the goals of hiding nothing, while directing attention to information that people like potential participants find most relevant—and evaluating the success of the communications in terms that allow institutional review boards to determine their adequacy.
Conclusion
Participants remembered the most relevant information best on the highlights version, following the revised Common Rule recommendation of adding a box with the most relevant facts. They remembered significantly more than did participants provided the full-length informed consent document. Moreover, that improvement came without affecting recall of items with lower or no relevance ratings. In all comparisons, participants receiving any variant of the concise version performed at least as well as did those receiving the full-length version. As discussed above, there were a priori reasons to think that each variant of the concise version might improve or degrade performance. The similarity of scores across the concise versions suggests that those factors balanced out or were immaterial.
Supplemental Material
icip1.2.2-c-msS1-20190216_final_online_supp – Supplemental material for Implementing a New Common Rule Requirement for Informed Consent
Supplemental material, icip1.2.2-c-msS1-20190216_final_online_supp for Implementing a New Common Rule Requirement for Informed Consent by Michael Yu, Baruch Fischhoff and Tamar Krishnamurti in MDM Policy & Practice
Supplemental Material
icip1.2.2-c-msS2-20190216_final_online_supp – Supplemental material for Implementing a New Common Rule Requirement for Informed Consent
Supplemental material, icip1.2.2-c-msS2-20190216_final_online_supp for Implementing a New Common Rule Requirement for Informed Consent by Michael Yu, Baruch Fischhoff and Tamar Krishnamurti in MDM Policy & Practice
Supplemental Material
icip1.2.2-c-msS3-20190216_final_online_supp – Supplemental material for Implementing a New Common Rule Requirement for Informed Consent
Supplemental material, icip1.2.2-c-msS3-20190216_final_online_supp for Implementing a New Common Rule Requirement for Informed Consent by Michael Yu, Baruch Fischhoff and Tamar Krishnamurti in MDM Policy & Practice
Supplemental Material
icip1.2.2-c-msS4-20190216_final_online_supp – Supplemental material for Implementing a New Common Rule Requirement for Informed Consent
Supplemental material, icip1.2.2-c-msS4-20190216_final_online_supp for Implementing a New Common Rule Requirement for Informed Consent by Michael Yu, Baruch Fischhoff and Tamar Krishnamurti in MDM Policy & Practice
Supplemental Material
icip1.2.2-c-msS5-20190216_final_online_supp – Supplemental material for Implementing a New Common Rule Requirement for Informed Consent
Supplemental material, icip1.2.2-c-msS5-20190216_final_online_supp for Implementing a New Common Rule Requirement for Informed Consent by Michael Yu, Baruch Fischhoff and Tamar Krishnamurti in MDM Policy & Practice
Footnotes
Acknowledgements
The authors would like to thank Adam Burston, Adwoa Bonsu, Sarah Moss-Horowitz, and Pauline Ting for their assistance in coding data for the preliminary study, and Dr. Robert M. Arnold as well as members of the CRISMA Center at the University of Pittsburgh for their helpful comments on this research.
All work was performed in the Department of Engineering and Public Policy and the Department of Social and Decision Sciences at Carnegie Mellon University and the Department of General Internal Medicine at the University of Pittsburgh.
The work has been presented at the Society for Medical Decision Making in November 2017.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Financial support for this study was independently provided by grants from ICON, PLC, the National Institutes of Health (NIH-5KL2-TR001856-02), and the Swedish Foundation for the Humanities and Social Sciences. All funding agreements ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Supplemental Material
*
Twenty-seven percent of participants reported not taking any medications, which is a common practice among people with asthma.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
