Abstract
In September 2010, the US Food and Drug Administration (FDA) published a final rule governing the requirements for expedited safety reporting for products subject to an investigational new drug application. The rule clarified the types of safety information that qualify for expedited reporting. Its intent was to improve the overall quality of safety reporting by reducing the number of uninterpretable individual reports sent to the FDA and clinical investigators. In December 2011, we surveyed pharmaceutical and biotechnology sponsors regarding their safety reporting practices. We convened a group of experts and a biostatistics work group to review the survey results and identify gaps between current practice and the final safety reporting rule. Most sponsors had not changed their approach to expedited reporting of serious adverse events. We devised recommendations to help sponsors optimize their premarket safety systems to reduce the number of uninformative expedited reports and ensure recognition of important safety issues for an investigational drug as early as possible in development.
Keywords
Introduction
On September 29, 2010, the US Food and Drug Administration (FDA) published a final rule, effective March 28, 2011, that established new reporting requirements for serious and unexpected suspected adverse reactions occurring in clinical trials conducted under an investigational new drug application (IND). 1 The FDA published a draft guidance simultaneously with the final rule and issued a final guidance on December 20, 2012. 2 Under former regulations, sponsors were routinely reporting to the FDA and clinical investigators in all ongoing studies serious adverse events for which there was often little reason to believe that the drug had caused the event. This practice often complicated the efforts of clinical investigators and the FDA to recognize genuine drug safety problems and diverted resources from activities more likely to improve patient safety. 1,3
The final IND safety reporting rule clarified that sponsors should not submit expedited safety reports for individual cases of serious and unexpected adverse events for which there is little reason to believe that the drug caused the event. The final rule is intended to improve the overall quality of safety reporting by reducing the number of uninterpretable individual reports, allowing the FDA and clinical investigators to focus resources on the assessment and communication of more meaningful data. 1–3
The IND safety reporting rule implicitly requires the sponsor to review safety data collected across all completed and ongoing studies in an IND, analyze these data in the aggregate, evaluate the available evidence, and make a judgment about the likelihood that the drug actually caused the serious adverse event. 1–3 To comply with these requirements, sponsors must have systematic approaches for evaluation and communication of safety signals.
However, sponsors have noted that the changes introduced in the IND safety reporting rule present some challenges in implementation related to various methodological and global regulatory issues. 4 Recognizing that a full understanding of current practices would be useful in informing future recommendations, the Clinical Trials Transformation Initiative (CTTI) initiated a project on which this article is based. The project’s intent was to elucidate sponsors’ current approaches to assessing and managing safety information from all trials and other sources during an IND drug development program. The project sought to obtain a deeper understanding not only of the collection and organization of safety data but also of the processing, management, assessment, and communication of potential safety signals. While some of these practices have been previously described, 5,6 there is a need for more information on the details of potential approaches for complying with the final IND safety reporting rule.
Methods
Overview
Eight months after the effective date of the final IND safety reporting rule, we conducted an Internet-based survey of industry’s safety reporting practices. Subsequently, we convened a 2-day expert meeting to discuss survey results, share insights and concerns related to IND safety assessments, identify gaps between current practice and an IND safety reporting system optimized to detect and communicate valid safety signals as early as possible, and offer proposed solutions to these identified gaps.
In addition, we formed a biostatistics work group composed of biostatisticians from industry, academia, and the FDA to explore methodological issues and to develop recommendations related to the frequency and methods of performing aggregate analyses, methods of handling masked data in ongoing studies, thresholds for expedited reporting of serious adverse events, and approaches to assessment of IND safety information across multiple studies and sources of safety information.
Survey of Industry Practices
The project team developed a survey containing 54 open-ended questions that addressed pharmaceutical sponsors’ safety reporting practices. The questions were intended to elucidate the organizational structures and standard operating procedures that pharmaceutical companies have developed to monitor the safety of products being studied under an IND application. The survey was distributed electronically to representatives from 14 pharmaceutical sponsors, ranging from small organizations to large global companies. The project team asked participating sponsors to submit free-text responses electronically in the survey’s open-ended response boxes. No character limit restrictions were placed on the response boxes, and sponsors were encouraged to provide a thorough description of their current practices and any processes under development to comply with the final rule.
CTTI staff processed the completed surveys by removing any identifying information about the respondent’s role or organization and compiled the responses into a single document. Each de-identified set of responses was assigned an identification number and a corresponding color code before it was sent to the project team for review. The project team summarized the compiled responses according to 4 themes: (1) organizational structures, (2) handling of unmasked data from ongoing trials, (3) approaches to integrating safety data across trials and other data sources, and (4) defining thresholds for escalation of safety signals. The intent behind compiling responses in this way was to provide brief summaries to participants in the expert meeting that would create a common starting place to guide discussions at the meeting. All attendees also received the full range of raw data to review so that they might generate their own insights. After the meeting, the biostatistics work group, who also attended the expert meeting, met separately to draft a set of recommendations on the topics addressed.
Results
Survey
Twelve of the 14 industry sponsors (86%) who were invited to participate completed the survey and answered all 54 questions.
Organizational Structures
In general, the survey revealed that most sponsors had some type of multidisciplinary team, often called the “safety team,” generally led by a safety physician, to conduct primary reviews of individual case reports within a given trial. Members of the safety team determine whether individual reports meet criteria for reporting to regulators and investigators. The safety team may also generate aggregate safety reports as warranted by cumulative data from a trial. The safety teams commonly include safety physicians, toxicologists, pharmacologists, biostatisticians, and clinical scientists. Although the safety team is generally led by a safety physician, sponsors indicated that the safety team may also include or interact with the clinical lead and medical monitor.
In addition to the primary safety team assigned to a given trial, 11 of the 12 respondents indicated that they used safety management teams (SMTs), as advocated in the Council for International Organizations of Medical Sciences (CIOMS) VI 7 and by the Safety Planning, Evaluation, and Reporting Team, 5,6 to coordinate and integrate the individual safety teams’ analyses of safety information across all trials of a particular drug or therapeutic area. Seven of the 11 sponsors who stated that they used SMTs indicated that the SMTs are not involved with single-case processing and assessment, though some members of the SMT (eg, lead safety physicians) may have such responsibilities. Some sponsors indicated that the SMTs are organized at the product level, while other sponsors indicated that SMTs are organized at the level of therapeutic area. For products under development in multiple therapeutic areas, sponsors who organize SMTs at the level of therapeutic area indicated that multiple SMTs may share responsibility for the same product, depending on the indications under study. In addition, the survey responses suggested that most sponsors have organizational structures above the level of the SMTs, sometimes called “cross-functional review teams,” “senior staff teams,” “executive global safety teams,” or “supervising risk management teams.” In some sponsor organizations, members of SMTs may also serve on these executive-level teams.
The survey showed that sponsors typically, though not universally, collect data into separate safety and clinical trial databases. A common approach is to create a single global safety database for each product and a separate clinical trial database for every clinical trial. Global safety databases generally include data related to all serious adverse events reported in association with the product across all trials for all therapeutic areas and indications but do not include other clinical trial data, such as labs, concomitant medications, or special study results unless reported in association with a serious adverse event. Those additional data elements, as well as the data related to the serious adverse events, are available in the individual clinical trial databases. The survey also showed that sponsors periodically reconcile the safety and clinical databases. The frequency for reconciliation is dependent on the nature of the product and the phase of the trial.
Handling of Unmasked Data From Ongoing Trials
Overwhelmingly, sponsors indicated that they do not review unmasked or treatment-stratified data from ongoing masked clinical trials when evaluating potential safety signals. Eleven of the 12 respondents explicitly stated that the primary review of such safety data is performed in a masked fashion. The only routine exception to this practice is the unmasking of individual case reports to meet requirements for expedited reporting. The survey responses indicated that, instead of looking for imbalances in event rates across the treatment groups of ongoing masked studies, the safety teams evaluate overall adverse event rates in the entire study population. The safety teams rely on historical data, previous experience, literature review, and other sources, rather than internal controls, to establish comparators for the observed adverse event rates.
Data monitoring committees (DMCs) with access to unmasked or treatment-stratified data could detect imbalances in adverse event rates across treatment groups of a single trial. However, it was not clear from the survey responses how often DMC charters assign such responsibilities or whether DMCs typically have the resources to conduct such analyses even if they were given access to the raw, unmasked data. In addition, the sponsors indicated that, while dedicated safety teams within the sponsor organization review the data at very regular intervals, the DMCs generally meet less frequently (eg, monthly, quarterly, or even semi-annually) to review the safety data. Sponsor responses were varied as to whether the final IND safety reporting rule would require changes in the remit and practices of DMCs; some indicated that they did see a need, while others did not.
A few sponsors indicated that they have occasionally used internal safety committees, firewalled from the primary review teams, to review aggregate data in an unmasked or stratified fashion, allowing the sponsor to search for safety signals on the basis of imbalances in event rates across treatment groups from all trials. Those sponsors, however, did not provide details regarding the criteria that might invoke using such internal safety committees.
Approaches to Integrating Safety Data Across Trials and Other Data Sources
Once a potential safety signal has been identified, the review teams may analyze more granular data from the individual clinical trial databases, including laboratory and other ancillary data, in addition to data from the global safety database. The responses, however, did not provide significant detail regarding the methodologies used to do this. The survey further revealed that while the review teams may examine the clinical database for additional insights, these teams remain masked to treatment groups until the clinical database is locked.
The survey showed that most respondents use external resources and experts to assist in the safety evaluation of their clinical trial data on an ad hoc basis only. An exception to this principle is the use of DMCs to review unmasked data or data stratified by treatment arm from ongoing clinical trials. Seven of the 12 sponsors, however, reported that they at least occasionally used a single DMC to oversee 2 or more trials in a development program, and 1 sponsor indicated that mechanisms exist to allow communication between DMC chairs if multiple DMCs are used within a single program. Many sponsors indicated that a single point of contact at the sponsor, often a member of the internal SMT or a more senior management representative, acts as either a liaison to or member of the DMC. However, other sponsors indicated that contact with the DMC is more limited (eg, through the clinical team only or as stated in the DMC charter).
The survey indicated that all sponsors use external data sources, such as literature reviews, existing registries, Centers for Medicare and Medicaid Services data, the FDA’s Adverse Event Reporting System, EudraVigilance, and class labeling, when assessing potential safety signals. Some sponsors stated that they use or are developing specific tools, including fractional reporting ratios, standardized incidence ratios, network meta-analyses, data visualization tools, Multi-Item Gamma Poisson Shrinker, disproportionality analyses, and any newer techniques developed to assess potential safety signals. Other sponsors indicated that they rely on descriptive statistics in making comparisons between incidence rates observed in external populations and those observed in the clinical trial, rather than using these specific tools to utilize these data sources.
Defining Thresholds for Escalation of Safety Signals
Survey respondents indicated that individual adverse events deemed to be (1) serious, (2) unexpected, and (3) suspected adverse reactions are reported to the FDA in an expedited fashion, as required by regulation. The sponsor’s determination regarding whether an event meets all 3 definitions is made by an individual reviewer with the support of a larger safety team. In determining whether the adverse event is a suspected adverse reaction (ie, adverse event for which there is a reasonable possibility that the drug caused the adverse event), 6 of the 12 sponsors explicitly stated that they still rely on the most conservative judgment to determine causality—that is, despite the new IND safety reporting rule, which assigns responsibility for causality to the sponsor, 6 of the 12 respondents reported that they still send individual case reports based on the investigator’s determination of causality for cases in which the investigator assesses the causality as positive even if the sponsor assesses the causality as negative. The remaining 6 respondents did not clearly indicate whether they have limited their expedited reporting of individual events to those cases where the sponsor determines that evidence of causality exists.
Some sponsors indicated that, in certain situations, they may establish quantitative thresholds to help identify safety signals. Such thresholds can be found in protocols, standard operating procedures, and DMC charters. In many more situations, however, sponsors indicated that signal generation and refinement require clinical judgment. Risk management committees, SMTs, senior governance teams, or cross-functional review boards may interpret existing data and determine specific management responses.
The majority of sponsors indicated that they have a lower threshold for submitting an expedited IND safety report to the FDA based on aggregate data than for updating the investigator brochure (IB), though some indicated that IB updates would accompany any IND safety report that was based on aggregate data.
Some sponsors indicated that external data sources are given more weight when they suggest the presence of a signal than when they provide reassurance that a signal is false. For example, sponsors specifically stated that such external data sources might be used to support the decision to send an expedited report based on aggregate data but would not be used to support a decision to not send an expedited report. Other sponsors indicated that a decision to send an aggregate IND safety report based on aggregate data would depend on all available factors—nonclinical data, toxicology reports, biological plausibility, and previous experience.
Expert Meeting
The expert meeting was convened on February 28 and 29, 2012. A total of 51 participants attended the expert meeting. Participants described challenges related to the handling of unmasked data in ongoing trials, a lack of established methods for systematically interrogating clinical and safety databases, difficulties in identifying objective thresholds for triggering communications and other management interventions in response to safety signals, and a lack of global harmonization. Participants agreed that addressing these challenges would help optimize current IND safety assessment and communication practices. Preliminary ideas to address these gaps were discussed at the meeting.
The project team leaders (1 each from academia, industry, and the FDA) subsequently integrated recommendations of the biostatistics work group with proposals from the February 2012 expert meeting and formulated draft project recommendations. After review by participants in the 2012 expert meeting, the biostatistics work group, and members of CTTI’s Steering Committee, 8 CTTI’s Executive Committee 9 adopted the final recommendations listed in Table 1. The recommendations describe an approach to tracking safety of investigational drugs or biologics throughout a development program in a way that is consistent with the final IND safety reporting rule.
Recommendations from the Clinical Trials Transformation Initiative regarding safety assessment and communication for investigational new drugs and biologics.
Critical concepts in these recommendations include prospective identification and standardized terminology for serious adverse events anticipated to commonly occur in the study population independent of drug exposure or as manifestations of the disease being treated. The recommendations encourage sponsors to conduct periodic analyses of these anticipated events by treatment group in the full development program. They further suggest that analyses using unmasked data from ongoing trials be performed by firewalled safety committees, internal or external to the sponsor, and that extreme care be applied to ensure the integrity of ongoing trials. Additional FDA guidance is requested on mechanisms by which firewalled safety committees might notify appropriate individuals at a sponsor company of a safety signal in a way that balances patient safety and the integrity of an ongoing trial.
In keeping with the final IND safety reporting rule, judgment regarding expedited reporting of anticipated serious adverse events should be driven by aggregate analyses. In addition, it is recognized that even for serious adverse events not anticipated to occur in the study population and not previously known to be strongly associated with drug exposure, it is often necessary to have more than 1 occurrence before the sponsor can judge that there is a reasonable possibility that the drug caused the event. If there is uncertainty or weak evidence of causality, sponsors could consider reporting these presumably uncommon events as individual events via expedited reporting mechanisms to the FDA.
Discussion
Despite the issuance of the FDA’s final IND safety reporting rule, our survey found that 8 months after its effective date, most sponsors had not changed their approach to expedited reporting of serious adverse events. Cautious about protecting the integrity of ongoing trials, sponsors generally do not analyze the frequency of unexpected, serious adverse events by treatment group until a trial has ended and the data have been unmasked. Rather, to identify safety signals, sponsors examine the overall frequency of an unexpected, serious adverse event in drug, placebo, and active comparator groups combined and compare this rate to historical controls in a similar population of patients. Furthermore, sponsors vary in the extent to which they report a systematic approach to analyzing aggregate safety data across a full development program while trials are ongoing. Although some sponsors engage external committees to conduct such aggregate analyses across multiple studies in a development program, most have not gone in that direction.
The proper protection of trial participants should be the first priority of a premarket system optimized for the assessment and communication of safety data. While sponsors had developed databases and procedures that allowed them to efficiently meet their regulatory obligations under the previous IND safety reporting rule, those systems were typically built around the collection and reporting of individual serious adverse events. Expedited reporting of individual cases that are largely uninterpretable cannot replace the critical function of a systematic and sequential review of accumulated data throughout the development phase of a drug or biologic. Given the sponsor’s access to all available data, under the final IND safety reporting rule the FDA placed the responsibility of final causality determination on the sponsor. The sponsor is uniquely positioned to evaluate safety data in its entirety and to communicate timely and interpretable information to both regulators and investigators. 2,3
Perhaps the most contentious issue at the expert meeting was related to concerns about unmasking safety data in ongoing trials and about thresholds for reporting serious and unexpected suspected adverse reactions to regulatory agencies. The recommendations in Table 1 would apply to all IND programs for all sponsors. It might be a challenge to constitute an independent DMC to meet the needs of systematic aggregate analyses in every industry-sponsored drug or biologic trial under an IND. In a September 2012 download of data in ClinicalTrials.gov, there were 2674 open drug or biologic trials with US sites sponsored by industry in phases 2 or 3 with more than 1 study arm, of which 1453 did not report having a DMC (K. Chiswell, PhD, senior biostatistician, Clinical Trials Statistics, Duke Clinical Research Institute, personal communication, September 20, 2013). There could be a mismatch between the demand for DMC personnel to support all these trials and the supply of DMC statisticians and other experienced personnel. Therefore, the recommendations in Table 1 include the concept of firewalled safety committees functioning within a sponsor organization provided that care is taken to preserve the integrity of ongoing studies when such committees incorporate unmasked data in aggregate analyses.
Although our recommendations (Table 1) seek to promote practices that will improve patient safety and facilitate regulatory review, they do not purport to be a comprehensive solution to all the challenges faced by IND sponsors. For instance, the recommendations do not address the difference between the FDA’s IND safety reporting rule and the International Conference on Harmonisation (ICH) guideline, ICH E2A, 10 with respect to who is responsible for making the causality assessment that determines reportability. Under the IND safety reporting rule, the sponsor considers the investigator’s causality assessment for unexpected and serious adverse events but submits an IND safety report only for those events for which the sponsor determines there is a reasonable possibility that the drug caused the event. In contrast, ICH E2A requires sponsors to report serious adverse events that either the sponsor or the clinical investigator deems possibly related to the study drug. 10 Sponsors have commented that the difference in the party responsible for the causality assessment that determines reportability is not only operationally complex but also raises concerns about legal liability for some sponsors. While harmonization on the issue of determination of causality would be desirable, it is important to note that there is a degree of existing flexibility built into European and US regulatory requirements that allows for approaches that might reduce the number of uninterpretable safety reports submitted to regulators and investigators under either system. The European Clinical Trials Directive and the CT3 guidance include provisions for defining categories of serious adverse events that will not require expedited reporting. 11,12 Serious adverse events that serve as efficacy endpoints and serious adverse events that are disease-related (ie, anticipated to occur in the study population) are covered under these provisions. Although the language of these European regulations appears to align with the FDA’s final rule against reporting study endpoints or anticipated serious adverse events in individual IND safety reports, further discussions with European regulators would be beneficial to explore the actual level of agreement on the issue of pre-specifying in a trial protocol anticipated serious adverse events that would not be reported as individual case reports in an expedited fashion.
We expect that adoption of the CTTI recommendations on IND safety assessment and communication would both reduce the number of uninformative expedited reports in IND studies and help ensure recognition of possibly important safety issues for an investigational drug as early as possible in a development program.
Conclusion
In response to uncertainty about how to best oversee patient safety during ongoing drug and biologic development programs, we have put forward a set of recommendations for safety monitoring and communications in drug or biologic development programs that attempts to balance concerns for patient safety and the integrity of ongoing clinical trials. Going forward, it would be helpful to have additional regulatory clarification and harmonization of any discrepant components of international safety reporting regulations.
Footnotes
Authors’ Note
The views expressed herein represent those of the authors and do not necessarily represent the views or practices of the authors’ employers or any other party.
Acknowledgments
We gratefully acknowledge the assistance of CTTI's project manager, Cheri Janning (Duke Translational Medicine Institute) and the following individuals who contributed to the project team: Elliott Levy, MD, and Janice Wherry, MD, PhD (both from Bristol Myers Squibb), and Robert Temple, MD, and Janet Norden, MSN, RN (both from the Center for Drug Evaluation and Research, US FDA). We also appreciate the input of the biostatistics work group, led by Janet Wittes, PhD, whose members and recommendations can be found on the CTTI website (
). We thank Amanda McMillan, MPH (Duke Clinical Research Institute) for editorial assistance with the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was conducted by in-kind contribution of effort by authors and organizations working with the Clinical Trials Transformation Initiative (CTTI; ), as well as staff effort and meetings supported by CTTI funding (derived both from pooled fees from member organizations and from a cooperative agreement, U19FD003800, awarded to Duke University by the US Food and Drug Administration).