Sage Journals: Discover world-class research

Abstract

Introduction:

Artificial intelligence (AI) tools continue to be developed and used within the life sciences. The impact of these tools on the biosecurity landscape surrounding mail-order DNA synthesis and how to address the impacts have not been critically examined in the literature.

Methods:

The impacts of AI-driven chatbots and biological design tools on the biosecurity landscape surrounding mail-order DNA synthesis were analyzed and described. The findings are informed by the authors' experience in the field.

Results:

Generally, chatbots lower barriers to access of information that could be misused while biological design tools may provide new abilities to users with the intent of misuse. Six recommendations to the United States Government that attempt to maximize the benefits of these new technologies while mitigating risks are provided.

Conclusion:

Mandating mail-order DNA synthesis providers to screen DNA synthesis orders is a critical safeguarding step that should be taken as soon as possible. Over time, biological design tools will reduce the effectiveness of such a regulation and actions should be taken now to limit the negative impacts in the future.

Introduction

By applying engineering principles to the genetic code of living systems, synthetic biology tools may be used to give organisms new capabilities and characteristics that can usefully address problems in public health, medicine, ecology, and agriculture, among other areas. The field of synthetic biology is rapidly advancing: whereas early practitioners were able to replace DNA in one organism with DNA from another, today, scientists are able to replace DNA in an organism with almost any genetic sequence.

Commercial companies that synthesize and mail DNA to customers is one way for researchers to access custom DNA. The ability to reliably obtain custom DNA from mail-order DNA synthesis companies, combined with an increased understanding of what genes do and how to introduce the DNA into living organisms, has served as the foundation for synthetic biology to make meaningful advances.

The field of synthetic biology, and the life sciences writ large, is now integrating artificial intelligence (AI) and machine learning (ML) to accelerate discovery.¹ The biosecurity community has been increasingly focusing on a category of ML models called large language models (LLMs), which can be applied to research as LLM-based chatbots or as LLM-based biological design tools (BDTs).^2–5 LLM-based chatbots—such as ChatGPT, Bard, and Claude—can enable users without programming expertise to access information synthesized from a variety of sources.

These tools learn a language's patterns and rules from vast amounts of natural language training data and use this knowledge to generate text-based responses to a user's query. If the training data include scientific sources such as journal articles, then LLM-based chatbots may answer queries about scientific and medical concepts and can even be used to design scientific experiments.^2,6 In contrast to LLM-based chatbots trained on natural language, LLM-based biodesign tools are trained on sequences of nucleic acids, amino acids, or glycans.^7–9

BDTs can produce outputs in these same “languages,” generating new molecules based on inherent patterns, structures, and rules. LLM-based BDTs are a relatively new subset of the larger collection of BDTs, which are purpose-built software tools to engineer biological molecules or systems. When combined with access to mail-order DNA synthesis, these technologies may help synthetic biologists design both the DNA sequences that underpin their work and the experimental protocol to use them.

The ability to generate and obtain custom nucleic acid sequences raises longstanding and widely recognized concerns regarding the potential for misuse.¹⁰ Although mail-order DNA synthesis is not the only way for someone with ill-intent to recreate or engineer existing pathogens and toxins, using this service could lower barriers for a nefarious actor.

Having recognized the biosecurity implications of mail-order DNA synthesis, the U.S. Department of Health and Human Services (HHS) in 2023 issued Screening Framework Guidance for Providers and Users of Synthetic Nucleic Acids,¹¹ an updated version of a 2010 guidance that was modified in light of public input and advancements in molecular biology, genetics, and DNA synthesis capabilities. Both versions of the guidance encourage providers to perform sequence and customer screening to evaluate the risk of misuse from fulfilling a specific order for a synthetic DNA product.

The guidance has historically not been required or legally enforceable, although many international mail-order DNA synthesis providers that are part of the International Gene Synthesis Consortium (IGSC) adhere to or exceed the screening recommendations in the HHS framework. However, the subsequent signing of the Executive Order on Safe, Secure, and Trustworthy AI¹² will require Federally funded synthetic DNA purchases to be from suppliers that meet yet-to-be determined screening standards. The development and implementation of these standards must consider how the capabilities of DNA synthesis may change and how the convergence of other technologies, such as AI and ML, affect the underlying technology and its potential safeguards.

In this study, we consider the effect of LLM-based chatbots and BDTs on the biosecurity paradigm of mail-order DNA synthesis. We discuss how LLM-based chatbots may impact barriers to accessing information prone to misuse and how BDTs could be used to exacerbate vulnerabilities or evade safeguards. We also discuss how the same technologies could be used to build or improve the same safeguards that could be evaded.

Finally, we provide recommendations that, if enacted, will inform the development and implementation of a screening protocol as required by the 2023 executive order to most effectively mitigate the risk of misuse of mail-order DNA synthesis in the age of AI. Although we recognize that not all genetic editing relies on synthesized DNA and that relevant advances in enzymatic and benchtop DNA synthesis¹³ could provide alternate sources of synthesized DNA, we suggest that regulating mail-order DNA synthesis is a vital course of action that should be implemented as part of a comprehensive biosecurity strategy.

Impact of AI/ML on Mail-Order DNA Synthesis Security

LLM-based chatbots and BDTs influence mail-order DNA synthesis security differently. LLM-based chatbots affect biosecurity by lowering barriers required to access information that would enable one to misuse biology. BDTs, however, may enable users to evade security measures and exacerbate existing vulnerabilities. Both chatbots and BDTs can be used to strengthen biosecurity measures as well. Because these technologies have distinct capabilities, use cases, users, and potential risks and benefits, their governance should also be considered separately. In this section, we present examples of the possible effect that LLM-based chatbots and BDTs can have on the potential to misuse mail-order DNA synthesis and consider the possible risks posed by each type of model, including impacts on current safeguarding measures.

Lowering Barriers to Access of Information That Enables Misuse

LLM-based chatbots can summarize and synthesize information found on the internet without the user having programming or subject-matter expertise.¹⁴ This information, however, could relate to the misuse of biology. For example, if a nefarious actor wanted help in identifying how to acquire restricted DNA, they could start by asking an LLM-based chatbot to identify an initial list of DNA synthesis providers that are not members of the IGSC. Using the chatbot's answer, the nefarious actor could further evaluate if any of the providers specify a screening process on their website.

Without the chatbot and with a bit more time, Google and pen and paper could be used to similar effect. Although there is no guarantee that companies identified through such an assessment do not screen orders, use of an LLM-based chatbot reduces the required time and a priori knowledge to perform the assessment. Among numerous other tasks, chatbots can help identify scientific protocols to follow as well as brainstorm and troubleshoot ideas. Chatbots thereby lower one barrier to accessing information that could enable the misuse of biology.

Although chatbots lower barriers to accessing this type of information, it is unlikely doing so will meaningfully expand the number of individuals who are able to misuse biology. Structured assessment is currently being undertaken to better characterize the impact of chatbots on the ability to develop a plan to misuse biology.¹⁵ Access and knowledge of how to access synthetic DNA is an early step in a long pathway to the misuse of biology. Subsequent steps become increasingly complex and complicated. They rely on explicit and tacit molecular biology knowledge and, often, living organisms.

Depending on the scenario of misuse and to protect the person performing the experiments, subsequent steps may also require additional laboratory infrastructure. It is hard to imagine a capable and motivated malicious actor being deterred by the time barrier required to identify a nonscreening provider. Alternatively, those who would be deterred by the time barrier are less likely to succeed in subsequent steps in misusing biology. Therefore, chatbots are unlikely to change a nefarious actor's ability to misuse biology. And even though it can be side-stepped, voluntary screening of DNA synthesis orders remains a valuable biosecurity practice. It creates a broad barrier to access of material prone to accidental and deliberate misuse.

Evading Safeguards

Biological design tools that can identify and design genetic sequences with desired functions have revolutionized science and medicine, for example, by engineering therapeutic antibodies and other biomanufactured proteins. However, BDTs could also help users to evade current DNA synthesis screening protocols by designing new sequences that are not included on the list of known pathogens that is checked before synthesis.

For example, a BDT user could design a protein that is predicted to be functionally and structurally similar to a known toxin but is encoded by a different DNA sequence. Under current screening guidance, this order would likely be fulfilled by most gene synthesis providers because the new sequence does not match a known sequence of concern. This method of evasion would be applicable to most, if not all, functional biomolecules but in the near term most feasible for individual genes (e.g., toxin genes).

Currently, use of this method of evasion to acquire novel pathogen genomes is not feasible because the use of BDTs to design novel genomes is not feasible. Given that the magnitude of potential harm ranges significantly between acquiring a toxin and a novel pathogen, Federal standards to address concerns with evading existing screening measures through recoding DNA should be informed by cost–benefit analysis that considers the capability of existing tools. However, such evasion techniques are unnecessary if a nefarious actor can order from a provider that does not screen DNA synthesis orders.

Currently, BDTs are at varying stages of development and require wide ranges and types of subject matter expertise to appropriately operate. These tools generally guide the engineering of an individual protein or biomolecule and have not been reported to have the ability to design entirely novel pathogens. Because BDTs operate within the context they are built, they currently are unlikely to succeed in suggesting how to impart new functionality to existing pathogens.

However, LLM-based chatbots may function as a useful tool for malicious actors to generate ideas and gather information on how to use known molecular and synthetic biology techniques to enhance pathogens or turn benign organisms pathogenic. We anticipate future developments in BDTs, including more accurate genotype-to-phenotype predictions, could result in the ability of technical experts to design previously unobserved proteins and pathogens that would evade list-based screening measures.

Exacerbating Vulnerabilities

The 2023 HHS Guidance does not immediately call for screening orders of short single stranded DNA (ssDNA) because these sequences are generally not unique enough to be an effective target; they appear many times and in various locations throughout nature. In addition, the cost of routinely investigating matches between a short sequence and a pathogen genome during screening would place undue burden on the company synthesizing the DNA.¹⁶ However, these short sequences could be misused if they are assembled into a larger double stranded DNA (dsDNA) sequence of concern using standard molecular biology protocols such as Gibson Assembly.¹⁷

This ability to order unscreened ssDNA and then assemble it into larger pieces of dsDNA results in a biosecurity vulnerability and BDTs combined with LLM-based chatbots may exacerbate it. Today, publicly available BDTs can help users identify the best ssDNA fragments to combine to make the desired dsDNA. However, technical challenges limit the range of dsDNA an actor can construct, as some pieces of dsDNA are more difficult to assemble from ssDNA than others. DNA containing long homopolymers, significant repeats, or low complexity are particularly challenging to assemble and mail-order DNA synthesis companies may even decline to attempt to provide them. Improved BDTs could potentially overcome such difficulties and potentially expand the extent to which ssDNA can be used to construct larger dsDNA constructs.

BDTs that help to find suitable sets of ssDNA to assemble into dsDNA are relatively mature compared with other types of BDTs. They are, therefore, currently better positioned to interface with LLM-based chatbots, similar to the recently reported GeneGPT, a chatbot that responds to human language prompts containing biomedical questions by providing executable code to interact with NIH databases and tools through an application programming interface (API).¹⁸

Although the researchers did not enable GeneGPT to directly interact with the tools and databases, they successfully demonstrated the potential of a direct connection. This process could be used to train a chatbot to interact with BDTs through an API and respond to human language prompts with specific ssDNA sequences to be synthesized and assembled into larger dsDNA. Based on the abilities of current BDTs and the technical expertise required to correctly assemble ssDNA into dsDNA, it remains unlikely that in the near-term chatbots integrated with current BDTs will expand the number of individuals that are able to misuse biology.

Opportunities: Building New Safeguards

Development of AI tools would benefit from a “violet teaming” process.¹⁹ Violet teaming is a newly coined term that takes inspiration from how red teaming methodologies are used to test cybersecurity. In cyber red teaming exercises, a red team attempts to penetrate a protected computer system to demonstrate the capacity to do harm.²⁰ A violet teaming process builds upon red teaming by identifying the threats arising from a technology, such as an AI system or tool, and then using the same technology to build tools that address the threat but without creating a harmful product. The goal is to create a feedback loop that maximizes the potential benefit of the technology while restricting the potential harms. This process embraces a security-oriented mindset throughout the technology development cycle and ensures biosecurity considerations are prioritized.

The previously discussed technologies that BDTs are built upon could be used to build tools that are part of a multilayered approach that supplements current list-based screening mechanisms. For instance, the underlying ML frameworks of AI tools that create phenotypic predictions from genotypic data (and evade sequence-based screening, as discussed in the Evading Safeguards section) could be used to develop a DNA synthesis screening tool that identifies sequences that are genotypically distinct but phenotypically similar to sequences of concern.²¹

Additional tools could predict protein structure of a requested DNA sequence and compare the predicted structure with those of pathogenic proteins. In addition, tools could be developed to predict and screen the most likely dsDNA products from an order of shorter ssDNA oligos, enabling cost- and time-efficient screening of ssDNA orders. Adoption of these tools, however, will require practical considerations to be overcome, such as establishing guidelines for fulfilling orders based on predictions as opposed to the definite knowledge of sequence similarity that is used today.

Recommendations

Minimizing the negative impacts and maximizing the benefits of AI/ML on mail-order DNA synthesis security should be a national priority. Importantly, effective policy will separately address LLM-based chatbots and BDTs due to their differences in technological maturity, required expertise to use, and impact on the threat landscape. With the inclusion of DNA synthesis screening in 2023's Executive Order on Safe, Secure, and Trustworthy AI, the United States has the opportunity to be a global leader to promote risk-reducing practices as international norms and to collaborate internationally. As the relevant government agencies consider how to achieve the goals set forth by the Executive Order, we offer the following recommendations to safeguard mail-order DNA synthesis for beneficent purposes: 1.

In establishing a screening framework as part of 4.4 (b)(i), the Director of OSTP should ensure such a framework includes screening protocols as described in the 2023 guidance or processes that are more stringent. Mandatory screening would identify orders that contain sequences of concern and lower the risk of their deliberate or unintentional misuse. This legislation would also mitigate the risk of an LLM-based chatbot suggesting companies that do not screen orders and make it more challenging for individuals to deliberately evade safeguards. Although some actors may be able to design new sequences that circumvent list-based screening, the 2023 guidance expands the scope of regulated sequences and is a viable first step to limit the harm associated with DNA synthesis. Most, if not all, U.S.-based gene synthesis companies already have processes in place to comply with the 2010 guidance that could be updated to align with the 2023 guidance.

Congress should enact legislation to only allow Federal funds to be used to purchase gene synthesis products from U.S.-based gene synthesis providers or gene synthesis providers that screen to the same standard. Codifying the safeguarding measures of the Executive Order into law will ensure that they are resilient to changes in administration. As part of this process, incentives for gene synthesis providers should be considered to ensure that costs to implement safeguarding measures do not create a competitive advantage for well-resourced providers.

Government action should be durable. For government action to be durable, it also must be flexible. To ensure advances in the ability to screen DNA synthesis orders (e.g., those described in Rec 4) can be realized to their fullest potential, government regulation should not be prescriptive as to define a single acceptable methodology to meet the requirement to screen. Durable and flexible regulation would not discourage innovation of new biosecurity tools and provide a pathway to implementation for technology developers.

The U.S. government should further invest in research and development efforts for screening tools that are more comprehensive than list-based sequence screening, and periodically update screening guidelines to implement them. Federal funding can play a key role in facilitating new screening technologies based on characteristics other than genetic sequence, similar to a protein product's predicted structure or function. These technologies can leverage the same AI/ML advances that are contributing to increasing risks and could be developed through a “violet teaming” process. An understanding of the performance characteristics (e.g., false-positive rate) of such new AI-enabled tools will be critical in defining cost-conscious and practical standards, and such characterization should be completed as part of development. This multilayered screening approach would make it more challenging for a person to acquire DNA with the intent for misuse through mail-order DNA synthesis. The screening standards that are developed in accordance with 4.4 (b)(i) of the Executive Order should allow for continuous assessment and updates, which would promote technological innovation.

The U.S. government should provide funding and resources to develop and implement biosecurity education and training for researchers. Robotics and laboratory automation has resulted in significant increases in the amount of biological data generated, and advances in AI and ML have resulted in increases in what can be done with that data. BDT developers, chatbot developers, and biologists who understand how their research and development outcomes could potentially be misused may make different decisions when designing, conducting, and disseminating research projects than they otherwise may have. For example, after biosecurity education, a researcher developing a BDT to support ssDNA design for Gibson Assembly may decide to include a screening step within the tool itself to prevent the tool's misuse.

Conclusions

Advances in synthetic biology and AI applications raise both novel national security concerns and emphasize underlying previously identified risks inherent to mail-ordered DNA synthesis. To address these threats, we suggest an approach that implements multiple overlapping policies to mitigate the threat landscape from multiple points. Implementation of a required screening framework will provide the foundation for DNA synthesis security and should be enacted immediately.

In conjunction with this process, biosecurity-oriented research and development investments will further safeguard against deliberate or unintentional misuse of synthesized DNA. It is important that these measures be pursued in parallel to balance prompt action with the need to develop more sophisticated regulatory technologies that can further mitigate biosecurity risks and safeguard their beneficial use.

Footnotes

Acknowledgments

The authors acknowledge the helpful discussion with Alexander Titus regarding violet teaming and thank the reviewers for their constructive feedback.

Authors' Disclosure Statement

M.E.W. has a patent pending related to biological design tools from prior employment at the Massachusetts Institute of Technology. The remaining authors have no conflicts to disclose.

Funding Information

This work was conducting as part of an unfunded collaboration among the authors. As such, no funding was received to complete this work.

References

Wong

, de la Fuente-Nunez

, Collins

. Leveraging artificial intelligence in the fight against infectious diseases. Science, 2023; 381(6654):164–170.

Lee

, Yoon

, Kim

, et al. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinform, 2020; 36(4):1234–1240.

Alley

, Khimulya

, Biswas

, et al. Unified rational protein engineering with sequence-based deep representation learning. Nat Methods, 2019; 16(12):1315–1322.

Sandbrink

JB.

Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools [Internet]. arXiv; 2023 [cited 2023 Oct 3]. Available from: http://arxiv.org/abs/2306.13952 [Last accessed: October 3, 2023].

Biosecurity in the Age of AI: Chairperson's Statement [Internet]. Helena Foundation; 2023. Available from: https://www.helenabiosecurity.org [Last accessed: October 3, 2023].

Bran

, Cox

, White

, et al. ChemCrow: Augmenting large-language models with chemistry tools [Internet]. arXiv; 2023 [cited 2023 Sep 6]. Available from: http://arxiv.org/abs/2304.05376 [Last accessed: September 6, 2023].

, Zhou

, Liu

, et al. DNABERT: Pre-trained bidirectional encoder representations from transformers model for DNA-language in genome. Bioinformatics, 2021; 37(15):2112–2120.

, Gupta

, Spaeth

, et al. Machine learning optimization of candidate antibody yields highly diverse sub-nanomolar affinity antibody libraries. Nat Commun, 2023; 14(1):3454.

Bojar

, Powers

, Camacho

, et al. Deep-learning resources for studying glycan-mediated host-microbe interactions. Cell Host Microbe, 2021; 29(1):132–144.e3.

10.

DiEuliis

, Carter

, Gronvall

. Options for synthetic DNA order screening, revisited. mSphere, 2017; 2(4); doi: 10.1128/msphere.00319-17

11.

HHS, Administration for Strategic Preparedness and Response (ASPR). Screening Framework Guidance for Providers and Users of Synthetic Nucleic Acids. Available from: https://aspr.hhs.gov:443/legal/synna/Pages/default.aspx [Last accessed: October 31, 2023].

12.

Biden

JR.

Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. The White House. 2023. Available from: https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ [Last accessed: October 31, 2023].

13.

Carter

, Yassif

, Isaac

Benchtop DNA Synthesis Devices: Capabilities, Biosecurity Implications, and Governance. Nuclear Threat Initiative: Washington, DC; 2023.

14.

Xiao

, Li

, Moon

, et al. Generative Artificial Intelligence GPT-4 Accelerates Knowledge Mining and Machine Learning for Synthetic Biology. ACS Synth Biol [Internet]. 2023 Sep 8 [cited 2023 Sep 12]; doi: 10.1021/acssynbio.3c00310

15.

Mouton

, Lucas

, Guest

. The Operational Risks of AI in Large-Scale Biological Attacks: A Red-Team Approach [Internet]. RAND Corporation; 2023 Oct [cited 2023 Oct 31]. Available from: https://www.rand.org/pubs/research_reports/RRA2977-1.html [Last accessed: October 31, 2023].

16.

Carter

, Friedman

. DNA Synthesis and BIosecurity: Lessons Learned and Options for the Future [Internet]. La Jolla, California: J. Craig Venter Institute; 2015. Available from: https://www.jcvi.org/research/dna-synthesis-and-biosecurity-lessons-learned-and-options-future [Last accessed: October 3, 2023].

17.

Gibson

, Young

, Chuang

, et al. Enzymatic assembly of DNA molecules up to several hundred kilobases. Nat Methods, 2009 May;6(5):343–345.

18.

Jin

, Yang

, Chen

, et al. GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information [Internet]. arXiv; 2023 [cited 2023 Sep 20]. Available from: http://arxiv.org/abs/2304.09667 [Last accessed: September 20, 2023].

19.

Titus

, Russell

. The Promise and Peril of Artificial Intelligence—Violet Teaming Offers a Balanced Path Forward [Internet]. arXiv; 2023 [cited 2023 Sep 13]. Available from: http://arxiv.org/abs/2308.14253 [Last accessed: September 13, 2023].

20.

Zenko

Red Team: How to Succeed by Thinking Like the Enemy. Basic Books: New York, NY; 2015.

21.