Abstract
Antibiotic resistance is compromising our ability to treat bacterial infections. Clinical microbiology laboratories guide appropriate treatment through antimicrobial susceptibility testing (AST) of patient bacterial isolates. However, increasingly, pathogens are developing resistance to a broad range of antimicrobials, requiring AST of alternative agents for which no commercially available testing methods are available. Therefore, there exists a significant AST testing gap in which current methodologies cannot adequately address the need for rapid results in the face of unpredictable susceptibility profiles. To address this gap, we developed a multicomponent, microscopy-based AST (MAST) platform capable of AST determinations after only a 2 h incubation. MAST consists of a solid-phase microwell growth surface in a 384-well plate format, inkjet printing–based application of both antimicrobials and bacteria at any desired concentrations, automated microscopic imaging of bacterial replication, and a deep learning approach for automated image classification and determination of antimicrobial minimal inhibitory concentrations (MICs). In evaluating a susceptible strain set, 95.8% were within ±1 and 99.4% were within ±2, twofold dilutions, respectively, of reference broth microdilution MIC values. Most (98.3%) of the results were in categorical agreement. We conclude that MAST offers promise for rapid, accurate, and flexible AST to help address the antimicrobial testing gap.
Introduction
The rapid emergence of clinical antimicrobial resistance presents a significant challenge for the treatment of bacterial infection. To guide appropriate therapy and preserve the efficacy of existing antimicrobials, hospital-based clinical microbiology laboratories perform antimicrobial susceptibility testing (AST) of bacterial isolates from patients to determine which drugs will be active against an infecting organism. The gold standard methods used for AST require manual preparation of an antimicrobial doubling dilution series in agar or broth that is tested for antimicrobial effect and minimal inhibitory concentration (MIC) determination. MIC values are defined as the lowest concentration of antimicrobial that results in visible growth inhibition of an organism after 16–20 h of incubation. 1 MIC results are interpreted based on consensus guidelines to classify the pathogen as susceptible (S), intermediate (I), or resistant (R) to the tested antimicrobial. 2
However, the technical complexity of manual dilution testing, even for a few isolates or drugs, precludes the use of these reference methods in hospital-based clinical laboratories. 3 Therefore, alternative testing methods are typically used, including various types of preformulated antimicrobial dilution panels, such as Sensititre (Thermo Fisher Scientific, Waltham, MA); methods that extrapolate MICs based on growth kinetics in the presence of a limited number of different antimicrobial concentrations, such as Vitek 2 (bioMérieux, Durham, NC); and diffusion-based methods that can predict MIC-based susceptibility categories for a subset of organism–drug combinations. 3 However, these tests are limited in that they incorporate only predetermined panels of relatively common antimicrobials, do not include recently approved agents developed for multi-drug-resistant pathogens, 4 require a long (>8–20 h) incubation, and for MIC surrogate methods, may not always correlate with reference MIC results, a problem particularly noted for multi-drug-resistant organisms.5–7
Importantly, a delay in the institution of active, appropriate therapy, ultimately guided by AST, is associated with significantly increased morbidity and mortality, especially in patients with bloodstream and other serious infections. 8 With the emergence of multi-drug-resistant pathogens, empiric therapy (treatment without antimicrobial susceptibility data) is now increasingly unreliable. Therefore, there exists a significant AST gap in which current methodologies cannot adequately address the need for rapid results in the face of unpredictable susceptibility profiles. 4
To address this gap, we have combined the use of inkjet-based digital dispensing technology, automated microscopy, and deep learning to develop a prototype microscopy-based AST (MAST) platform. Importantly, the platform permits performance of AST for any antimicrobial at will after a 2 h incubation using commercially available equipment and consumables. Here we describe the development and validation of the MAST prototype.
Materials and Methods
Bacterial Strains and Antimicrobials
Escherichia coli ATCC 25922, Enterobacter cloacae ATCC 13047, Klebsiella pneumoniae ATCC 13883, Pseudomonas aeruginosa ATCC 27853, and Acinetobacter baumannii 17978 were obtained from the American Type Culture Collection (Manassas, VA). Strains were stored at −80 °C in tryptic soy broth (BD Diagnostics, Franklin Lakes, NJ) containing 50% glycerol (Sigma-Aldrich, St. Louis, MO).
Meropenem was from Ark Pharm (Arlington Heights, IL), cefepime was from Chem Impex (Wood Dale, IL), gentamicin was from Alfa Aesar (Haverhill, MA), and ciprofloxacin was from US Biological (Salem, MA). Antibiotic stock solutions were prepared in water for manual dilution-based testing or in water containing 0.3% polysorbate-20 as required for liquid handling by the HP D300 digital dispenser (HP Inc., Palo Alto, CA). 9 We previously determined through extensive analysis that polysorbate-20 at the concentrations used in assay wells has no effect on MIC determinations for all antibiotics examined.5,10
Broth Microdilution Susceptibility Testing
Broth microdilution (BMD) was performed according to Clinical Laboratory and Standards Institute (CLSI) guidelines. 1 Specifically, serial twofold dilutions of antimicrobials were manually prepared in cation-adjusted Mueller–Hinton broth (CAMHB, BD Diagnostics) in sterile, polystyrene 96-well plates (Evergreen Scientific, Los Angeles, CA) in a 50 µL volume. Bacteria were grown overnight at 35 ± 2 °C in ambient air on tryptic soy agar containing 5% sheep’s blood. Several colonies were suspended in sterile 0.9% saline (Sigma-Aldrich) to a density corresponding to 0.5 McFarland as measured using a DensiCHEK Plus Colorimeter (bioMérieux). This suspension was diluted 1:150 in sterile CAMHB, and 50 µL was added to antibiotic-containing wells, resulting in an inoculum density of approximately 5 × 105 colony-forming units (CFU)/mL in a 100 µL assay volume. Inoculated plates were incubated at 35 ± 2 °C in ambient air for 16–20 h. The MIC was defined as the lowest antimicrobial concentration resulting in complete inhibition of growth as determined visually. 1 BMD experiments were performed in triplicate for each antibiotic–organism combination with the modal result recorded as the reference MIC. 5
Preparation of Solid Microwell Surfaces in Assay Plates
In preliminary experiments, we evaluated several agents for solidifying CAMHB, including BD Bacto agar (1.5% w/v, BD Diagnostics), gellan gum (1% w/v, trade name GELRITE, Research Products International, Mt. Prospect, IL), polyacrylamide (15% v/v, Thermo Fisher Scientific), and poloxamer 407 (15% w/v, trade name Pluronic F-127, Sigma-Aldrich). We identified 15% poloxamer 407 as the ideal solidifying agent (described in the “Preparation of Solid Microwell Surfaces in Microtiter Plates” section of Results), and all future experiments were therefore conducted with 15% poloxamer 407 in CAMHB (hereafter called CAMHB-P). Prior to use, CAMHB-P was centrifuged at 4000g at 4 °C for 10 min to remove small particles of media/poloxamer 407 that may interfere with microscopy. The cleared solution was kept on ice, and 10 µL was added to wells of clear polystyrene 384-well plates (Greiner Bio-One, Monroe, NC) using a manual pipette. Immediately after preparation, plates were centrifuged at 3500g at 4 °C for 5 min to ensure complete coverage of wells with CAMHB-P and stored at −80 °C until use.
Digital Dispensing of Bacterial Suspensions
Prior to dispensing, organisms were grown for 16–20 h at 35 ± 2 °C in ambient air on tryptic soy agar containing 5% sheep’s blood. Several colonies were suspended in 0.9% NaCl containing 0.3% polysorbate-20 (NaCl-P20) and adjusted to the desired density using a handheld colorimeter. Suspensions were directly loaded into D4+ or T8+ cassettes (HP Inc.), and dispensed into wells of 384-well plates at volumes indicated using the HP D300 Digital Dispenser (HP Inc.).
Quantification of Bacterial Dispensing Precision
Precision of bacterial inkjet dispensing was quantified by two complementary methods: CFU determination and microscopic cell counting. For CFU quantification, 50, 150, 250, and 350 nL of a 0.5 ± 0.05 McFarland suspension of bacteria in NaCl-P20 were dispensed into wells of a 384-well plate containing 50 µL of CAMHB. The inoculated media was diluted 1:1000 in sterile CAMHB, and 100 µL was plated on Mueller–Hinton plates (Remel, Lenexa, KS). Plates were incubated for 16–20 h at 37 °C in ambient air, and colonies were counted to determine CFU per milliliter according to CLSI guidelines for inoculum density determination. 1 Each experiment was performed in triplicate, with three wells sampled per experiment. Standard curves comparing dispensed volume with bacterial density (CFU/mL) were generated using a custom Python (version 3.5.0) script, Matplotlib (version 2.0.0), 11 and NumPy (version 1.11.3). 12
For quantification by direct imaging, a 0.5 McFarland suspension in NaCl-P20 was diluted 1:5, and 200 nL was dispensed into the center of each of 240 wells of a 384-well plate containing CAMHB-P. The outer two rows and columns of wells were not used to avoid edge effects resulting from evaporative loss. Plates were kept at room temperature prior to imaging at the Harvard Center for Biological Imaging (HCBI, Harvard University, Cambridge, MA) using a Zeiss Cell Observer microscope (Zeiss, Oberkochen, Germany) operating in bright-field mode with a 40× air objective (0.6 NA, 2.9 mm working distance) and an automated mechanical stage. The x and y position of each well was determined automatically using a preloaded plate map. The z position for optimal focus was adjusted manually. A single field corresponding to the center of each well was imaged as a z series of 40 µm with a step size of 2 µm. Image stacks were projected using the extended depth-of-focus module within the Zeiss Zen Blue software. Individual bacteria in resulting images were manually segmented and counted using the particle analysis function in ImageJ. 13 Twelve wells were imaged in each of three independent experiments. A bar graph was generated using a custom Python script (version 3.5.0) and the Matplotlib library. 11
Microscopy-Based AST
Antimicrobials were applied by digital dispensing using the HP D300, as previously described, 10 into 384-well plates (Greiner Bio-One) equilibrated to 35 ± 2 °C containing 10 µL of solid CAMHB-P. Final antimicrobial concentrations in solidified wells ranged from 0.004 to 1 µg/mL for ciprofloxacin and meropenem, 0.016 to 4 µg/mL for cefepime, and 0.03 to 8 µg/mL for gentamicin. Two hundred nanoliters of bacteria in NaCl-P20 suspended at a density of 0.1 McFarland was then delivered to the center of each well by digital dispensing of bacterial suspensions. Immediately after inoculation, plates were incubated at 35 ± 2 °C for 2 h, and held at room temperature for 2 h during transport and imaging.
Plates were imaged with a Zeiss Cell Observer inverted microscope with settings described in the “Quantification of Bacterial Dispensing Precision” section. Following manual selection of the z position for the first well in the series, the microscope stage independently advanced to subsequent wells based on a plate map with the z position updated automatically using the autofocus feature of the microscope control software (Zeiss Zen Blue software versions 2.1 and 2.3). For each well, a z series of 40 µm (step size = 2 µm) was automatically collected and projected to form a single image using the extended depth-of-focus module in the Zeiss Zen Blue software (versions 2.1 and 2.3). Custom Python scripts were used to export individual images as .jpeg files with the appropriate organism name, antibiotic concentration, and well location encoded in the filename.
Image Classification by Deep Learning
Six replicate dilution series for each antibiotic–organism combination were imaged in three independent MAST experiments yielding 3202 training images. Each image was manually classified as “growth” or “inhibition” based on known morphological characteristics of growing or inhibited cells.14–16 Each annotated image had dimensions of 2048 × 2048 pixels (px) and was cropped into 64 nonoverlapping 256 × 256 px images, which were further augmented by random blur and rotation to generate 512 augmented images per annotated image, for a total of 1,639,424 training images. From these data, 80% was used for training a convolutional neural network (ConvNet) and 20% was used for validation. During training, each 256 × 256 px image was additionally augmented “on the fly” by random crop to generate a 220 × 220 px input image for the network. ConvNet architecture followed the Visual Geometry Group (VGG) style 17 with small (3 × 3 px) stacked convolutional kernels with stride and pad equal to 1 px. The network consisted of eight convolutional layers, three fully connected layers, and four max pool layers. The number of feature maps was increased by a factor of two after every spatial pooling layer, resulting in an overall bipyramidal architecture. All convolutional layers were batch normalized, 18 with learned scale and shift, followed by a rectified linear unit (ReLU) nonlinearity. The first two fully connected layers were regularized by dropout, with a dropout probability of 0.5. The final activation was a two-way softmax, corresponding to the categories “growth” and “inhibition.”
Training was done using mini-batch stochastic gradient descent (batch size 32), based on backpropagation with momentum. The loss function was cross-entropy, with additional regularization in the form of L2 weight decay. All networks were trained from random initializations using the Xavier initialization scheme. 19 The initial learning rate was set to 0.0003, and decayed according to an inverse schedule. Momentum was 0.9, and weight decay was 0.0001.
For the purposes of evaluating whole images, all image crops for a given image were evaluated to calculate three parameters: mean image inhibition probability, median image inhibition probability, and proportion of crops with an inhibition probability of >0.5.
Optimization of MIC Calls
To optimize the ability to call MICs based on ConvNet output, we collected a separate dataset, independent of the training and validation datasets, which we refer to as the optimization dataset. We used the MAST assay to collect images from all combinations of antibiotics and organisms listed in “Bacterial Strains and Antimicrobials” on 3 separate days (180 MIC assays × 10 antibiotic concentrations = 1800 images). Images in the optimization dataset were evaluated using our trained ConvNet, and all output parameters were recorded (mean image inhibition probability, median inhibition probability, and proportion of crops with an inhibition probability of >0.5).
For each output parameter, a custom Python script was used to model results from each dilution series as a sigmoidal curve of the form
For each antibiotic–parameter combination, we iteratively set thresholds (ranging between 0 and 0.99) that represented the point on the sigmoidal curve above which all results would be called “inhibited.” We then calculated the MIC accuracy at each threshold and identified the optimal parameter and threshold combination that resulted in the highest MIC accuracy. If the highest accuracy was achieved at multiple potential threshold values for a given parameter, the median of those values was designated as the optimal threshold cutoff value.
Proof-of-Principle Application of MAST Assay for End-to-End MIC Determination
A verification dataset, independent of the training, validation, and optimization datasets, was collected using the MAST assay on 3 separate days using all combinations of antibiotics and organisms listed in “Bacterial Strains and Antimicrobials.” Each of these 1800 images was evaluated using our ConvNet-based pipeline, and MICs for the 180 dilution series were calculated using the optimal parameter and thresholds determined in the “Optimization of MIC Calls” experiments described above. A single dilution series (n = 10 images) was omitted from further analyses based on an autofocus failure that compromised curve fitting and MIC determination.
Log2 differences between MAST and BMD MICs were used to evaluate the precision and accuracy of MAST. Taking into account the inherent variability in BMD, log2 differences of ±1 were considered to be in essential agreement, and thus equivalent as defined by CLSI guidelines. 20 Each MAST result was also assigned a categorical interpretation of susceptible, intermediate, or resistant based on CLSI criteria 2 and compared with the categorical result from BMD to determine categorical agreement.
Results
The rate-limiting step in a traditional AST readout is the threshold for bulk microbial growth detection, by either optical density determination (as in the Vitek 2) or human visualization of bacterial growth (in reference AST methods). Therefore, the fastest phenotypic AST readout presumably should be approached by microscopic visualization of the effects of antimicrobials on the replication of individually resolved bacterial cells. The idea of MAST was born on this premise.
The technical requirements for MAST included the need to visualize microorganisms on a solidified microwell surface, the ability to apply doubling dilution series of any antimicrobials desired, the ability to deliver organisms consistently at desired concentrations, and the acquisition and automated classification of images as “growth” or “inhibition” with subsequent MIC calls. The investigation of each MAST component is discussed in turn, along with preliminary validation of the end-to-end platform concept.
Preparation of Solid Microwell Surfaces in Microtiter Plates
For efficient automated microscopic observation, bacteria must be located in a reproducible 3D location within microwells. In traditional liquid media used for AST testing, cells are distributed randomly throughout the volume of fluid and are highly mobile based on Brownian motion or intrinsic motility, and thus unsuitable for single-cell imaging. Therefore, we considered the use of a solid growth surface containing standard nutrient medium (CAMHB) for image acquisition. The requirements for the solidifying agent included the formation of surfaces in microwells with consistent height, the absence of inhibitory effects on bacterial growth, and amenability to pipetting.
We first evaluated standard microbiological agar. However, the solution was difficult to pipette consistently due to rapid and practically irreversible solidification at <40 °C. Next, we investigated gellan gum, an anionic polysaccharide alternative to agar; 2 polyacrylamide; and poloxamer 407, a hydrophilic, nonionic copolymer, as alternative solidifying agents. Gellan gum supported bacterial growth, but preparation of consistent microwell surfaces was not possible due to phase transition characteristics similar to those of traditional agar. Polyacrylamide surfaces were exceptionally easy to prepare, but proved inhibitory to bacteria.
By contrast, a solution of 15% poloxamer 407 was identified as an ideal solidifying agent. Aqueous solutions of poloxamer 407 are liquid at 4 °C, allowing for facile pipetting of solutions kept on ice. Solidification occurs at ~20 °C and is thermally reversible, allowing for the preparation of plates at room temperature (with poloxamer 407 solution on ice), followed by centrifugation at 4 °C to ensure the substrate is evenly distributed on the bottom of wells. Prepared plates can be frozen indefinitely with no effect on the integrity of growth surfaces. Further, CAMHB solidified with poloxamer 407 was found to support the growth of all common Gram-negative bacterial pathogens tested (data not shown).
Digital Dispensing of Bacteria
The use of solid surfaces contained within single wells of 384-well plates allows for multiple organism–antibiotic combinations to be tested in a single experiment. However, the use of small wells presented a technical challenge in that (1) organisms must be quantitatively delivered without disturbing the integrity of the growth surface, (2) volumes must be small (nanoliter size) to avoid flooding of the surface that would lead to an uneven distribution of organisms and the associated requirement for drying, and (3) delivery must be spatially precise to allow efficient positioning of an automated microscope over fields containing organisms. Manual pipetting or pin transfer cannot satisfy these requirements. Therefore, we investigated bacterial cell dispensing using the HP D300 as an alternative method.
Bacterial cell dispensing is a novel application of inkjet printing technology. Of note, the HP D300 inkjet printer ( Fig. 1A ) used in our studies was designed to dispense droplet volumes ranging from 11 pL to 10 µL per manufacturer’s specification 9 from a single stock solution loaded into a reagent cassette ( Fig. 1B ). We previously used this technology to prepare doubling dilution series of antimicrobials in 384-well plates in liquid media, 10 and in MAST, we used the same technique to apply antimicrobial dilutions to solid surfaces ( Fig. 1C,D ). We furthermore hypothesized that we could also use this technology to deliver any desired quantity of microorganism into microwells from a single bacterial suspension by choosing the appropriate droplet dispense size ( Fig. 1E ).
MAST assay. (
Therefore, here we evaluated the ability of the HP D300 to dispense bacteria using two complementary techniques.
First, we evaluated the ability of the HP D300 to quantitatively deliver bacteria to microwells by measuring the number of cells dispensed in a given volume through CFU determination. We performed this evaluation with five organisms representative of the most common Gram-negative pathogens (E. coli, K. pneumoniae, E. cloacae, P. aeruginosa, and A. baumannii) to discern whether unique species-specific physical properties might impact the ability of cells to travel through the cassette channel, thus affecting dispense accuracy.
In linear dynamic range studies, we dispensed between 50 and 350 nL of a bacterial suspension and quantified the number of cells delivered to a well. We constructed a standard curve and found that the dispense volume was predictive of CFU (average R2 = 0.96), indicating precise and reproducible cell dispensing ( Fig. 2 ). Experiments were repeated on 3 separate days, and the day-to-day coefficient of variation was, on average, 34%. For each dispense volume, precision was evaluated for three wells spanning a series of 96 wells. No significant difference in bacterial dispense numbers was detected across wells (ANOVA, p > 0.05).
Digital dispensing of bacteria suspensions. Standardized suspensions of bacteria were dispensed into liquid media using the HP D300. (
Second, after considering quantitative precision of dispensing, we evaluated spatial precision in a manner relevant to MAST. We dispensed bacteria into the center of wells and counted the number of digitally dispensed bacteria in a single central field using the Cell Observer microscope. This methodology allowed us to simultaneously evaluate spatial precision, quantitative precision, and reproducibility of digital dispensing. We found that 100% of wells imaged contained bacteria in the central field, and that cells were well separated and evenly dispersed (see Fig. 3A,B for representative images). The average number of bacteria per field in the single selected central field across wells ranged, on average, from 150 to 260 (1.5–2.5 cells per 1000 µm2). The day-to-day coefficient of variation was 23% across all species tested ( Fig. 3C ). There was no significant difference in bacterial dispense numbers across all wells (ANOVA, p > 0.05).
Representative cell densities immediately after digital dispensing. Standardized suspensions of (
During these studies, we found that bacteria could be clearly resolved in images using a long-working-distance 40× objective mounted on an inverted microscope, with bright-field optics, and standard polystyrene 384-well plates. Notably, use of optical-quality 384-well plates with thin plastic film bottoms did not visibly improve image resolution of bacteria growing on CAMHB-P surfaces. Further, phase-contrast and differential interference contrast (DIC) imaging of wells surprisingly led to inferior bacterial image resolution, perhaps because of optical properties and the thickness of the plastic well bottom and poloxamer separating the objective and bacterial growth surface (total working distance of approximately 2.9 mm).
Microscopy-Based AST
We then combined bacterial and antibiotic dispensing with automated microscopy to perform microscale, “agar dilution” AST assays inside wells of a 384-well plate. Here, bacterial growth was monitored by automated microscopy ( Fig. 1F ) and images were classified as “growth” or “inhibition” using a machine learning algorithm to determine the MIC (Fig. 1G). For proof-of-concept studies, we chose to use four different antibiotics with clinical utility against Gram-negative bacteria. Each of these drugs was also specifically chosen based on unique and well-defined alterations in cell morphology at their MIC, all of which were observed using the MAST assay.14–16 Exposure to ciprofloxacin, a fluoroquinolone class, DNA gyrase inhibitor, resulted in elongated cells that failed to divide ( Fig. 4A ). Exposure to cefepime, a cephalosporin β-lactam-based cell wall synthesis inhibitor, resulted in filamented bacteria with a pronounced central bulge ( Fig. 4B ). Gentamicin, an aminoglycoside class protein synthesis inhibitor, blocked cell growth, but did not appreciably alter morphology ( Fig. 4C ). Exposure to meropenem, a carbapenem β-lactam-based cell wall synthesis inhibitor, resulted in rounded spheroplasts ( Fig. 4D ). We presumed that this range of cell morphologies would appropriately challenge the automated image classification methods discussed in subsequent sections.
Representative morphologies of inhibited E. coli ATCC 25922. Antibiotics and E. coli ATCC25922 were dispensed into microwells and automatically imaged with a Zeiss Cell Observer microscope after <4 h incubation. Panels represent the central field of a microwell containing (
Image Classification by Deep Learning
To call MICs, at least one image field must be analyzed per microwell in a doubling dilution series. When testing multiple drugs against multiple organisms with a typical dilution series for each drug, including 10 doubling dilutions, manual inspection and classification of the resultant large image sets would be time-prohibitive and inconsistent with future clinical translation. Therefore, we considered using machine learning for automated classification of MAST images.
Specifically, we examined the ability of a deep ConvNet to classify images and accurately determine MIC values following only a 2 h incubation period. Importantly, this accelerated time window contrasts with the 16–20 h incubation period required for reference MIC determinations and similar extended incubation periods required for commercial AST methods. The ConvNet programming structure is inspired by the organization of neurons in the visual cortex, and these models have recently achieved human-level performance on large-scale image classification tasks.21,22 This approach takes a raw image as input and learns to extract relevant features for the particular task at hand. In other words, it does not rely on preconceived human biases regarding what features are likely important for optimal discrimination, but rather simultaneously learns which features to extract and how to combine them to achieve accurate classification, based on optimizing its output relative to a labeled training set.
Our network was trained and validated on a set of 3202 full images (2048 x 2048 px) collected in three independent experiments in an attempt to capture the totality of biological and technical variability in the MAST assay and produce a model that will generalize to a diverse set of conditions. After training, the network reached a peak classification accuracy of 90% on the held-out validation set at the per-image-crop (220 × 220 px) level. After mapping image crops back to whole images, regions representing growth or inhibition were manually inspected and found to correspond to expected growth/no growth morphologies (data not shown).
Optimization of MIC Calls
To determine MICs, each image, which corresponds to a specific antimicrobial dilution, must be classified as growth or inhibition. Our image classification algorithm first evaluated 64 nonoverlapping crops from each image and determined the mean inhibition probability, median inhibition probability, and number of crops with an inhibition probability of >0.5. However, it was unknown which of these parameters and what threshold cutoff for each parameter would result in the most accurate MIC calls. Furthermore, we noted that absolute values of these parameters varied by approximately 20% on a day-to-day basis, suggesting the need for an adaptive algorithm that would be robust to these differences. Specifically, the effects of biological and technical variability were mitigated by modeling the results for each parameter (median inhibition probability, mean inhibition probability, and number of crops with an inhibition probability of >0.5) across each dilution series as a sigmoidal curve.
We collected an optimization dataset for the purpose of determining the best way to evaluate the ConvNet output. We then iteratively evaluated all possible discrimination thresholds normalized to each sigmoid curve for each antibiotic/organism/parameter for correct MIC prediction. Although the ConvNet was trained on images classified based on the morphological characteristics of cells, here we evaluated and optimized performance against reference BMD MIC values determined for each dilution series.
Upon optimization of the discriminatory threshold for each parameter for all antibiotic–organism combinations, we identified the proportion of image crops with an inhibition probability of >0.5 as the optimal parameter for MIC determination. Importantly, we confirmed our prediction that a sigmoid curve would robustly model our data (average R2 = 0.93).
Proof of Principle, End-to-End Testing of MAST Assay
To evaluate the “end-to-end” MAST assay, we collected a large verification dataset that was independent of the training, validation, and optimization datasets. We then compared the performance of our optimized MAST assay (the test method), which included only a 2 h growth incubation, with that of BMD (the reference method), which used a standard 16–20 h growth incubation. Sigmoidal curve fitting was equally robust in this phase across all antimicrobial–organism combinations (average R2 = 0.91, Fig. 5 ).
Graphical representation of MAST MIC assay output for E. coli ATCC 25922. (
Notably, MAST MIC essential agreement (±1 twofold dilution) was 95.8% compared with BMD. Most of the MIC results (99.4%) were within ±2 twofold dilutions of modal reference BMD results ( Table 1 ). MIC determinations for each antibiotic considered separately were similarly accurate and ranged from 93.3% to 100%. Notably, a 100% MIC accuracy was found for cefepime despite this antibiotic presenting a substantial challenge to the classification algorithm due to filamentation of cells near the MIC ( Fig. 4B ). Furthermore, categorical agreement (interpretation of organisms as susceptible, intermediate, or resistant) for the susceptible strain set between MAST and BMD was 98.3% ( Table 1 ), indicating a very low level of false resistance (major errors).
Evaluation of MAST Accuracy.
Reference MIC derived from BMD.
CI = 95% confidence interval.
Discussion
Extensive antimicrobial resistance among Gram-negative pathogens reduces the reliability of empiric antimicrobial therapy, underscoring the critical importance of timely AST in the clinical microbiology laboratory to guide therapy. Importantly, delay in institution of active therapy is correlated with increased mortality during serious infection. 8 However, laboratory tests in current use require 8–24 h for results.
Rapid AST systems are a potential solution to address this issue and may be approached broadly through two pathways: phenotypic testing or genotypic testing. Genotypic assays call resistance based on the presence of specific resistance elements. However, such assays are limited to evaluating known resistance determinants, and thus are insensitive to novel mechanisms. Further, they are typically unable to determine exact MICs, and thereby direct therapy based on known pharmacodynamic relationships. Therefore, genotypic methods lack sensitivity and specificity.
In contrast, rapid phenotypic assays have the capability of determining exact MICs. Furthermore, well-established and validated pharmacodynamic relationships between drug exposure in vivo and the MIC are predictive of response to therapy. 23 Accordingly, CLSI guidelines recognize phenotypic testing as the current methodology for determining susceptibility. 2 For Gram-negative organisms in particular, guidelines have moved away from detecting or inferring the presence of specific resistance elements to guide therapy. 24
Here, we developed and performed preliminary validation of a rapid phenotypic AST method based on observation of individual cells. Microscopic AST methods present logistical and technical challenges, one of which is the need to immobilize cells for observation. In most existing rapid AST instruments, this is accomplished by suspension of cells in a matrix or restriction of cells to a microfluidic channel. 25 These procedures are complex and have no predicate in current clinical microbiology practice.
In contrast, the MAST assay immobilizes bacteria by dispensing cells on a solid microwell surface analogous to 1000-fold miniaturized agar plates and, in essence, replicates the existing agar dilution reference method. 1 Using inkjet technology, inoculation of an entire assay plate (up to 240 wells) can be accomplished quickly (<1 min) in an operator-independent manner with a single pipetting step. The steps required for MAST assay performance (including preparation of bacterial suspensions) are similar to those for currently used AST systems and could be performed by a technologist without additional training.
Clinical automated AST systems typically include only a fixed and limited set of antimicrobials; furthermore, only a limited number of antimicrobial dilutions are tested per drug. 3 In contrast, MAST allows preparation of plates dynamically using any number of antimicrobials with dilution series of any size in a high-density 384-well format. Similar to bacterial dispensing, an antimicrobial doubling dilution series can be prepared very quickly (<10 s) with a single pipetting step to load antimicrobial stock solution into a dispensing cassette, as we described previously. 10 Flexibility to prepare any doubling dilution series at will allows for testing antimicrobials at concentrations relevant to multiple species that may have different breakpoints on a single microplate. 2 Concentrations can easily be tailored to include changes in breakpoints that are recommended by agencies such as CLSI to accommodate the evolving understanding of the relationships between MIC and clinical outcome. 4 The ability to test antimicrobials at low concentrations ensures that standard quality control organism MIC ranges are on scale so that assay performance can be appropriately monitored, something that is often not possible with current clinical systems, which only include dilution series bracketing clinical breakpoints.2,5
The large amount of image data generated by MAST led to the need for automated interpretation capabilities. Therefore, we developed a machine learning–based image classification algorithm. In this first-stage conception of MAST, a ConvNet was trained based on human classification of a large set of training images collected after 2 h of incubation. Importantly, images were collected from five organisms exposed to varying concentrations of four antibiotics over 3 separate days, such that the training set captured the biological and technical variability that might be seen in a clinical setting.
Ultimately, our image analysis pipeline contained three levels of image interpretation: (1) ConvNet image classification, which returned the per-image-crop probability of inhibition, a feature that in itself predicted the growth status in image crops with >90% accuracy. However, this output did not make a direct prediction regarding whole images. (2) Results from all image crops for each image were therefore pooled to provide image statistics (mean inhibition probability, median inhibition probability, and proportion of image crops with an inhibition of >0.5). (3) Data from all images in a dilution series were then modeled as a sigmoid curve and classified based on thresholds that optimized the accuracy of MIC calls.
This optimized algorithm was then applied to a new set of images and the accuracy compared with the BMD MIC. Importantly, the accuracy met essential agreement metrics established by the Food and Drug Administration (FDA) 26 and standards in the field for evaluating new AST methodology in comparison with the BMD reference methodology. 27 Specifically, the essential agreement was 95.8% (±1 dilution from the reference MIC), exceeding the recommended 95% essential agreement threshold.
In this preliminary analysis, a susceptible strain set was tested. Even so, it is notable that we found only three interpretive errors. All these interpretive errors were found with cefepime and P. aeruginosa ATCC 27853, which has a reference MIC only one doubling dilution below the categorical breakpoint. 2 Moreover, the method appeared to perform equally well testing E. coli, E. cloacae, K. pneumoniae, P. aeruginosa, and A. baumannii, constituting the overwhelming majority of Gram-negative pathogens causing human infection. These organisms are also often associated with problematic drug resistance and therefore are of high relevance for accelerated AST testing. 28 Importantly, although our image analysis pipeline is not optimized for processing speed, a single 10-step doubling dilution series can still be evaluated in as little as 70 s through use of a dedicated graphics processing unit (GPU), allowing rapid image classification. Automated MIC calls based on sigmoidal modeling of ConvNet output are essentially instantaneous.
In a first-generation MAST platform, we were able to achieve a high degree of accuracy after only 2 h of incubation. Nevertheless, classification performance could potentially be improved even further through several measures: (1) Further training targeted to individual species or antimicrobials. Specifically, we suspect that the several different morphologies associated with both species and antimicrobial effects may have marginally compromised accuracy. In the future, species-specific models could be trained, as many clinical laboratories now have systems such as matrix-assisted laser desorption/ionization time of flight (MALDI-TOF), which allow speciation of organisms within minutes. (2) Shifts up or down one doubling dilution of MAST modal MICs compared with BMD may relate to inherent differences observed in other contexts between solid and broth assays (i.e., reference agar dilution vs. BMD tests). 1 (3) Use of a 2 h incubation was a high bar, and it is likely that even without further modification MAST could achieve higher essential agreement if incubation periods were marginally longer. (4) Lastly, our human classification of the training dataset was labor-intensive and may not have been consistently accurate over the large numbers of images examined. Therefore, the underlying ConvNet accuracy could likely be improved through training based on modal reference MIC for each isolate–antimicrobial combination rather than visual classification of each individual image. This would reduce time and potential error inherent to human subjective interpretation and allow for use of much larger training datasets
Notably, typical rapid AST systems require custom-made consumables. 25 In contrast, the MAST assay relies entirely on off-the-shelf consumables, including standard polystyrene 384-well plates, poloxamer 407, and antibiotic powder. When fully utilized, a MAST assay plate can accommodate testing of four pathogens against six antibiotics each, resulting in a consumable cost of <$3.50 per antibiotic–organism combination (inclusive of the D300 cassette, microtiter plate, poloxamer 407, and antibiotic) based on list price. Furthermore, all required instrumentation is commercially available. Importantly, MAST utilizes only a subset of features of the HP D300 and Zeiss Cell Observer microscope. Indeed for the latter, the only features used were standard light microscopy optics, a long-working-distance 40× lens, and a mechanical stage. Therefore, instrumentation could conceivably be simplified to further reduce costs during the development of a next-generation platform contained within a single instrument.
Here, we demonstrate proof of principle for a flexible AST platform that allows highly accurate MIC determination after only a 2 h incubation. Importantly, the system allows determination of AST for any concentration of any antimicrobial at will in an operator-independent manner. This flexibility extends to incorporation of new antimicrobials that may not be available in commercial panels for years yet will likely be required to treat clinically emerging multi-drug-resistant pathogens. Further, MAST is a true MIC methodology that allows all relevant concentrations of antimicrobials to be tested without the need for MIC extrapolations that may not work reliably for resistant pathogens. 29 Taken together, proof of principle was obtained for the MAST platform that will address a critical unmet need for flexible, extremely rapid AST diagnostics.
Footnotes
Acknowledgements
We thank the Harvard Center for Biological Imaging for infrastructure (Cell Observer microscope) and technical support. J.E.K. was provided with an HP D300 digital dispenser and associated consumables for use in his research laboratory, and received an honoraria and travel support from Tecan Inc. (Morrisville, NC). The Kirby laboratory also received HP D300 consumables from HP Inc. Tecan and HP had no role in the study design, data collection/interpretation, manuscript preparation, or decision to publish.
Authors’ Note
The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic healthcare centers, or the National Institutes of Health.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was conducted with support from Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Research Resources and National Center for Advancing Translational Sciences, National Institutes of Health Award UL1 TR001102) and financial contributions from Harvard University and its affiliated academic healthcare centers. Thea Brennan-Krohn was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health pediatric infectious diseases research training grant T32HD055148.