Process Evaluation of an Open Architecture Real-Time Molecular Laboratory Platform

Abstract

The needs of molecular diagnostic laboratories that perform both Food and Drug Administration–cleared as well as laboratory-developed tests are usually not met on a single analytical platform. Furthermore, little information is available about the direct impact of molecular automation on labor costs and efficiency in clinical laboratories. We performed a process impact analysis from time and motion studies of a novel molecular diagnostic robotic system designed to automate sample preparation, extraction, and analysis. All 27 preanalytical tasks were quantified for the amount of time spent preparing 24 specimens for analysis. These steps were completed in 899 s (14 min, 59 s) followed by 7887 s (131 min, 27 s) of instrument operation independent of operator control (walk-away time). Postanalytical results evaluation required 1 min per specimen. The instrument automatically extracted the nucleic acid from the specimen, added the eluted DNA to the amplification reagents, and performed the analysis. Only 12% of the total instrument operations required relatively unskilled human labor. Thus, the availability of automated molecular diagnostic instruments will facilitate the expansion of molecular testing in the clinical laboratory because they reduce operator costs with respect to time and complexity of the tasks they are asked to perform.

Introduction

The rapid proliferation of molecular testing in clinical, research, and industrial settings has been limited by the relative complexity of performing the extraction and amplification of nucleic acids. Advanced training and experience are currently required to ensure proper performance and interpretation of molecular tests. Despite the complexity of molecular tests, the steps involved in specimen processing and analysis have gradually been reengineered to enable them to be automated using programmable robotic liquid handlers. For example, nucleic extraction using adsorption to the wall of a glass test tube has been improved by using small magnetic particles that can bind and release nucleic acids. The magnetic particles can then be aggregated using a magnet and washed to remove interfering substances, and the attached nucleic acid may then be eluted using specific buffers. Automation can be designed and programmed to complete these procedures because all of the extraction activities can be performed by laboratory robotics aided by the use of magnetic capture. Automation reduces hands-on time, increases quality and throughput of results, and enables less specialized labor to perform complex analytical methods. From a process-oriented perspective, there are discrete steps involved with the analysis of nucleic acids, namely, preanalytical, DNA/RNA extraction, amplification, detection, and data reduction. To determine the potential benefits of automating molecular testing, these steps must be quantified for processing time and difficulty.

Analytical performance is the most common evaluation performed for new automated diagnostic devices in order to obtain Food and Drug Administration (FDA) clearance and to assist laboratories in determining the optimal performance that may be achieved for a given instrument price. ¹ However, there is a growing need to evaluate diagnostic instruments to determine how automation improves laboratory process efficiency and lowers overall costs. A previous study examined the process efficiency of a fully automated high-throughput robotic molecular diagnostics platform (BD Viper System with XTR Technology; BD Diagnostics, Sparks, MD). ² The BD Viper XTR System demonstrated that only 35 min of total preanalytical and postanalytical hands-on operator time was required out of the total 206 min required to complete one analytical run that generated 184 analytical results. ² When additional instruments were added in a simulation model, a significant increase in productivity was calculated, demonstrating that a single laboratory scientist could generate almost 3000 results per day. Despite the benefits of a high-throughput robotic system, many laboratories do not require this level of throughput. The needs of smaller laboratories center on using both FDA-cleared in vitro diagnostic molecular assays and those designed and operated in house to enable rapid response to unmet needs and emerging threats. Thus, a standardized platform that accommodates both vendor-supplied in vitro diagnostic reagents and laboratory-developed assays would serve to optimize overall lab efficiency. The evaluation of the process efficiency provided by a fully automated molecular testing platform that has both open and vendor-certified reagents has not been published, because until recently, no such open systems existed. Thus, we decided to evaluate this open system to determine the impact of a versatile molecular laboratory instrument on labor requirements and processing time on laboratory workflow. ³

To determine the degree of process efficiency afforded by the fully automated molecular diagnostics platform, we measured the specific tasks, complexity of the tasks, and the time required to operate a next-generation molecular instrument. Tasks were quantified as either complex, moderately complex, or routine. Temporal activity was also defined as hands-on time (total time involved in direct manual labor), hands-off time (time required when waiting for critical events to complete), and walk-away time (time available for completing other tasks in the laboratory while the instrument performs its automated operations).

Materials and Methods

A commercially available instrument that performs FDA-cleared and laboratory-designed assays was identified from currently available models. The BD MAX System (BD Diagnostics) was tested because it was the one model that provided full process automation from sample tube to analytical result as well as allowed user-defined methods to be implemented. The hardware consisted of a four-position syringe pump–based pipette articulated on a programmable Cartesian (X, Y, Z linear axis) robot that serviced lab-on-a-chip–type reaction chambers. The robot was capable of acquiring pipette tips as necessary and disposing of the tips after the pipetting steps were completed. Specimens, reagents, and test tubes were arranged in a unitized reagent strip (URS; Fig. 1 ) by the user according to a prescribed protocol. The BD MAX System is designed to accept a bar-coded specimen container (called the sample buffer tube [SBT]) able to accommodate multiple specimen types, including plasma, serum, urine, cerebral spinal fluid, stool, direct swabs, and transport media. Each tube was scanned to read the bar codes as they were placed into a transport and mixing rack. A work list was automatically generated by the computer as the specimen bar codes were acquired. The capped tubes contained in the transport rack were then placed into a mixing instrument that allowed the entire group of specimens to be mixed simultaneously by rotary vortex action (VWR Multi-Tube Vortex, VWR Cat no. 58816-115; VWR, Radnor, PA). Reagents were placed in the Unitized Reagent Strip (URS) specifically engineered for the BD MAX System that prevents reagent mix-up and allows positive bar-code reading of reagents and specimens prior to the analytical run. Once mixed, the entire rack of specimens and the transport rack were loaded into the BD MAX System. No operator time is required to orient the tubes containing the two-dimensional (2D) bar codes that are printed in multiple orientations for access by the reader. For the purposes of evaluating the efficiency of the instrument, FDA-cleared protocols with commercially available reagents were employed. We selected the FDA-cleared methicillin-resistant Staphylococcus aureus (MRSA) assay (BD Diagnostics) created for the instrument by the manufacturer.

Figure 1.

Unitized Reagent Strips (URS). The URS consists of a rack that has experimental vials molded into the rack embodiment. Pipette tips are inserted into the larger vials. Reagent vials can be inserted into the rack, providing an organized assembly of reagents for specimen DNA or RNA extraction as well as PCR master mix (polymerase, nucleotides, cofactors, and buffer components). Processing tubes accessible by the robot are also present in the URS. To avoid placing tubes in the wrong location, the URS is also color coded.

Results

The manufacturer-supplied reagents contain an internal control, thus enabling all 24 positions to be used to generate patient results. If the instrument is operated using user-supplied reagents, the BD MAX System has five channels that can be used for detection, thereby allowing up to 120 results to be generated per run.

Normally, the BD MAX System is maintained with the power-on state; otherwise, a 1 min initialization protocol prepares the instrument for operation. The subsequent preanalytical processes include scanning the reagent bar code so that the reagent information can be automatically entered into the appropriate field in the computer containing reagent information. The first step in the hands-on time involves putting specimens in the SBT ( Table 1 ; Fig. 2 ). The SBT contains an appropriate amount of specimen buffer to properly hydrate and adjust the pH of each specimen. The SBT is labeled with two bar codes; the linear code is used for scanning in the work list as each tube is loaded into the specimen container rack. A 2D code is also used by the instrument to automatically scan all the tubes using a mirror that enables the codes to reach the scanning camera. This step is critical for maintaining the proper order of each specimen in the work list, which was generated by using the handheld bar-code reader and scanning each tube in turn. From this point on, as long as the tubes remained in the rack, there was no opportunity to get them out of order. The entire rack of specimens was mixed simultaneously on the multitube vortex while in the specimen rack.

Table 1.

Eight Steps Involved in Performing a Complete Automated PCR Analysis of Methicillin-Resistant Staphylococcus aureus. ^a

Step	Individual Process Steps	Seconds/Specimen	Seconds/24 Specimens	Minute/24 Specimens	% Total Activity	Activity Class
1	Preanalytical	18	432	7.2	3.9	Hands-on
2	Vortex mixing	60	60	1	0.54	Hands-free
3	Computer setup	30	30	0.5	0.27	Hands-on
4	Assemble reagents	6	144	2.4	1.3	Hands-on
5	Accession specimens	12	173	2.88	1.56	Hands-on
6	Load instrument	15	60	1	0.55	Hands-on
7	Instrument performs	140	7887	131.45	91.1	Hands-free
8	Postanalytical	1	24	0.4	0.22	Hands-on
Total		342	8870	147.83	100

a

The timing studies for each step were performed for one specimen (column 3) or for a full run of 24 specimens (columns 4 and 5). The total time for the analysis is displayed in the bottom row. The fraction of time required for each step is displayed as a percentage of the total time (column 6). Whether the step requires operator hands-on performance (hands-on) or the instrument performs the step automatically (hands-off) is shown in column 7. The step number (column 1) corresponds to the steps displayed diagrammatically in Figures 2 and 3 .

Figure 2.

Process flow diagram. The linear process flow is depicted with each discrete step in the preanalytical, automated, or postanalytical processes listed in sequentially arranged boxes. The numbers listed in the preanalytical process flow diagram correspond to the graphical representation ( Figures 3 and 4 ) as well as in the tabular description ( Table 1 ).

The system has the provision for using multiple lots of reagents in the same run, thus obviating reagent waste during lot changes. The instrument is also capable of performing DNA, RNA, and total nucleic acid amplification and/or extraction in the same run but segregated into separate racks. During the run, each specimen is mixed with reagents obtained from its URS ( Fig. 1 ). Color-coded bar-code labels are used to ensure the avoidance of placing reagents in the wrong location. An automated liquid-handling pump system mixed each specimen with magnetic beads, which bind nucleic acids and allow for purified product to be automatically eluted after washing away potentially inhibitory substances. The final assembled specimen, master mix, primers, probes, and buffer were then mixed and transferred to microfluidic PCR cartridges for thermal amplification and qualitative or quantitative measurement using a built-in fluorometer. Each microfluidic-based amplification cartridge may be used for up to 24 individual amplifications. Amplicons are sealed into each PCR cartridge using a wax valve to prevent escape of amplified nucleic acids. This obviates the significant amount of labor required to determine the source of specimen contamination and excessive cleaning necessary to prevent future laboratory contamination outbreaks.

The BD MAX System is equipped with five fluorescent channels to support a wide range of fluorescent molecules for assays such as TaqMan, Scorpion/Molecular Beacon, MGB, and SYBR Green using the following excitation and emission wavelengths (excitation 464–490, 520–540, 555–598, 622–636, 673–683; emission 510–530, 559–571, 618–638, 657–670, 710–790).

The manual and automated steps involved with operating the BD MAX System were evaluated for labor time commitment and walk-away time in which the operator is free to perform other tasks in the laboratory ( Fig. 2 ; Table 1 ). The steps involved with preparing the specimens, placing them in the specimen racks, reading the bar codes, and then preparing the instrument for operation are quite similar when running the instrument in its various modes (described in detail below). It was determined that all the steps required to run the instrument were considered to be routine in complexity.

The time to complete each task was measured by observation and recording the time from a standard stopwatch ( Table 1 ; Fig. 3 ). All preanalytical tasks for 24 specimens were completed in 899 s (~15 min), and the walk-away time during which the instrument performs the extraction and quantifying of the nucleic acid products is a variable that is dependent on the specific needs of each assay. For example, FDA-cleared assay times can have relatively wide assay ranges, for example, 1 h, 53 min (113 min) for 24 group B Strep (also available from BD) specimens and 2 h, 6 min (126 min) for 24 MRSA specimens, which were for the assay tested in our studies. Once the instrument is initialized, if multiple runs are planned, the BD MAX System allows the user to open the lid 45 to 81 min later depending on the number of specimens being processed and the assays on the instrument. In our studies in which we used the FDA-cleared MRSA assay, the lid could be opened and a new run initiated 81 min, 39 s after the initiation of the previous run. Using a “nested assay” approach to operating this robotic instrument allows an operator to load reagents 81 min, 39 s after the commencement of a run ( Fig. 4 ). Because the preanalytical activities (steps 1–6 in Table 1 ) require 15 min for a full run, optimal efficiency may be achieved by performing these steps while the instrument is performing the specimen processing and extraction protocols. During an 8 h day, five runs may be performed while still leaving four 74 min, 39 s periods and one 131 min, 27 s period free so that other laboratory tasks may be performed ( Fig. 4 ).

Figure 3.

Elapsed time line plot. The elapsed time and individual time blocks for each step in the manual, automated, and postautomation processes are displayed for one analytical run. The numbers appearing in each block correspond to the steps listed in Figure 2 and Table 1 . The total elapsed time is shown at the top of the plot. Each block width is proportional to the amount of time required to complete each step. The preautomation steps have also been displayed as an expanded timeline below the main plot for clarity.

Figure 4.

Eight hour elapsed time plot with superimposed runs. The system process throughput was calculated for an 8 h day, allowing for a total productivity of 120 specimens. Individual runs are shown overlapping in the diagram because specimen preparation and loading the analyzer may be performed prior to completion of the previous run, allowing for increased productivity.

Instrument cleanup consisted of wiping the instrument work surfaces to remove potentially contaminating nucleic acids. This wipe down was followed by a deionized water wipe down followed by 70% alcohol, and finally the surfaces were dried with lint-free cloths. Sample racks were also be cleaned between each run. To optimize throughput, having extra racks would obviate the need to stop to clean the racks between runs.

Because the BD Max may be operated in various modes, it can provide efficient workstation functionality to allow the operator to optimize an assay developed in house. For example, the BD MAX System may be used to extract and purify nucleic acids for other downstream operations. Previously purified nucleic acids may be amplified by PCR or amplified and then detected by the fluorometer in qualitative endpoint or quantitative real-time mode. Operator labor involved with each of these operations would be similar in that the same steps would be involved with sample introduction into the specimen tube whether it is a raw specimen or previously extracted DNA or RNA. Because the instrument may be used to analyze between 1 and 24 specimens per run, the labor time required for the operator ranges from 3.33 min (140 s) to 14 min, 59 s (899 s; Fig. 3 ). Similarly, postanalytical labor time was found to range from 1 to 24 min. The automation time varies from 1 h, 30 min to 2 h, 48 min depending on the FDA-cleared test selected from the available menu.

Discussion

Recent advancements in automated sample preparation have reduced the number of preanalytical steps necessary for the extraction and cleanup of the DNA or RNA from whole blood, swabs, or other input sources. Standalone extraction workstations are available that accept primary specimens and yield a high-purity DNA/RNA product after cleanup steps that use magnetic capture particles. ⁴ Analytical workstations are also available that will amplify and quantify purified DNA or RNA targets. However, complete systems that provide extraction, amplification, and analysis are offered only in closed systems that do not allow each laboratory to add their own reagents and design their own protocols. ⁵ Thus, an open system was selected for this process evaluation because it allowed operators to design and run their own tests while also using FDA-cleared molecular diagnostic reagents.

Previously, the BD MAX System was evaluated for its clinical utility in measuring Clostridium difficile from infected individuals. ³ BD MAX analytical performance was confirmed with culture followed by matrix-assisted laser desorption/ionization and toxin analysis demonstrating comparable sensitivity and specificity. However, during these studies, the BD MAX System was not evaluated for its impact on laboratory process efficiency. Previously, a process evaluation was published on a fully automated molecular diagnostics platform (BD Viper XTR System) that demonstrated that the BD Viper XTR System required 35 min of employee burden during a total analytical time of 196 min (3 h, 16 min) to analyze 92 specimens and 4 controls, generating 184 analytical results for Chlamydia trachomatis and Neisseria gonorrheae. ² That process evaluation demonstrated that 18% of the analytical time demanded operator input, resulting in an average of 315 analytical results per employee labor hour. However, the BD Viper System does not allow laboratory-designed assays to be developed and run on the instrument.

The BD MAX System examined during this evaluation had a process efficiency requiring human labor to be involved in only 9% of the total analytical time. The elements that contributed to minimal hands-on time included preanalytical bar-code scanning and work list linking, fixed geometry of reagent and specimen racks, two 12 position analysis regions, a pipetting robot with four stepper motor–driven pipettes, automated specimen extraction, microfluidic cartridges for sample heating in a real-time PCR heating block with fluorescent optics, and automated data reduction capabilities. The integration of these various components into a fully automated robotic system not only reduced the hands-on time but also reduced the analytical complexity. The system design also reduced the chance of operator errors that might occur during analytical preparation. For example, the bar codes on the specimens are automatically read by the instrument to determine if they match the generated work list. In addition, reagents are bar coded and color coded to ensure that they are placed in the appropriate location. Complex tasks required to produce a new molecular test would require highly skilled operators who could adjust the assay chemistry, volumes, thermocycling temperatures, and duration in order to obtain a robust assay. Once an optimal assay is developed, future testing can be easily managed by a routine operator.

The BD MAX System has also optimized the reagent formulation to enable a wide variety of specimens to be used by the system. For example, the ability to introduce various medical specimens including plasma, serum, urine, cerebral spinal fluid, stool, direct swabs, and transport media for automated nucleic acid extraction provides great flexibility in sample selection. Using the open format mode will require a trained molecular biologist to optimize the reagent formulations as well as PCR conditions, but once this is accomplished, the instrument may be operated by a less skilled technologist. Furthermore, the operator may use his or her own PCR reagents or use preoptimized reagents that have been dried into reaction mix tubes and made available from the manufacturer. The BD MAX System allows users to define various protocols using the operator-controllable software in order to arrive at optimal self-designed assays.

In conclusion, using molecular diagnostics automation, the total analytical time for 24 specimens varies between 1 h, 53 min (113 min) and 2 h, 8 min (128 min) for currently available FDA-cleared assays. However, operator time is approximately 15 min irrespective of the amplification and analysis time. Furthermore, all operator tasks were scored as “routine” because the robot performed all of the operator-sensitive steps. Therefore, this evaluation demonstrated that an open system that provides for FDA-cleared tests as well as laboratory-designed tests can provide process improvement in molecular diagnostic laboratories that have relatively complex testing menus and varied production schedules. Future work will focus on the process evaluation of novel complete molecular automation platforms as they emerge in the marketplace.