Time-Motion Analysis of Four Automated Systems for the Detection of Chlamydia trachomatis and Neisseria gonorrhoeae by Nucleic Acid Amplification Testing

Abstract

Next-generation diagnostics for Chlamydia trachomatis and Neisseria gonorrhoeae are available on semi- or fully-automated platforms. These systems require less hands-on time than older platforms and are user friendly. Four automated systems, the ABBOTT m2000 system, Becton Dickinson Viper System with XTR Technology, Gen-Probe Tigris DTS system, and Roche cobas 4800 system, were evaluated for total run time, hands-on time, and walk-away time. All of the systems evaluated in this time-motion study were able to complete a diagnostic test run within an 8-h work shift, instrument setup and operation were straightforward and uncomplicated, and walk-away time ranged from approximately 90 to 270 min in a head-to-head comparison of each system. All of the automated systems provide technical staff with increased time to perform other tasks during the run, offer easy expansion of the diagnostic test menu, and have the ability to increase specimen throughput.

Introduction

Nucleic acid amplification tests for the detection of Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) have evolved from being completely manual to fully-automated systems in a relatively short period of time.^1,2 The degree to which automation is incorporated into each platform is one of the key features that distinguish them from one another. However, there are other important factors to consider, including technical expertise required for operation, ease of system set up, maintenance and reagent preparation, number of available work shifts, laboratory testing volume, and space requirements. Time-motion studies can assist in identifying features or characteristics of each system, which can be used to determine the best fit for a particular clinical laboratory setting. There are few time-motion studies reported in the literature that have evaluated the characteristics of these new platforms, particularly for CT/NG testing.^3–7 Head-to-head evaluations that assess multiple systems using a standardized approach provide critical information to laboratories considering adoption of a new platform. In this study, four automated systems routinely used for the detection of CT/NG were assessed for total run time, hands-on and available walk-away time associated with a run, and ease of instrument set up, operation, and maintenance.

Materials and Methods

A team of time-motion analysts (Argent Global Services, Oklahoma City, OK) specializing in process engineering and management were invited to the Indiana University Infectious Diseases Laboratory and to TriCore Reference Laboratories to observe and measure each step involved in producing diagnostic test results for CT/NG using the ABBOTT m2000 system (m2000), the Becton Dickinson Viper System with XTR Technology (Viper), the Gen-Probe Tigris DTS system (Tigris), and the Roche cobas 4800 (c4800) system. The m2000, Viper, and c4800 were evaluated at Indiana University, whereas the Tigris was evaluated at TriCore Reference Laboratories. Measurements were taken during two full runs, on separate days, on each platform with the exception of the Tigris. Full runs included 96-well formats for m2000, Viper, and c4800; Tigris included runs of 109 and 132 samples, respectively, out of a total instrument capacity of 250 samples, as this was the number of samples received in the laboratory on the days when timing occurred. Viper testing was performed in extracted mode using recently approved second-generation reagents for CT/NG detection (Q^x). All steps associated with the testing of CT/NG on each platform were measured. Timing data were used to establish total run time, hands-on time, and walk-away time for each system. For the purposes of this study, the total run time included all steps associated with CT/NG testing, excluding specimen receiving, accessioning, and transportation. Daily maintenance, along with initial set up of the instrument and specimens, and recapping if necessary postdetection, was included in the timing. The total hands-on time was defined as all interactions with instrumentation and wait times for which the operator was required to be present. The total walk-away time, as defined by the time-motion analysts, included all automation and wait times greater than 30 minutes in which the operator was available to do other tasks and was not required to be present. Thus, a conservative but logistically relevant estimation of walk-away time was used for this study. Each system was assigned a complexity rating by Argent based on their prior experience in evaluating molecular assay workflow, by direct observation of the molecular systems evaluated in this study, and by taking into consideration the Clinical Laboratory Improvement Amendments (CLIA) categorization guide for assessing assay complexity. All of the systems evaluated in this study were categorized as high complexity by CLIA, which requires operator training equivalent to an associate’s degree in a laboratory science. Similar to the CLIA categorization guidelines, the complexity rating constructed by the time-motion analysts evaluated each system for ease of set up and daily maintenance, reagent and specimen preparation, specimen processing, and system operation. A complexity score was formulated for each category based on criteria such as risk of mistakes and contamination, ease of mistakes, system user friendliness, software interface, reagent usability, result integrity, and laboratory information system connectivity. On a scale of 1 to 5, a complexity score of 1 was considered an uncomplicated task, 2 slightly less complicated, 3 neutral, 4 slightly more complicated, and 5 complicated. Measurements of instrument footprint and total space requirements were also obtained.

Results and Discussion

Total run time, hands-on time, and walk-away time are shown in Figure 1 . In our hands, the run time was 3 h 57 min for c4800, 4 h 30 min for Viper, 5 h 59 min for Tigris, and 6 h 20 min for m2000. The elapsed time from initiating a run to the first instance of walk-away time ranged from 33 to 77 minutes and was shortest for the c4800 and longest for the Tigris. Uninterrupted walk-away time, during which technical staff could perform other tasks, was at least 86, 121, 168, and 270 min for c4800, m2000, Viper, and Tigris, respectively. There were additional stretches of 111 and 186 min of walk-away time, respectively, for c4800 and m2000. The percentage of walk-away time versus hands-on time ranged from 62% to 84% ( Fig. 2 ). The time required to complete daily maintenance, reagent preparation, system set up and postdetection shut down by technical staff is categorized in Table 1 . Complexity scores are summarized in Table 2 , with Tigris being slightly less complicated to run and m2000 slightly more complicated. Instrument footprint and total space requirements are shown in Figure 3 . Instrument footprints for the fully automated systems varied from 22.2 ft² (m2000) to 31.4 ft² (c4800), with total space requirements, including instrument footprint and additional workspace requirements, ranging from 49.1 ft² (Viper) to 93.9 ft² (Tigris).

Figure 1.

Total cycle time.

Figure 2.

Walk-away versus hands-on time.

Table 1.

Operator and system interactions (in minutes).

System	Daily Maintenance	Reagent Preparation	System Setup	Postdetection Shut Down	Postdetection Shut Down without Recapping Samples
c4800	5	6	14	8	3.5
m2000	11	13	11	4	4
Tigris	35	9	15	9	9
Viper	21	2	29	20	7

Table 2.

System complexity ratings. ^a

System	Instrument Setup	Daily Maintenance	Reagent Preparation	Specimen Preparation	Specimen Processing	Amplification and Detection	Post Detection	Resulting	Mean Score
c4800	2	2	3	3	3	4	3	3	2.9
m2000	3	3	3	3	4	4	3	3	3.3
Tigris	2	4	3	2	2	2	2	2	2.4
Viper	3	4	2	3	3	3	3	3	3

a

Complexity scale: 1= uncomplicated task, 2 = slightly less complicated, 3 = neutral, 4 = slighlty more complicated, 5 = complicated.

Figure 3.

Instrument and workspace requirements.

The use of professional time-motion analysts allows for a minimally biased evaluation of these automated platforms using standardized metrics: both a standardized protocol for data collection and interviews with laboratory staff to capture each system’s capabilities. This study included evaluation of total run time, determination of complexity levels for operation, and all necessary interactions required by technical staff to successfully complete a run. All of the systems evaluated in this study could easily complete a full run of samples for CT/NG testing within an 8-h work shift. This is advantageous in laboratories with only one shift or in settings where a different analyte is tested during each shift. Expressing walk-away time as a percentage of the total run time, these automated systems freed technical staff to perform other tasks during at least 60% of each diagnostic test run. The hands-on time for the Viper was slightly higher than for the other systems evaluated but included a preincubation and cool-down step for specimens prior to initiating the run. The Tigris required the most amount of time to perform daily maintenance, whereas system setup was similar for all platforms except Viper, which required a longer set-up period due to manual barcode scanning of the sample rack. Reagent preparation was shortest for the Viper and similar for the remaining systems; postdetection shut-down time was closely related for all systems when the time spent recapping samples was excluded. As expected, the automated systems did not require complex user interactions based on the complexity scores assigned by the time-motion analysts, which were found to be neutral to uncomplicated. The ease of using these systems may allow for a single technician to operate multiple platforms or for these systems to be performed outside the specialized molecular laboratory. There are several critical timing events associated with each of these systems, which require the technician to return to the instrument during the run to avoid instrument stoppage or invalidation of the run. This includes initiating a run within 1 h of scanning reagents (c4800), transferring the prepared amplification plate to the real-time instrument within 60 to 90 min (m2000, c4800) of preparation, replenishing amplification plates and pipet tips on the Viper within 90 min when the instrument is used in high-throughput mode, and replenishment of multitube units on the Tigris to avoid suspension of the run. All four of the instrument footprints were similar in size, but the Tigris required almost 64 ft² of additional bench space for testing. This consideration may not be as crucial in large clinical core testing facilities but is relevant in small to mid-sized laboratories in which available open floor and bench space may be at a premium.

There are several limitations to this study. Individual laboratories may perform noncritical multitasking steps differently from one another, which could lead to slight disparities in timing when comparing results between laboratories. Walk-away time was defined as 30 min for this study; however, there are periods of less than 30 min in which experienced technical staff may be able to walk away from the system to perform other tasks. There is a level of subjectivity in the complexity rating, which was formulated by the time-motion analysts. System reliability and customer support were not captured as part of this analysis.

In today’s clinical environment, laboratories are being asked to increase productivity and services offered, but at the same time maintain or reduce staffing levels. The complete automation of diagnostic testing is crucial for laboratories to meet these ever-increasing demands. All of the fully automated systems evaluated in this study provide an excellent means for incorporating CT/NG testing, in addition to other analytes that use the same platform, into a busy clinical laboratory. Head-to-head evaluation demonstrated that the c4800 had the shortest run time and largest percentage of hands-free walk-away time, the m2000 instrument had the smallest footprint, Viper required the least amount of total workspace, and the Tigris was the easiest to operate. All of these factors should be taken into consideration when deciding on which system would fit best in a particular setting. In conclusion, all of these systems have the ability to increase specimen throughput, are user friendly, provide technical staff with time to perform other tasks during the assay run, and offer an opportunity for future expansion of the laboratory test menu.