Abstract
A survey of directors of screening organizations was conducted in 2001 to evaluate their perceptions of the current vs. desired state of high-throughput screening (HTS) automation. The survey encompassed attributes such as automation flexibility, throughput and operation. These and other automation attributes were ranked based on importance to the respondent and/or the limitations these attributes imposed on the screening organization.
INTRODUCTION
Robotic microplate automation first appeared in the mid 1980's and evolved slowly to form the necessary hardware basis to support high-throughput drug discovery screening (HTS). During the 1990's HTS automation and related technologies evolved rapidly and now, in the opinion of some, has reached a first plateau of maturity. Screening groups have developed their approach to HTS automation over this time through multiple trial and error, and many have now settled on a strategy that works for them. Excellent surveys have been conducted that focus on the broad subject of high-throughput screening. 1 This survey focuses specifically on the current state of screening automation vs. the desired state of the same automation among those who set policy and direction for their screening organizations. Questions were posed related to three aspects of screening automation: flexibility, throughput, and process complexity. Additionally the survey sought ranking of various automation attributes from two standpoints: 1) The technical importance of those attributes, and 2) The limitations those attributes place on the screening organization. The survey, the resulting data and the author's interpretation of those data are presented in this paper.
SURVEY METHOD
Forty directors of US-based HTS organizations were invited to participate in the survey. Seventeen chose to participate and thus received a detailed summary of the results. The survey was initially piloted and refined for clarity and relevance with four participants. Instructions and/or clarification for all participants were accomplished via email or voice communications after the participant had reviewed the survey. An MS Excel version of the survey is shown in Figure 1. The actual survey was a menu-driven document that minimized or eliminated the potential for multiple responses per question.
MS Excel version of HTS automation survey.
RESULTS
The survey was designed for data from one section to support that from another. For instance, there are three questions and six ranking choices related to the subject of automation flexibility. This allowed for a multi-dimensional evaluation of the subject and requires discussion of data from different parts of the survey as each subject is presented below.
FLEXIBILITY
Automation flexibility is a complex subject. Most will agree that this attribute is important, but exactly how and when flexibility is needed and achieved is more difficult to understand. Figure 2 shows the current vs. desired survey data from the following questions related to flexibility:
Flexibility — related results for all respondents. White bars show current situation, black bars show desired situation.
Frequency of implementing new process on system (i.e., requiring system reconfiguration)
Time required to implement new process on system
Organizational philosophy toward screening automation
Within the ranking of the importance of automation attributes shown in Figure 5, the following attributes related to flexibility:
Throughput-related results for all respondents. White bars show current situation, black bars show desired situation.
Complexity-related results for all respondents. White bars show current situation, black bars show desired situation.
Ranking of automation attributes.
Assay flexibility
Ease of programming
Ease of reconfiguring
And within the ranking of the organization limiting attributes shown in Figure 6:
Time required for assay development
Time to validate assay on automated system
Time to reconfigure automation for new processes
Ranking of operation-limiting attributes.
The flexibility of a system to be reconfigured for new assays ranked 2nd (Figure 5) in the importance of automation attributes, however, there is a desire to do such reconfigurations less often (current vs. desired mean: 35.8 vs. 41.3 weeks — Figure 2a). When such reconfiguration must be done, respondents want to do it faster (current vs. desired mean: 9.1 vs. 4.2 days — Figure 2b), though the time required for such reconfiguration was not felt to be organization limiting, ranked 7th (Figure 6). Almost all groups (16 of 17 — Figure 2c) currently use multiple automation systems, reconfigured periodically or dedicated to a process (not reconfigured) to support their HTS process. Only one group reported that they utilize a single system which is reconfigured often. This distribution did not change in the “desired” data.
THROUGHPUT
System throughput is a focus statistic for automated screening systems and ever-growing throughput has been a hallmark of HTS automation development. Throughput can be considered on a per-screen basis or on an overall, program basis, and is influenced by a number of factors. Figure 3 shows the current vs. desired data related to throughput from the following questions: 3a) Typical time to complete a single primary screen for one target
Type of throughput that characterizes your primary screening operation
Typical period of continuous unattended operation
Downtime due to system failure (percentage)
Within the ranking of the importance of automation attributes shown in Figure 5, the following attributes related to throughput:
Throughput
System reliability
Vendor support
And within the ranking of the organization limiting attributes shown in Figure 6:
Throughput per system
Forced automation downtime
Vendor support
There is a strong desire reported to reduce the time required to complete a screen. The respondents currently average 25.6 days to complete a single target screen and would like to significantly reduce that time to 11.6 days (Figure 3a).
Nine of the seventeen respondents would like to increase their throughput by one level (i.e., from MTS to HTS, or HTS to UHTS — Figure 3b), while eight groups desire no increase in throughput. Six of 17 aspire to achieve UHTS levels. No group is currently operating at UHTS levels, with several groups at medium (MTS) and low (LTS) throughput levels. Per-system automated throughput ranks third (Figure 5) in importance of automation attributes but ranks eighth (Figure 6) among attributes that limit the organization.
Currently respondents average 9.6 hours of walkaway, unattended automation operation per system and desire to increase this to an average of 15.4 hours (Figure 3c). However, it should be noted that 12 of 17 respondents felt that no increase in unattended operation time was required. The overall desired average increase was largely influenced by one respondent who desired a walkaway time of 48 hours.
System reliabilty ranked first in importance as a system attribute (Figure 5), but system downtime ranks last, tenth, as an attribute limiting organizations (Figure 6). Respondents currently average 9.9% automation downtime due to system failure and desire an average of 2.1% (Figure 3d). Only three indicated that they desired no change to their system downtime.
COMPLEXITY
Improvements and innovations in assay biochemistry have steadily resulted in simpler screening protocols. How does this impact the current and desired complexity of HTS automation? Figure 4 shows the current vs. desired data related to complexity from the following survey questions:
Typical number of unit operations (steps) per protocol (assay complexity)
Frequency of protocols requiring some type of process branching on automated system
Frequency of running parallel protocols/assays on system simultaneously
Continuous automation between primary screening and secondary and/or follow-up screening
Within the ranking of the importance of automation attributes shown in Figure 5, the following attributes related to complexity:
Ease of End-User Operation
Physical Size of System
Ease of Programming
System Cost
And within the ranking of the organization limiting attributes shown in Figure 6:
Ease of End-User Operation
Time to Validate Assay on Automated System Respondents would like to continue simplifying assays, reducing the current average of 7.3 unit operations/protocol to a desired 5.3 (Figure 4a). No respondent desired to see this number increase. There is a moderate trend away from assays requiring branching within a process (Figure 4b), a moderate trend toward running multiple automated processes in parallel (Figure 4c) and a stronger trend toward continuous automation between primary and secondary screening (Figure 4d).
System size and cost ranked very low in importance, ninth and tenth in Figure 5. Ease of end user operation and ease of programming, ranked mid-range in importance, fifth and sixth in Figure 5. Time to validate assays on a system and ease of end user operations also ranked mid-range among attributes that limit organizations, fifth and sixth in Figure 6.
A BROADER LOOK AT LIMITATIONS
This survey also evaluated a broader spectrum of attributes that might limit screening organizations to determine if other facets of the screening operation and surrounding organization are more or less limiting than automation. The top-ranked organization-limiting attribute reported in the survey (Figure 6) is the time required for assay development. The limiting attributes ranked second, third and fourth relate to the operations surrounding a screening operation: second — Up and downstream infrastructure; third — Result computation and evaluation; and fourth — Data management.
CONCLUSIONS
Almost all of the respondents have achieved flexibility in their organization by using multiple automation systems. The ability to update and adapt these systems to meet new assay needs is considered very important, but most would prefer to do this roughly on an annual basis.
While UHTS gets much attention, none of the groups surveyed were currently operating at UHTS levels, though some have automation that is capable of such. Up and downstream infrastructure realities have proven to be a hindrance to pushing toward ultra throughput. A subtle difference in throughput goals is indicated in that UHTS is not the goal of the majority of the groups responding, but the majority do desire to shorten the time required to complete a screen. While individual assay protocols continue to be simplified, there is an interest in extracting more information per automated run by operating different processes in parallel or linking them end-to-end.
The data reveal several interesting contrasts in what people feel is important in an automated system vs. what is impacting their organization. For instance, system throughput is not reported to be organization-limiting, but continues to be moderately important (third) as an automation attribute. Additionally, system reliability is ranked as the number one attribute, respondents desire a significant decrease in system downtime, but forced system downtime ranks last as an organization-limiting attribute.