A New Method with Flexible and Balanced Control of False Negatives and False Positives for Hit Selection in RNA Interference High-Throughput Screening Assays

Abstract

The z-score method and its variants for testing mean difference are commonly used for hit selection in high-throughput screening (HTS) assays. Strictly standardized mean difference (SSMD) offers a way to measure and classify the short interfering RNA (siRNA) effects. In this article, based on SSMD, the authors propose a new testing method for hit selection in RNA interference (RNAi) HTS assays. This SSMD-based method allows the differentiation between siRNAs with large and small effects on the assay output and maintains flexible and balanced control of both the false-negative rate, in which the siRNAs with strong effects are not selected as hits, and the restricted false-positive rate, in which the siRNAs with weak or no effects are selected as hits. This method directly addresses the size of siRNA effects represented by the strength of difference between an siRNA and a negative reference, whereas the classic z-score method and t-test of testing no mean difference address whether the mean of an siRNA is exactly the same as the mean of a negative reference. This method can readily control the false-negative rate, whereas it is nontrivial for the classic z-score method and t-test to control the false-negative rate. Therefore, theoretically, the SSMD-based method offers better control of the sizes of siRNA effects and the associated false-positive and false-negative rates than the commonly used z-score method and t-test for hit selection in HTS assays. The SSMD-based method should generally be applicable to any assay in which the end point is a difference in signal compared to a reference sample, including those for RNAi, receptor, enzyme, and cellular function. (Journal of Biomolecular Screening 2007:645-655)

Keywords

strictly standardized mean difference d+-probability restricted false-positive rate completely balanced error level high-throughput screening hit selection

References

Fire A. , Xu S. , Montgomery MK , Kostas SA , Driver SE , Mello CC : Potent and specific genetic interference by double-stranded RNA in Caenorhabditis elegans. Nature 1998;391:806-811.

Hannon GJ , Zamore PD : Small RNAs, big biology: biochemical studies of RNA interference. In Hannon GJ (ed): RNAi: A Guide to Gene Silencing. New York: Cold Spring Harbor Laboratory Press, 2003:87-108.

Mahanthappa N. : Translating RNA interference into therapies for human diseases. Pharmacogenomics 2005;6:879-883.

Check E. : Silent running: the race to the clinic. Nature 2006;442: 614-615.

Zuck P. , Murray EM , Stec E. , Grobler JA , Simon AJ , Strulovici B. , et al: A cell-based beta-lactamase reporter gene assay for the identification of inhibitors of hepatitis C virus replication. Anal Biochem 2004;334:344-355.

MacKeigan JP , Murphy LO , Blenis J. : Sensitized RNAi screen of human kinases and phosphatases identifies new regulators of apoptosis and chemoresistance. Nat Cell Biol 2005;7:591-600.

Nybakken K. , Vokes SA , Lin TY , McMahon AP , Perrimon NA : Genome-wide RNA interference screen in Drosophila melanogaster cells for new components of the Hh signalling pathway. Nat Genet 2005; 37:1323-1332.

Pelkmans L. , Fava E. , Grabner H. , Hannus M. , Habermann B. , Krausz E. , et al: Genome-wide analysis of human kinases in clathrin- and caveolae/raft-mediated endocytosis. Nature 2005;436:78-86.

Bard F. , Casano L. , Mallabiabarrena A. , Wallace E. , Saito K. , Kitayama H. , et al: Functional genomics reveals genes involved in protein secretion and Golgi organization. Nature 2006;439:604-607.

10.

Espeseth AS , Huang Q. , Gates AT , Xu M. , Yu Y. , Simon AJ , et al: A genome wide analysis of ubiquitin ligases in APP processing identifies a novel regulator of BACE1 mRNA levels . Mol Cell Neurosci 2006;33:227-235.

11.

Zhang J-H. , Chung Tdy , Oldenburg KR : A simple statistical parameter for use in evaluation and validation of high throughput screening assays. J Biomol Screen 1999;4:67-73.

12.

Zhang J-H. , Chung Tdy , Oldenburg KR : Confirmation of primary active substances from high throughput screening of chemical and biological populations: a statistical approach and practical considerations. J Comb Chem 2000;2:258-265.

13.

Zhang J-H. , Wu X. , Sills MA : Probing the primary screening efficiency by multiple replicate testing: a quantitative analysis of hit confirmation and false screening results of a biochemical assay. J Biomol Screen 2005;10:695-704.

14.

Wu X. , Sills MA , Zhang J-H. : Further comparison of primary hit identification by different assay technologies and effects of assay and screen measurement variability. J Biomol Screen 2005;10:581-589.

15.

Gunter B. , Brideau C. , Pikounis B. , Liaw A. : Statistical and graphical methods for quality control determination of high-throughput screening data. J Biomol Screen 2003;8:624-633.

16.

Brideau C. , Gunter B. , Pikounis B. , Liaw A. : Improved statistical methods for hit selection in high-throughput screening . J Biomol Screen 2003;8: 634-647.

17.

Kevorkov D. , Makarenkov V. : Statistical analysis of systematic errors in high-throughput screening. J Biomol Screen 2005;10:557-567.

18.

Malo N. , Hanley JA , Cerquozzi S. , Pelletier J. , Nadon R. : Statistical practice in high-throughput screening data analysis. Nat Biotechnol 2006; 24:167-175.

19.

Zhang XD : A pair of new statistical parameters for quality control in RNA interference high throughput screening assays. Genomics 2007; 89:552-561.

20.

Zhang XD , Espeseth AS , Chung N. , Ferrer M. : Evaluation of a novel metric for quality control in an RNA interference high throughput screening assay. Biocomp 2006:385-390.

21.

Zhang XD , Espeseth AS , Chung N. , Holder DJ , Ferrer M. : The use of strictly standardized mean difference for quality control in RNA interference high throughput screening experiments. American Statistical Association Proceedings 2006:882-886.

22.

Zhang XD , Yang XC , Chung N. , Gates AT , Stec E. , Kunapuli P. , et al: Statistical analysis in RNA interference high throughput screening experiments. Paper presented at The First RNAi Meeting, Cold Spring Harbor, NY, November 2005.

23.

Zhang XD , Yang XC , Chung N. , Gates AT , Stec E. , Kunapuli P. , et al: Exploring statistical methods for hit selection in RNA interference high throughput screening experiments. American Statistical Association Proceedings 2005:723-729.

24.

Zhang XD , Yang XC , Chung N. , Gates AT , Stec E. , Kunapuli P. , et al: Robust statistical methods for hit selection in RNA interference high throughput screening experiments. Pharmacogenomics 2006;7:299-309.

25.

Cohen J. : The earth is round (p < 0.05). Am Psychol 1994;49:997-1003.

26.

Kirk R. : Practical significance: a concept whose time has come. Educ Psychol Meas 1996;56:746-759.

27.

Tukey JW : Exploratory Data Analysis. Reading, MA: Addison-Wesley, 1977.

28.

Zhang XD , Ferrer M. , Espeseth AS , Marine S. , Stec E. , Crackower M. , et al: The use of strictly standardized mean difference for hit selection in primary RNA interference high throughput screening experiments. J Biomol Screen. In press.

29.

Vysochanskij DF , Petunin YI : Justification of the 3-sigma rule for unimodal distribution. Theory of Probability and Mathematical Statistics 1980;21:25-36.