The Minimum Significant Ratio: A Statistical Parameter to Characterize the Reproducibility of Potency Estimates from Concentration-Response Assays and Estimation by Replicate-Experiment Studies

Abstract

The authors show by illustration that procedures used to validate the reliability of single-concentration high-throughput screens such as the signalwindowand Z' factor do not ensure sufficient reliability in potency estimates fromconcentration response assays. They develop theminimumsignificant ratio as a statistical parameter to characterize the fold change between 2 compounds run in the same experiment that can be considered a real difference and use this parameter to characterize the reliability of the assay. They adaptmethods described by Bland and Altman to develop a simple set of 2 experiments to estimate theminimum significant ratio and show that this protocol can identify assays that lack reproducibility. The methods are then extended to validate the equivalency of the same assay run by multiple laboratories.

Keywords

minimum significant ratio replicate-experiment studies potency reproducibility statistical methods

References

Sittampalam GS , Iversen PW , Boadt JA , Kahl SD , Bright S , Zock JM , et al: Design of signal windows in high throughput screening assays for drug discovery. J Biomol Screen 1997;2:159-169.

Zhang J-H , Chung TDY , Oldenburg KR : A simple statistical parameter for use in evaluation and validation of high throughput screening assays. JBiomol Screen 1999;4:67-73.

Taylor PB , Stewart FP , Dunnington DJ , Quinn ST , Schulz CK , Vaidya KS , et al: Automated assay optimization with integrated statistics and smart robotics. J Biomol Screen 2000;5:213-225.

Iversen PW , Eastwood BJ , Sittenpalam GS : A comparison of assay performancemeasures in screening assays: signalwindow, Z'-factor and assay variability ratio. J Biomol Screen. Forthcoming.

Eastwood BJ , Chesterfield AK , Wolff MC , Felder CC : Methods for the design and analysis of replicate-experiment studies to establish assay reproducibility and the equivalence of two potency assays. In Gad S (ed): Drug Discovery Handbook. Hoboken, NJ: John Wiley and Sons, 2005:667-688.

Bland JM , Altman DG : Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307-310.

Sun D , Whitty A , Papadatos J , Newman M , Donnelly J , Bowes S , et al: Adopting a practical statistical approach for evaluating assay agreement in drug discovery. J Biomol Screen 2005;10:508-516.

Gallant AR : Nonlinear Statistical Models. New York: John Wiley and Sons, 1987.

Ratkowsky DA , Reedy TJ : Choosing near-linear parameters in the fourparameter logistic model for radioligand and related assays. Biometrics 1986;42:575-582.

10.

Kenakin T : Pharmacologic Analysis of Drug-Receptor Interaction. 3rd ed. Philadelphia: Lippincott-Raven, 1997:209-210.

11.

Vander Heyden Y , Nijhuis A , Smeyers-Verbeke J , Vandeginste BGM , Massart DL : Guidance for robustness/ruggedness tests in method validation. J Pharm Biomed Anal 2001;24:723-753.

12.

Eastwood BJ : Comparing two measurement devices: review and extensions to estimate new device variability. Computing Science and Statistics 2001;33:462-471.

13.

Sall DJ , Bastian JA , Briggs SL , Buben JA , Chirgadze NY , Clawson DK , et al: Dibasic benzo[b]thiophenes derivatives as a novel class of active site directed thrombin inhibitors. 1. Determination of the serine protease selectivity, structure-activity relationships and binding orientation. J Med Chem 1997;40:3489-3493.

14.

Smith GF , Craft TJ , Gifford-Moore DS , Coffman WJ , Kurz KD , Roberts E , et al: A family of arginal thrombin inhibitors related to efegatran. Semin Thromb Hemost 1996;22:173-183.

15.

Herron DK , Goodson T Jr. , Wiley MR , Weir LC , Kyle JA , Yee YK , et al: 1,2-Dibenzamidobenzene inhibitors of human factor Xa. J Med Chem 2000;43:859-872.

16.

Hubert CL , Sherling SE , Johnston PA , Stancato LF : Data concordance from a comparison between filter binding and fluorescence polarization assay formats for identification of ROCK-II inhibitors. J Biomol Screen 2003;8:399-409.