Nuisance Compounds,PAINS Filters,and Dark Chemical Matter in the GSK HTS Collection

Abstract

High-throughput screening (HTS) hits include compounds with undesirable properties. Many filters have been described to identify such hits. Notably, pan-assay interference compounds (PAINS) has been adopted by the community as the standard term to refer to such filters, and very useful guidelines have been adopted by the American Chemical Society (ACS) and subsequently triggered a healthy scientific debate about the pitfalls of draconian use of filters. Using an inhibitory frequency index, we have analyzed in detail the promiscuity profile of the whole GlaxoSmithKline (GSK) HTS collection comprising more than 2 million unique compounds that have been tested in hundreds of screening assays. We provide a comprehensive analysis of many previously published filters and newly described classes of nuisance structures that may serve as a useful source of empirical information to guide the design or growth of HTS collections and hit triaging strategies.

Keywords

high-throughput screening nuisance compounds PAINS filters frequent hitters inhibitory frequency index

Introduction

High-throughput screening (HTS) is a critical tool in the identification of lead molecules for novel therapeutic targets in drug discovery.^1,2 Over the past 20 years GlaxoSmithKline (GSK) has continuously invested in the development and curation of its HTS collection. After the merger of Glaxo Wellcome and SmithKline Beecham in 2001, the creation of a combined GSK HTS collection started with an extensive quality control (QC) program for all compounds,³ the definition and refinement of substructure and property filters to remove compounds not suitable for HTS,⁴ the development of an integrated strategy for collection growth,⁵ and extensive compound synthesis and compound acquisition efforts in line with this strategy.⁶

The issue of identifying useless primary screening hits caused by nuisance compounds has been well known since the advent of HTS in the early 1990s.⁷ Several articles have been published describing a number of chemotypes that display nonspecific effects, like chemical reactivity, assay signal interference, and colloidal aggregation.^8,9 Baell coined the term pan-assay interference compounds (PAINS),^10–14 which has been adopted as an imprecise synonym of nuisance. There are notable significant additional works^15–18 that use chemical substructures represented by the convenient SMARTS notation (SMiles ARbitrary Target Specification) for identifying compounds to be flagged for potential nuisance. Since automation impacted chemistry and screening, it has become routine work for many pharmaceutical companies to catalog reactive or undesirable compounds.^19–21 More recently, the American Chemical Society (ACS) has adopted clear guidelines to filter out nuisance hits in articles submitted to any ACS journal.²² One characteristic of nuisance compounds not often cited is their inconsistent behavior: a bad actor is rarely a hit in every assay in which it is tested; instead, it shows up as interference given specific conditions, such as the presence of an important free cysteine thiol group in a protein or the use of a particular wavelength at a certain pH in an assay. Therefore, a fit-for-purpose collection of filters for a screening collection requires more than a handful of specific assays using a single technology tested against a small number of compounds sharing a chemical feature, a common criticism of the original PAINS paper that has been recently backed by questioning many filters based on the analysis of the bioactivity profile of all compounds in PubChem.^23–30

In 2008, an analysis of frequent hitters in HTS campaigns conducted in GSK revealed 750 noisy compounds that had been subject to more than 13,000 dose–response experiments. Subsequently, we set out to measure the activity profile of every compound of the GSK HTS collection over hundreds of campaigns using the inhibitory frequency index (IFI), a pragmatic and intuitive measure of promiscuity.³¹ The promiscuity of a chemotype can be assessed by measuring the IFI profiles of analogs bearing a substructure of interest across different target classes and assay technology, allowing us to distinguish nonspecific nuisance effects from specific assay interference or bona fide biological activity within a particular target class.

In this article, we show the promiscuity profile of the whole GSK HTS collection comprising more than 2 million unique compounds. We provide a comprehensive analysis of many previously published filters and newly described classes of nuisance structures that may serve as a useful source of empirical information to guide the design or growth of HTS collections and hit triaging strategies.

Materials and Methods

Compounds and Assays

The GSK screening collection comprises approximately 2 million diverse compounds at any given time, with continuous addition of new chemicals and exhaustion or disposal of old samples.^6,32 Over the last 10 years 2,586,543 compounds have been tested at a compound concentration of 10 µM in GSK in at least 50 of 334 primary HTS assays (both biochemical and cellular) looking for inhibitors or antagonists of a diverse set of targets (excluding kinases). A smaller number of assays looking for agonists are not considered in this study. The distribution of target classes and technologies is shown in Figure 1 . Kinase assays (47) were excluded from the analysis of nuisance filters because kinase inhibitors, even if specific for one particular kinase, tend to be active against many kinases at the relatively high concentration tested, distorting the analysis of nuisance profiles. However, kinases assays were included in the assessment of target and technology bias.

Figure 1.

Distribution of HTS assays run in GSK per target class (A) and assay technology (B).

Inhibitory Frequency Index

We used inhibition ≥ 50% as a uniform cutoff to define hits; for most screening campaigns, this cutoff is above the statistical cutoff calculated to select hits for secondary confirmation (typically performed in duplicates);³³ the result computed for the IFI calculation is the median of all available results at 10 µM for every compound.

IFI is defined as the proportion (%) of nonkinase assays in which a compound shows inhibition ≥ 50%,³, calculated with eq 1:

I F I = \frac{\begin{array}{l} Number of HTS assays (exc . kinases) \\ in which median % inhibition \geq 50 % \end{array}}{Total number of HTS assays (exc . kinases)} * 100

We only considered for the analysis compounds that have been tested in at least 50 HTS assays.

“IFI with kinases” was calculated using the same formula but including kinase assays in the analysis.

The IFI analysis hereby reported for every compound in the record of GSK HTS was conducted in January 2017. In January 2012, a similar analysis was performed using results from 185 HTS assays.

Compound Classification according to IFI

Based on heuristics, compounds with IFI > 2 were considered noisy. We used the following bins to classify compounds according to their IFI:

IFI = 0: Inactive or “dark chemical matter”³⁴

0 < IFI ≤ 1: Specific active

1 < IFI ≤ 2: Moderately active

2 < IFI ≤ 5: Somewhat noisy

5 < IFI ≤ 10: Noisy

10 < IFI: Very noisy

Nuisance Filters

Publicly available SMARTS substructure filters have been obtained from Jadhav et al.,⁸ PAINS¹⁴ (the original PAINS filters were listed in SLN and were converted to SMARTS internally), the structural alert tables in the ChEMBL 22 database,¹⁷ the “patterns” web service in the ZINC 15 database,¹⁸ BMS filters,¹⁵ and those that had a SMARTS representation from Lilly.¹⁶ GSK CTC (Cleaning the Collection) filters were developed in GSK after analysis of noisy compounds in the first IFI analysis in-house conducted in 2008.

Filters were described by the following characteristics:

N = number of compounds identified in GSK collection belonging to the filter. Note that filters with < 10 members were not further analyzed.

Fraction of noisy compounds = proportion of compounds in the filter that are somewhat noisy, noisy, or very noisy (i.e., IFI > 2).

Target class bias = A filter shows target bias if one to three target classes fall above the bias cutoff.

Assay technology bias = A filter shows assay technology bias if one to three assay technologies fall above the bias cutoff.

Chemical structures were extracted from the GSK registry in a standardized SMILES format as the parent structure (salts removed and neutral form). SMARTS matching was performed using the ChemAxon Java toolkit (JChem 16.7.4, http://www.chemaxon.com) with data processing in Pipeline Pilot (Pipeline Pilot 2017R2, Biovia, https://www.3ds.com/products-services/biovia/).

In order to assess target class and assay technology bias, we first obtained a summary measure for each filter and assay technology/target class as follows: first, determined the percentages of assays (among assays testing a specific assay technology or target class) that a specific compound showed > 50% inhibition. Then, calculated the median of this percentage across all compounds within a given filter and assay technology or target class. Nonnoisy compounds (i.e., IFI with kinase ≤ 2) and compounds that were tested in < 50 assays were excluded from these calculations. Next, a summary measure-based bias cutoff was calculated across all filters via the mean + 3*SD approach. The bias cutoffs for target classes are 20.92% and 21.33% for assay technologies. As a last step, we assessed the number of assay technologies or target classes above the cutoff for each filter. Filters with one to three assay technologies or target classes above the cutoff were considered biased. Once too many (i.e., more than three) assay technologies or target classes are above the bias cutoff, this filter is considered as having overall high promiscuity but no actual assay technology or target class bias.

Statistical Methods

An exact test for proportions was used to compare the fraction of noisy compounds identified by the various filters to the overall fraction of noisy compounds found in the GSK HTS collection, which is p_Overall = 0.1287. This fraction was calculated by taking the ratio of the total number of noisy compounds (i.e., compounds with IFI > 2) to the total number of compounds in the GSK HTS collection. These significance tests were used to classify filters as “potential useful,” “ambiguous,” or “not useful,” as described below. Two-sided tests with a p value < 0.05 were used to determine significance.

Results

Promiscuity Profile of GSK Collection

As shown in Figure 2 , a large proportion of compounds in the GSK collection are dark chemical matter (48.3% did not hit at 50% inhibition or greater in any HTS assay, including results for kinases). There is only a small proportion (0.5%, not including kinases) of very noisy compounds that were for the most part removed from the screening collection soon after they were identified in 2008.

Figure 2.

The IFI distribution of the GSK collection and 1090 marketed drugs routinely tested in GSK HTS screens.

Promiscuity Profile of Some Approved Drugs

The IFI profile of marketed drugs included in the GSK collection is also shown in Figure 2 . Interestingly, it is quite similar to the profile of the whole collection. There are many drugs in the inactive bin (33%). It should be noted that (1) some of these drugs are agonists or activators (not captured by IFI) and (2) the collection of assays tested in GSK did not include every drug target. As previously described,³⁵ there are also some drugs exhibiting noisy behavior at 10 µM. Upon further analysis of the 15 noisiest marketed drugs ( Suppl. Table S1 ), some are toxic molecules with antiparasitic or antineoplasm activities and others are very potent (picomolar or better) and specific to one or a few related targets that do show activity at the HTS concentration of 10 µM against other targets. We do need to remind ourselves what Paracelsus observed some 500 years ago, sola dosis facit venenum (all substances are poisons; there is none that is not a poison. The right dose differentiates a poison from a remedy).³⁶

Dark Chemical Matter

The estimated proportion of dark chemical matter in the Novartis collection (14%)³⁴ contrasts with the value of 48.3% for the GSK collection hereby reported. This difference could be due to the nature of the chemical compounds in every collection or the type of assays run. Indeed, the GSK collection has been largely populated with compounds selected to maximize chemical diversity, and the proportion of inactive compounds closely follows the results of a simulation based on our mathematical model⁵ when this collection is presented in silico against a typical portfolio of screens. In contrast, a chemogenomics model as pursued by Novartis might be expected to yield more compounds with multiple activities. Another plausible explanation for the difference observed may result from using cutoffs based on statistical analysis of activity distribution to define hits per assay even if applying “a particularly stringent definition for biological inactivity in an assay (absolute z-score < 2)” (Novartis criterion) instead of the uniform 50% inhibition used to determine IFI (GSK criterion). Statistical cutoffs based on 3 standard deviations of negative controls or average activity across all compounds tested in HTS tend to be far lower than 50% inhibition and do include a significant proportion of random false positives. It should be noted that using a 50% inhibition cutoff is likely to result in a higher false-negative rate.

When IFI values recorded in 2012 for about 2 million compounds are compared with their 2017 values with 88 additional assay results, some interesting trends are observed ( Table 1 ). As described by Novartis’s authors,³⁴ dark chemical matter is not expected to be dark forever. When tested against different targets, 7.4% of the 2012 inactive compounds do exhibit some level of activity. Unexpectedly, 0.1% even exhibit a somewhat noisy behavior (i.e., 2 < IFI ≤ 5). Analysis of the activity profile of these compounds shows that most are kinase inhibitors that hit phenotypic assays (not excluded from the IFI analysis). Every IFI bin shows consistency between 2012 and 2017 classification, with more compounds changing bins from the intermediate levels (bold, Table 1 ).

Table 1.

Change in IFI Distribution of the GSK Collection from 2012 to 2017.

		2012
		IFI = 0	0 < IFI ≤ 1	1 < IFI ≤ 2	2 < IFI ≤ 5	5 < IFI ≤ 10	10 < IFI
	Total no. compounds	887,580	371,848	394,707	308,082	66,991	12,989
2017	IFI = 0	92.5%	0.0%	0.0%	0.0%	0.0%	0.0%
	0 < IFI ≤ 1	6.4%	94.3%	35.9%	0.4%	0.0%	0.0%
	1 < IFI ≤ 2	0.9%	5.4%	60.2%	27.8%	0.1%	0.0%
	2 < IFI ≤ 5	0.1%	0.3%	3.9%	70.1%	28.2%	0.1%
	5 < IFI ≤ 10	0.0%	0.0%	0.0%	1.7%	69.8%	17.5%
	10 < IFI	0.0%	0.0%	0.0%	0.0%	1.9%	82.4%

Definition of New Nuisance Filters

When the structures of nuisance compounds in the GSK collection were analyzed, some new substructure filters were identified between 2008 and 2010 using a variety of approaches. Data-driven analysis³⁷ was developed initially for analysis of HTS data. A variation on the method to use binary class information has been used successfully to identify chemotypes unstable in DMSO.^38,39 As IFI is on a scale of 0%–100% (though heavily skewed to the right for our collection, as the majority of compounds have IFI values ≤ 1), it is a natural end point for data-driven analysis. A number of whole-molecule and substructure-type descriptors are used to label the molecules, with each label being considered independently to assess whether compounds with the label have a significantly higher mean IFI than the mean of the whole data set (see Harper et al.³⁷ for details). Descriptors considered here include reduced graphs³⁷ and reduced graph fragments, Bemis–Murcko frameworks⁴⁰ (full atom, bond graph, and graph), rings and ring assemblies, recap fragments,⁴¹ and substructure matches against previously defined filters.

New filters were derived from the analysis of significant descriptors, tested for specificity and agreed on by chemists involved in hit triage. Examples are shown in Table 2 . Note that since originally described internally, some of these filters have been published with similar structural information (e.g., azapteridines, described in 2012 by Bruns and Watson¹⁶). A visual summary of the process described above to identify nuisance filters is included in the supplemental material ( Suppl. Fig. S1 ).

Table 2.

Nuisance Filters Identified by Structure Analysis of Noisy Compounds in the GSK HTS Collection.

Filter	SMARTS	Structure
Thiooxopyrimidinedione	[#6]C=C1C(=O)NC(=S)NC1=O
Thioacetal	[CX4]1NC(S[CX4]1)C=O
Thiazolidinedione	[$([$([#1,*])]N1C(=S)SC(=Ca)C1=O),$([nX3]1sccc1=O)]
Thiazolequinolineamine	N(c1ccccc1)c1ccnc2cc(ccc12)-c1nccs1
Sulfenamide	[$([#6,#16]n1sc(=O)c2ccccc12),$(O=c1s[n]c2ccc(cc12)),$(nsc=O),$(sn(-a)c(=O)c),$(s[nH,$(nA)]c(=O)c)]
Pyrimidinesulfone	[#6]S(=O)(=O)c1ncc([Cl,Br])c(n1)C([#7])=O
Pyridoguanidine	[$([#7]-,=C(!@-,!@=N!@-,!@:c:1n:a:a:a:a:1)!@-,!@=N!@-,!@:c:1:a:a:a:a:a:1),$([nX2r6]([c!$(c[OH])!$(c=O)])c[N;$(N=C([NH2])[NH]a),$([NH]C([NH2])=Na)])]
Pyridinylpropene	[#6][C;!H0]=[C;!H0]c1ccncc1
Phenolbenzylamine	[$([NH]([c,CX4])[c,CX4]),$([NH2][c,CX4])][C;!H0]c1:a:a:a:a:c1[#8]
Phenazine	[#7]c1ccc2nc3ccccc3nc2c1
p-Nitrosulfur	[#6]Sc1ccc(cc1)N(~[OD1])~[OD1]
o-Sulfonamide benzamide	[$(O=C(Nc1ccccc1)c1ccccc1NS(=O)(=O)c1ccccc1),$(c1ccccc1S(=O)(=O)[NH]c2ccccc2C(=O)[NH]c3ccccc3)]
Nitrothienylpropene	[#6][C;!H0]=[C;!H0]c1ccc(s1)N(~[OD1])~[OD1]
Nitrofuranylpropene	[#6][C;!H0]=[C;!H0]c1ccc(o1)N(~[OD1])~[OD1]
Nitrobenzodioxazole	[$([OD1]~N(~[OD1])c1cccc2nonc12),$(O=N(=O)c1cccc2nonc12)]
Napthalineimide	[#6]N1C(=O)c2cccc3cccc(C1=O)c23
Maleimide	O=C1NC(=O)C=C1
Isothiazolinone	O=C1CCSN1
Imidazolediones	[#6]N1C(=O)C(=O)c2ccccc12
Halogenated benzimidazole	[#6]n1c(Br)nc2cc(Cl)c(Cl)cc12
Disulfide	[#6]SS[#6]
Dimethoxyaniline	[#6;D1]Oc1ccc([$([NH]([c,CX4])[c,CX4])])cc1O[#6;D1]
Dihydropyrazoleone	[#6]C=C1C([#6])=NN(C1=O)c1ccccc1
Dihydroisoquinoline	[$([#6][C;!H0]=[C;!H0]C1NCCc2ccccc12),$([#6][CH]=[CH]C1=NCCc2ccccc12)]
Chromenooxazinone	C1NCc2ccccc2O1
Azapteridine	[$([#6]n1ncac2c1nc(=O)n([#6])c2=O),$(O=c1nc(=O)nc2nn[c;!$(c=O)]nc12)]
Thiadiazolesulfinyl	O=Cc1nnsc1S(=O)[#6]
o-Nitrosulfur	[#6]Sc1ccccc1N(~[OD1])~[OD1]

These filters were not described in the literature in 2008. The structure depictions are provided as a guide, and the SMARTS queries must be used to retrieve the compounds accurately. Recommended public SMARTS viewer: http://smartsview.zbh.uni-hamburg.de/.

Analysis of Nuisance Filters

The promiscuity profile of nuisance filters in the GSK collection is shown in Figure 3 . A perfect nuisance filter would identify a set of compounds that are all noisy (the noisy fraction would be 1). There are only two filters in this category that happen to identify the same number of compounds in the GSK collection (14) and thus appear as a single overlapping dot in the scatterplot. These filters are thiadiazolesulfinyl (described in Table 2 ) and dihydroxybenzene.⁸ On the other end of the spectrum, there are eight apparently useless filters that fail to identify even a single noisy compound in the GSK collection. The identity and characteristics of all filters showed in this plot are available in Supplemental Table S2 .

Figure 3.

Overview of nuisance filters analyzed. “N Total” represents the number of compounds in the GSK collection identified by every filter, and “Fraction Noisy” the fraction of compounds in every filter showing IFI > 2. Only unique and head-of-family filters with at least 10 compounds tested are shown. Shapes depict labels by statistical analysis: potentially useful (circles), ambiguous (diamonds), and not useful (crosses). Colors indicate recommended course of actions: remove (red), extreme caution (orange), proceed with caution (yellow), and not a PAIN (green). The physicochemical filters are labeled and shown as blue stars.

For many published filters (96), there were < 10 compounds in the GSK collection and they are not subject to further analysis ( Suppl. Table S3 ). Many filters have been published with identical criteria, and they have been treated as a single filter in this work. Other filters defined independently share similar criteria; we have grouped these related filters in families; for every family, we have selected the filter identifying a higher fraction of noisy compounds (IFI > 2). There are 347 “head-of-family” or “unique” filters after this step (marked in Suppl. Table S2 ; note that some unique filters have been described more than once in the literature, as acknowledged in the column “source”). Figure 3 shows graphically a statistical analysis to qualify the usefulness of a nuisance filter to detect true nuisance clusters. If the proportion of noisy compounds in a filter is

Significantly larger than the overall fraction of noisy compounds, which is p_Overall = 0.1287 (see Statistical Methods section), then this filter is labeled “potentially useful” in identifying nuisance compounds

Not significantly different from the overall fraction of noisy compounds (p_Overall = 0.1287), then the filter is labeled “ambiguous” in identifying nuisance compounds

Significantly smaller than the overall fraction of noisy compounds (p_Overall = 0.1287), then this filter is labeled “not useful” in identifying nuisance compounds

Figure 4 shows examples of the distribution of IFI bins for all compounds in filters in the three categories identified. Supplemental Table S2 provides a summary of critical information for every filter.

Figure 4.

Promiscuity profile for a few nuisance filters exemplifying the differences observed in enrichment in noisy compounds.

Using the statistical criteria above, we identified 198 potentially useful, 102 ambiguous, and 47 not useful filters. However, practical application of filters is not a binary decision of “screen” or “not screen.” Compounds can be kept in a collection but marked as suspicious; a compound class may be kept in the collection but not added to by further acquisitions. For some of the filters with large membership, there is a loss of many compounds with no evidence of bad behavior, for example, the ChEMBL Hetero-Hetero filter identifies 55,350 noisy compounds but also labels 305,470 well-behaved compounds. Removal of all these compounds would seriously degrade the hit-finding potential of a compound collection.

There is also the degree of noisiness, for example, the small number of biotin derivatives in the set are noisy (18/20 with IFI > 2) and often very noisy (12/20 with IFI > 5), presumably in specific assay formats.

Statistical significance can mask limitations due to the domain of applicability of the filter. For example, the ChEMBL Filter34_isotope filter (containing elements that are not natural isotopes) has 8/14 matches from the same chemotype, varying in the amine of the amide (and position of the isotope). Outside of the chemotype, it is not clear why deuterated compounds would be frequent hitters.

At the lower end, there are filters that clearly do not label frequent hitters but do remove compounds that it would be sensible not to work on. For example, nitrosamine is a known problematic structure due to genetic toxicology.⁴²

Therefore, we have taken the initial filter classification and labeled it according to recommended action:

Fraction noisy ≤ 0.2: Not a PAIN

This is not a filter that describes frequent hitters or cross-assay interference. It may describe compounds that are not desirable due to other effects.

For compounds with fraction noisy > 0.2, three further criteria are used:

○ p value (two sided exact) < 0.05

○ Fraction noisy ≥ 0.5

○ Fraction noisy (IFI_10) ≥ 0.1

If a filter meets all three criteria, removal of compounds from the collection is recommended.

If a filter meets two of the criteria, extreme caution is advocated, as the molecules have a high risk of being nonselective, interference, or false positives.

The remaining filters, which show some but not overwhelming evidence of high IFI, are marked proceed with caution. This will translate to different decisions according to an organization’s position on risk taking. For GSK, this generally translates to flagging of the physical compounds and no further acquisitions. Other organizations may decide to screen and deal with any problems afterwards.

In summary, as disclosed in Supplemental Table S2 , 169 filters are “not a PAIN” (i.e., the filter is not useful), 121 “proceed with caution,” 31 “extreme caution,” and 23 “removal.”

The “extreme caution” and “removal” categories collectively identify 20,445 compounds from the GSK collection, of which 11,745 (57%) are noisy. The “proceed with caution” category identifies 830,977 compounds, of which 277,970 (33%) are noisy, the majority of which (166,451, or 60%) are marked by the physicochemical filters of clogP > 5, molecular weight > 500 Da, or violators of Lipinski’s rule of 5 (Ro5).⁴³ As shown in Figure 3 , the fraction of noisy compounds among those with a molecular weight > 500 (35% noisy), clogP > 5 (47%), and Lipinski’s Ro5 fail (44%), corroborating the well-documented promiscuity associated with these properties.⁴⁴

These proportions highlight the difficulties associated with the derivation of “nuisancephores”: many compounds identified by these filters are inactive, and complete removal of these compounds from a screening collection using literature PAINS filters is not recommended. The drafting of fit-for-purpose substructure filters requires close attention to detail and, ideally, deep understanding of the mechanism of action that explains the pan-active behavior associated with the nuisancephore.

Target Class or Assay Technology Bias

High promiscuity might be associated with biologically meaningful pan-activity, as in the case of kinase inhibitors. It can also be the result of assay technology-specific interference without a detrimental effect in medicine (e.g., optical interference).

Eight potentially useful filters were identified as showing target bias in up to two target classes. These filters are shown in Supplemental Table S4 .

Twenty-one potentially useful filters were identified as showing assay technology bias in up to three technology classes, with most biases occurring in three technology classes: “other technologies,” “absorbance,” and “alphalisa.” These filters are marked accordingly in Supplemental Table S5 .

The relatively low number of filters with target class (8/196) or assay technology bias (21/196) informs the debate about how universal filters can be described using a small number of assays. As can be observed in Supplemental Table S2 , filters derived from a single assay (cruzain, a thiol protease)⁸ or from a handful of assays in a single technology (AlphaScreen)¹⁰ can be very informative of pan-activity provided that the target or assay technology is a good sentinel of generic mechanisms of interference, as in both examples.

Utility of Abbott Tiers

In addition to structural filters, there is a large body of literature that attributes responsibility for the nuisance behavior of HTS compounds to physicochemical properties^45,46 (e.g., molecular weight, clogP, rotatable bonds, and a combination of properties and activity). The Abbott Physicochemical Tiering⁴⁷ system groups compounds based on physicochemical properties, representing a pragmatic approach in the selection of hits to follow-up, without removing compounds from an HTS collection. There are seven tiers, with Tier 1 compounds satisfying rule of 4 (Ro4), number of aromatic ring (NAR) < 4, and fraction of SP3 carbon atoms (fsp3) > 0.5. Tier 2 compounds satisfy Ro4 and NAR < 4. Tier 3 compounds simply satisfy Ro4. Tier 4 compounds satisfy Ro5, NAR < 4, and fsp3 > 0.5. Tier 5 contains compounds that satisfy Ro5 and NAR < 4. Tier 6 contains those compounds that simply pass Ro5. Tier7 compounds fail Ro5.

Supplemental Figure 2S shows the IFI profiles of Tiers 1–7 for the entire GSK HTS collection. The Tier 7 IFI profile is qualitatively very different from those of the lower tiers, and we do see a trend of an increased fraction of noisier compounds in the higher tiers. The upper tiers (5–7) can signal HTS hits of lower priority for follow-up.

Sample Contaminants as Nuisance

Although most publications describing nuisance screening hits have focused on the chemical structures of the false positives, a less widely discussed, but equally common effect, is interference activity due to minor contaminants in the screening samples. Even compounds that appear to be of high purity (defined by liquid chromatography–mass spectrometry [LC-MS] and nuclear magnetic resonance [NMR]) have been shown to lose activity upon re-preparation or enhanced purification due to removal of impurities invisible to these commonly used analytical techniques. Over the years, many potential sources of contamination have been suggested but have ultimately been difficult to prove. Occasionally, these contaminants might even behave as specific effectors (e.g., a plastic contaminant⁴⁸).

At GSK, we experienced this problem in the evaluation of hits from an HTS campaign looking for inhibitors of the tyrosine kinase ZAP70 using a fluorescence resonance energy transfer (FRET)-based assay. Detailed investigation revealed that the observed inhibition was due to a contaminant introduced during purification by an acidic ion exchange cartridge. Although the exact identity of the isolated contaminant remains unknown, its biological activity was consistent with activity observed with samples where the contaminant was present as a minor component. Activity therefore had been erroneously attributed to the compound structure. Supplemental Figure S3 shows the dose–response curves for two ZAP70 diversity screening hits, before and after the purification process, along with the dose–response curve for the isolated ion exchange extract.

Broader analysis showed that the activity of this contaminant had been observed in several unrelated screens, and the investigation into potential sample contaminant now forms part of our evaluation of putative hits.

Furthermore, some combinatorial arrays seem to be noisy due to the presence of impurities left behind in the synthesis or purification of the compounds, and these impurities can be present in different levels confounding spurious activity from the contaminant with bona fide structure–activity relationship (SAR).

Examples of IFI for a few noisy arrays are included in Supplemental Table S6 .

Another example of sample contaminants showing inhibitory activity has been observed on samples purified with trifluoroacetic acid (TFA) that upon final evaporation might retain excess acid. Compounds registered as TFA salts in the GSK collection (>200,000) show a fraction of noisy compounds of 19.15%, above the average of 12.85%. Specific assays involving pH-sensitive reactions tend to suffer from this artifact.

Discussion

Nuisance compounds have contaminated hit lists since the early days of HTS. We, like many other groups, find filters essential to prevent the inclusion of undesirable compounds in screening collections and to allow their exclusion from hit lists. It is hard to properly define these filters due to the relatively moderate activity of nuisance compounds when tested against diverse targets using diverse assay technologies, and due to the heterogeneous profile of analogs that often do not exhibit an obvious promiscuous profile. In our work, we have found it necessary to employ a variety of filter strategies: substructures for reactivity, desirability, stability, and true PAINS, as well as molecular property, promiscuity (IFI), and sample provenance considerations.

Throughout this work, we have employed heuristics and customized methods to define the promiscuity of compounds and interrogate the validity of PAINS filters. Other groups have reached similar conclusions using different tools. Most reports have used PubChem, a public repository of screening data on a reasonably sized compound collection. A precedent in a pharma collection was published by Nissink and Blackburn exploring frequent hitters in the AstraZeneca HTS collection.⁴⁹

Our evaluation of the published PAINS filters in the GSK screening collection identified some filters that do not group compounds with significant promiscuity. Nevertheless, even these filters might be useful if the potential bioactivity from members of the class can be ascribed to undesirable chemical properties. As with all areas of drug discovery, overreliance on one particular set of rules or principles can be counterproductive. Data-driven analysis combined with good practice in hit qualification is required to ensure the quality of hits selected for progression.⁵⁰

Although we have shared a particular classification of PAINS filters, we are also making available relevant data for the readers to draw their own conclusions after contrasting with their own experience.

Supplemental Material

Supp_Material_for_Nuisance_Compounds_PAINS_Filters__by_Chakravorty_et_al – Supplemental material for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection

Supplemental material, Supp_Material_for_Nuisance_Compounds_PAINS_Filters__by_Chakravorty_et_al for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection by Subhas J. Chakravorty, James Chan, Marie Nicole Greenwood, Ioana Popa-Burke, Katja S. Remlinger, Stephen D. Pickett, Darren V. S. Green, Martin C. Fillmore, Tony W. Dean, Juan I. Luengo and Ricardo Macarrón in SLAS Discovery

Supplemental Material

Supp_Table_2S_Nuisance_Compounds_PAINS_filters__Chakravorty_et_al – Supplemental material for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection

Supplemental material, Supp_Table_2S_Nuisance_Compounds_PAINS_filters__Chakravorty_et_al for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection by Subhas J. Chakravorty, James Chan, Marie Nicole Greenwood, Ioana Popa-Burke, Katja S. Remlinger, Stephen D. Pickett, Darren V. S. Green, Martin C. Fillmore, Tony W. Dean, Juan I. Luengo and Ricardo Macarrón in SLAS Discovery

Supplemental Material

Supp_Table_3S_Nuisance_Compounds_PAINS_filters__Chakravorty_et_al – Supplemental material for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection

Supplemental material, Supp_Table_3S_Nuisance_Compounds_PAINS_filters__Chakravorty_et_al for Nuisance Compounds, PAINS Filters, and Dark Chemical Matter in the GSK HTS Collection by Subhas J. Chakravorty, James Chan, Marie Nicole Greenwood, Ioana Popa-Burke, Katja S. Remlinger, Stephen D. Pickett, Darren V. S. Green, Martin C. Fillmore, Tony W. Dean, Juan I. Luengo and Ricardo Macarrón in SLAS Discovery

Footnotes

Acknowledgements

The authors acknowledge the contributions of many scientists from the department formerly known as Molecular Discovery Research, which created and curated the screening collection and developed and ran the HTS assays that have made this work possible. We are indebted to colleagues that worked with the authors in the GSK Cleaning the Collection team: Andy Brewster, Ralph Rivero, Katherine Widdowson, Sunny Hung, Pat Brady, Ian Churcher, David Drewry, and Tony Jurewicz. We thank Kelly Locke and Mel Leveridge for supplying ZAP70 results. We also thank Michael A. Walters (University of Minnesota, Department of Medicinal Chemistry) and colleagues for inspiring discussions over the years.

Supplemental material is available online with this article.

Declaration of Conflicting Interests

The authors disclosed the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors were employed by GlaxoSmithKline R&D Pharmaceuticals, and their research and authorship of this article was completed within the scope of their employment with GlaxoSmithKline R&D Pharmaceuticals.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: All authors were employed by GlaxoSmithKline R&D Pharmaceuticals, and their research and authorship of this article was completed within the scope of their employment with GlaxoSmithKline R&D Pharmaceuticals.

References

Macarron

Banks

M. N.

Bojanic

et al . Impact of High-Throughput Screening in Biomedical Research. Nat. Rev. Drug Discov. 2011, 10, 188–195.

Rees

Gribbon

Birmingham

et al . Towards a Hit for Every Target. Nat. Rev. Drug Discov. 2016, 15, 1–2.

Lane

S. J.

Eggleston

D. S.

Brinded

K. A.

et al . Defining and Maintaining a High Quality Screening Collection: The GSK Experience. Drug Discov. Today 2006, 11, 267–272.

Nadin

Hattotuwagama

Churcher

Lead-Oriented Synthesis: A New Opportunity for Synthetic Chemistry. Angew. Chem. Int. Ed. Engl. 2012, 51, 1114–1122.

Harper

Pickett

S. D.

Green

D. V.

Design of a Compound Screening Collection for Use in High Throughput Screening. Comb. Chem. High Throughput Screen. 2004, 7, 63–70.

Drewry

D. H.

Macarron

Enhancements of Screening Collections to Address Areas of Unmet Medical Need: An Industry Perspective. Curr. Opin. Chem. Biol. 2010, 14, 289–298.

Rishton

G. M.

Reactive Compounds and In Vitro False Positives in HTS. Drug Discov. Today 1997, 2, 382–384.

Jadhav

Ferreira

R. S.

Klumpp

et al . Quantitative Analyses of Aggregation, Autofluorescence, and Reactivity Artifacts in a Screen for Inhibitors of a Thiol Protease. J. Med. Chem. 2010, 53, 37–51.

Thorne

Auld

D. S.

Inglese

Apparent Activity in High-Throughput Screening: Origins of Compound-Dependent Assay Interference. Curr. Opin. Chem. Biol. 2010, 14, 315–324.

10.

Baell

J. B.

Holloway

G. A.

New Substructure Filters for Removal of Pan Assay Interference Compounds (PAINS) from Screening Libraries and for Their Exclusion in Bioassays. J. Med. Chem. 2010, 53, 2719–2740.

11.

Baell

J. B.

Screening-Based Translation of Public Research Encounters Painful Problems. ACS Med. Chem. Lett. 2015, 6, 229–234.

12.

Baell

J. B.

Feeling Nature’s PAINS: Natural Products, Natural Product Drugs, and Pan Assay Interference Compounds (PAINS). J. Nat. Prod. 2016, 79, 616–628.

13.

Dahlin

J. L.

Baell

Walters

M. A.

Assay Interference by Chemical Reactivity. In Assay Guidance Manual, Sittampalam

G. S.

Coussens

N. P.

Brimacombe

K., et al., Eds.;

Eli Lilly & Company and the National Center for Advancing Translational Sciences: Bethesda, MD, 2004.

14.

Saubern

Guha

Baell

J. B.

KNIME Workflow to Assess PAINS Filters in SMARTS Format. Comparison of RDKit and Indigo Cheminformatics Libraries. Mol. Inform. 2011, 30, 847–850.

15.

Pearce

B. C.

Sofia

M. J.

Good

A. C.

et al . An Empirical Process for the Design of High-Throughput Screening Deck Filters. J. Chem. Inf. Model. 2006, 46, 1060–1068.

16.

Bruns

R. F.

Watson

I. A.

Rules for Identifying Potentially Reactive or Promiscuous Compounds. J. Med. Chem. 2012, 55, 9763–9772.

17.

Bento

A. P.

Gaulton

Hersey

et al . The ChEMBL Bioactivity Database: An Update. Nucleic Acids Res. 2014, 42 (Database issue), D1083–D1090.

18.

Sterling

Irwin

J. J.

ZINC 15—Ligand Discovery for Everyone. J. Chem. Inf. Model. 2015, 55, 2324–2337.

19.

Leach

A. R.

Bradshaw

Green

D. V.

et al . Implementation of a System for Reagent Selection and Library Enumeration, Profiling, and Design. J. Chem. Inf. Comput. Sci. 1999, 39, 1161–1172.

20.

Charifson

P. S.

Walters

W. P.

Filtering Databases and Chemical Libraries. J. Comput. Aided Mol. Des. 2002, 16, 311–323.

21.

Hann

Hudson

Lewell

et al . Strategic Pooling of Compounds for High-Throughput Screening. J. Chem. Inf. Comput. Sci. 1999, 39, 897–902.

22.

Aldrich

Bertozzi

Georg

G. I.

et al . The Ecstasy and Agony of Assay Interference Compounds. ACS Cent. Sci. 2017, 3, 143–147.

23.

Capuzzi

S. J.

Muratov

E. N.

Tropsha

Phantom PAINS: Problems with the Utility of Alerts for Pan-Assay Interference Compounds. J. Chem. Inf. Model. 2017, 57, 417–427.

24.

Jasial

Bajorath

How Frequently Are Pan-Assay Interference Compounds Active? Large-Scale Analysis of Screening Data Reveals Diverse Activity Profiles, Low Global Hit Frequency, and Many Consistently Inactive Compounds. J. Med. Chem. 2017, 60, 3879–3886.

25.

Dahlin

J. L.

Nissink

J. W.

Strasser

J. M.

et al . PAINS in the Assay: Chemical Mechanisms of Assay Interference and Promiscuous Enzymatic Inhibition Observed during a Sulfhydryl-Scavenging HTS. J. Med. Chem. 2015, 58, 2091–2113.

26.

Dahlin

J. L.

Walters

M. A.

How to Triage PAINS-Full Research. Assay Drug Dev. Technol. 2016, 14, 168–174.

27.

Gao

Margolis

B. J.

Strelow

J. M.

et al . Beyond PAINs: Chemotype Sensitivity of Protein Methyltransferases in Screens. ACS Med. Chem. Lett. 2016, 7, 156–161.

28.

Lagorce

Oliveira

Miteva

M. A.

et al . Pan-Assay Interference Compounds (PAINS) That May Not Be Too Painful for Chemical Biology Projects. Drug Discov. Today 2017, 22, 1131–1133.

29.

Pouliot

Jeanmart

Pan Assay Interference Compounds (PAINS) and Other Promiscuous Compounds in Antifungal Research. J. Med. Chem. 2016, 59, 497–503.

30.

Whitty

Growing PAINS in Academic Drug Discovery. Future Med. Chem. 2011, 3, 797–801.

31.

Gamo

F. J.

Sanz

L. M.

Vidal

et al . Thousands of Chemical Starting Points for Antimalarial Lead Identification. Nature 2010, 465, 305–310.

32.

Green

D. V.

Pickett

S. D.

Methods for Library Design and Optimisation. Mini Rev. Med. Chem. 2004, 4, 1067–1076.

33.

Coma

Herranz

Martin

Statistics and Decision Making in High-Throughput Screening. Methods Mol. Biol. 2009, 565, 69–106.

34.

Wassermann

A. M.

Lounkine

Hoepfner

et al . Dark Chemical Matter as a Promising Starting Point for Drug Lead Discovery. Nat. Chem. Biol. 2015, 11, 958–966.

35.

Ekins

Williams

A. J.

Finding Promiscuous Old Drugs for New Uses. Pharm. Res. 2011, 28, 1785–1791.

36.

Langman

L. J.

Kapur

B. M.

Toxicology: Then and Now. Clin. Biochem. 2006, 39, 498–510.

37.

Harper

Bravi

G. S.

Pickett

S. D.

et al . The Reduced Graph Descriptor in Virtual Screening and Data-Driven Clustering of High-Throughput Screening Data. J. Chem. Inf. Comput. Sci. 2004, 44, 2145–2156.

38.

Hussain

Harper

Blaxill

et al . Use of Data Mining to Help Identify Compounds That Are Unstable in DMSO. Presented at 8th International Conference on Chemical Structures, Noordwijkerout, Netherlands, 2008.

39.

Blaxill

Holland-Crimmin

Lifely

Stability through the Ages: The GSK Experience. J. Biomol. Screen. 2009, 14, 547–556.

40.

Bemis

G. W.

Murcko

M. A.

The Properties of Known Drugs. 1. Molecular Frameworks. J. Med. Chem. 1996, 39, 2887–2893.

41.

Lewell

X. Q.

Judd

D. B.

Watson

S. P.

et al . RECAP—Retrosynthetic Combinatorial Analysis Procedure: A Powerful New Technique for Identifying Privileged Molecular Fragments with Useful Applications in Combinatorial Chemistry. J. Chem. Inf. Comput. Sci. 1998, 38, 511–522.

42.

Tricker

A. R.

Preussmann

Carcinogenic N-Nitrosamines in the Diet: Occurrence, Formation, Mechanisms and Carcinogenic Potential. Mutat. Res. Genet. Toxicol. Test. 1991, 259, 277–289.

43.

Lipinski

C. A.

Lombardo

Dominy

B. W.

et al . Experimental and Computational Approaches to Estimate Solubility and Permeability in Drug Discovery and Development Settings. Adv. Drug Deliv. Rev. 2001, 46, 3–26.

44.

Leeson

P. D.

Young

R. J.

Molecular Property Design: Does Everyone Get It?

ACS Med. Chem. Lett. 2015, 6, 722–725.

45.

Dahlin

J. L.

Walters

M. A.

The Essential Roles of Chemistry in High-Throughput Screening Triage. Future Med. Chem. 2014, 6, 1265–1290.

46.

Zuegg

Cooper

M. A.

Drug-likeness and increased hydrophobicity of commercially available compound libraries for drug screening. Curr. Top. Med. Chem. 2012, 12, 1500–1513.

47.

Cox

P. B.

Gregg

R. J.

Vasudevan

Abbott Physicochemical Tiering (APT)–a unified approach to HTS triage. Bioorg. Med. Chem. 2012, 20, 4564–4573.

48.

Stewart

Drexler

D. M.

Leet

J. E.

et al . Labware Additives Identified to Be Selective Monoamine Oxidase-B Inhibitors. J. Biomol. Screen. 2014, 19, 1409–1414.

49.

Nissink

J. W.

Blackburn

Quantification of Frequent-Hitter Behavior Based on Historical High-Throughput Screening Data. Future Med. Chem. 2014, 6, 1113–1126.

50.

Kenny

P. W.

Comment on the Ecstasy and Agony of Assay Interference Compounds. J. Chem. Inf. Model. 2017, 57, 2640–2645.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.46 MB

0.09 MB

0.03 MB