Quality Issues of Research Antibodies

Abstract

According to several recent studies, an unexpectedly high number of landmark papers seem to be not reproducible by independent laboratories. Nontherapeutic antibodies used for research, diagnostic, food analytical, environmental, and other purposes play a significant role in this matter. Although some papers have been published offering suggestions to improve the situation, they do not seem to be comprehensive enough to cover the full complexity of this issue. In addition, no obvious improvements could be noticed in the field as yet. This article tries to consolidate the remarkable variety of conclusions and suggested activities into a more coherent conception. It is concluded that funding agencies and journal publishers need to take first and immediate measures to resolve these problems and lead the way to a more sustainable way of bioanalytical research, on which all can rely with confidence.

Introduction

Reproducibility of scientific studies has been repeatedly put into question. ^{1 8} International companies, such as Amgen and Bayer HealthCare, were not able to reproduce most of the examined landmark papers in their field.^9,10 Of the research expenses presumably wasted in the USA, by far the largest fraction of about 36% is assigned to biological reagents, ⁸ which amounts to losses of about 10 billion US$ annually. Worryingly, there seems to be no correlation between the number of citations or the impact factor of the respective journal and the reproducibility of a scientific study. In many negative examples discussed, antibodies play a significant role. Although first discussions have been sparked at least 20 years ago, ^{11 13} the efforts to improve this situation have not yet led to fundamental advancements. A new series of articles focusing on quality control (QC) of antibodies showed the persistent interest and relevance of this problem. ^{14 16} In a recent paper, ¹⁷ it was estimated that more than 300 companies offer more than 2 million antibodies for research. In a comment to this article, it was disclosed that a large bioinformatics company independently tested more than 6,000 commercial antibodies from 26 suppliers. More than 75% of these antibodies were nonspecific or did not work at all. Furthermore, the consortium Human Protein Atlas examined more than 5,000 commercial antibodies, ¹⁸ from which more than 50% could not be used in the anticipated application.

This article tries to offer a systematic analysis of the problems involved and makes an effort to combine most of the suggestions and discussions into a more comprehensive and prioritized list of proposed actions. This should improve the quality of research and applications conducted with antibody reagents considerably and lead to a more sustainable model of research antibody development and production.

The Status Quo

1. 1.

   Development and production of antibodies

   This article will consider antibodies of many sources, polyclonal antibodies from blood serum, monoclonal antibodies produced by the hybridoma technology of Kohler and Milstein, ¹⁹ and also recombinant antibodies. It seems to be obvious that at least basic information about the antibody production method should be supplied by an antibody manufacturer or reseller. Fortunately, this is achieved in most cases. In the case of hapten or peptide antibodies, a special problem occurs quite frequently. Although the manufacturer should have complete information about the hapten, the linker, the immunogen, and the carrier protein, many companies declare this information as proprietary. In the field of research antibodies, the use of antibodies of unclear genesis is severely limited, since the synthesis of suitable enzyme conjugates and other immunoreagents can only be made based on a detailed knowledge of the structure of the antigen. Particularly, if the manufacturer of the antibody does not offer any compatible conjugates or standards, it seems to be irrational to withhold this information. In the case of protein antibodies, the information is often not disclosed, whether the antibody was made against a native, denatured, or fragmented antigen.

2. 2.

   Traceability of antibodies

   The unambiguous identification of antibodies is of utmost importance. According to a recent publication, ²⁰ only 44% of all antibodies mentioned in publications can be identified at all. This fraction also does not correlate with the impact factor of the journal. If a proper identification label cannot be assigned to an antibody, most of the antibody characterization has to be performed by each user. Otherwise, the properties of an antibody remain unknown to the respective user. Surprisingly, this status is nearly the rule and not the exception for many publications. In a strict sense, these papers should not be considered to be a part of science, since the respective experiments cannot be reproduced independently. In the end, these publications need to be invalidated.

   Annoyingly, many antibody resellers assign new clone or other arbitrary numbers to well-known antibodies to obscure their sources. To make it even worse, some antibody companies sell different antibodies (preparations) under the same order number, which is often the case with polyclonal antibodies. Moreover, many researchers spend very little effort to assign the genuine clone numbers to their used antibodies and simply cite an order number of a reseller—even if the company laudably cites original publications and clone designations. If the order number changes or the company goes out of business, this assignment is lost. The lack of proper antibody identification is a major source of quality problems, since characterization data cannot be consolidated or updated.

3. 3.

   Concentration and activity of antibodies

   The aging of an antibody is highly dependent on the transport and storage conditions, and hence, this aspect is difficult to control. Usually, the manufacturer states a protein or IgG concentration on the label or data sheet. However, this is only a substitute for the relevant information, the concentration of active antibody. For poly-clonal antibodies, the determination of a concentration is particularly difficult, since even the IgG content is nearly irrelevant in this context. Antigen-affinity-purified antibodies also show this problem, since it is not guaranteed that all antibodies contained in such a preparation have the same or even any activity in a specific assay or application. Hence, the current concentration or activity of the reagent might be not as high as expected. The precision and accuracy of spectrophotometric protein determination methods are poor anyway. Most of these methods should be considered as semiquantitative. Only by relatively laborious methods, such as amino acid analysis or isotope dilution mass spectrometry with the respective tryptic peptides, a quantitative determination of the antibody concentration might be achieved. Unfortunately, these methods are not showing any difference between active and denatured antibodies and may be sensitive to impurities. In addition, they are rarely used for research antibodies due to the relatively high cost and effort involved.

4. 4.

   Affinity constants

   The most prominent property of an antibody is its ability to form antibody-antigen complexes. This reaction can be described by simple physicochemical rules, such as the law of mass action. Today, there are many different techniques available to determine the affinity (or equilibrium) constant of an antibody-antigen reaction. Most popular seems to be the surface-plasmon resonance technique, ²¹ which is also known as Biacore method. However, other techniques, such as enzyme-linked immunosorbent assay (ELISA),^22,23 surface acoustic wave sensors, ²⁴ equilibrium dialysis, and others ²⁵ are used for this purpose. Presupposed that the measuring conditions are defined, affinity constants can be considered to be a stable characteristic and valid essentially forever. Therefore, it is a pity that the determination of affinity constants is rarely performed for antibodies and their respective antigens. An affinity constant is a perfect property of an antibody-antigen pair and has an outstanding influence on nearly all applications. At the moment, only a very small fraction of antibodies are sold with affinity data.

5. 5.

   Cross-reactivities or selectivity

   Several definitions of cross-reactivities of antibodies have been proposed, from which only one gained broad popularity, first published by Abraham.^26,27 In many applications, 100% selectivity (specificity) ^{28 32} or broad group selectivity^27,33 is preferred. However, due to the lock-and-key mechanism of the antibody-antigen interaction, unwanted and even completely unexpected cross-reactivities (interactions of more or less unrelated compounds) are always possible. ³⁴ The practical question is only, whether the respective cross-reactivity is relevant for the specific analytical application or not. For instance, a synthetic cross-reactant, which is not occurring in any real samples, is obviously irrelevant in practice. ³⁵ One of the major limitations of cross-reactivity studies today is their arbitrary selection of the cross-reactants tested. Considering the millions of known chemical compounds or billions of proteins, which may be present in complex samples, it is obvious that only a very small fraction of them can be tested separately. Today, many commercial antibodies are sold with no or only embarrassingly poor cross-reactivity data. Quite often, “Cross-reactivity: 100% Analyte X” is given as the sole information, which is equivalent to “Cross-reactivity not tested.” For antibodies against proteins, the knowledge of the bound epitope would be very helpful. However, this information is nearly never available. This might be due to the difficulty and cost of such an epitope scan. Nevertheless, an antibody against a protein, where the bound epitope is unknown, has to be considered to be only partially characterized. A similar, but even more critical, problem is the identification of matched pairs of antibodies, ³⁶ suitable for the setup of a sandwich immunoassay, which belongs to the noncompetitive assays. For such a matching pair, you need two antibodies, which bind two different epitopes without steric hindrance. For immunohistochemical applications, usually no chemically defined (and verified) antigens are available, and therefore, cross-reactivities in the usual sense cannot be determined. The similar situation might apply for Western blots, where the occurrence of a single band with the approximately adequate molecular weight is often seen as an adequate QC. The selection of known test samples (positive and negative controls) is considered to be a good approach. However, cells or tissues have to be assigned to a type or condition by some other means, eg, the assessment of one or several pathologist(s). In difficult cases, this assessment might differ significantly and hence make the definition of standard samples uncertain. In addition, standard or reference samples of tissue are only available in limited numbers, which makes it impossible to guarantee long-term availability. A new batch of the same tissue might be different in some hidden properties and hence might behave different with some antibodies. Selectivity of antibodies in biological samples is also tested by RNA interference, which should inhibit the gene expression of the respective antigen. ³⁷ Particularly, siRNA-based (small-interfering RNA) assays are often used for antibody validation. Experiments with knockout animals frequently lead to withdrawals of papers, since an antibody should not detect a nonexisting target. Even some sort of inhibition test or preadsorption experiments with a peptide antigen might not be sufficient to support the claimed selectivity of an antibody. ¹² For environmental or other inherently complex and unpredictable samples, the use of immunograms (HPLC-ELISA and LC-ELISA) might be helpful. ³⁸ They were used to identify unexpected cross-reactants, eg, in wastewater samples, ³⁴ in endocrinological studies,^39,40 or in phytochemistry. ⁴¹

6. 6.

   Application tests

   Many antibodies at least carry some information, such as ELISA, WB, and IHC, which are the abbreviations of some major immunochemical techniques. However, only in some cases, more detailed protocols are given, and unfortunately, some experience in a specific assay can hardly be transferred to another, which means that such an application statement is only of preliminary relevance. Often, applications are the only relevant information about a commercial antibody (preparation) available.

7. 7.

   Reference materials and standards

   Sometimes the manufacturer offers a positive control that might be a sample of the respective antigen. This can be very helpful for activity tests and assay development. However, often, information about this positive control is also scarce. Nearly all problems that apply to the antibodies themselves also apply to the standards. Certified reference materials from organizations, such as NIST (National Institute of Standards and Technology), BAM (Bundesanstalt für Materialforschung und -prüfung), or others, are nearly never available, mainly due to stability and diversity issues. Other reference materials of any quality level might also be difficult to obtain. Considering the nearly complete lack of these materials (in relation to the 2 million commercial antibodies), it seems to be not realistic to hope for significant improvements in the near future. However, materials for RNA interference might be provided for control purposes for some assay formats.

8. 8.

   Long-term availability

   After some years, most of the published antibodies are not available anymore. Considering the significant investment of time and money in the development of the respective antibodies, this seems to be an unjustifiable waste of resources and knowledge. Surprisingly, the continuous loss of antibodies of unique structure and properties, which cannot be regained, is widely ignored. In this context, the recent suggestion to prefer recom-binant antibodies ⁴² makes much sense, since in the case of a published sequence of a recombinant (or monoclonal) antibody, even the complete loss of a clone could be reversed by DNA synthesis. A few clones are stored in long-term depositories, preferentially in different aliquots at different locations. Unfortunately, due to financial limitations, insufficient risk assessment, and considerations about intellectual property, adequate safeguards are often not taken, which finally leads to the deplorable situation today. A risky situation is the dependency of researchers and routine analytical chemists from one antibody supplier. Sometimes, polyclonal antibodies or test kits of commercial suppliers are exchanged without any notice, which might lead to panic in the affected analytical laboratories, when they discover an unexplainable deviation. An acquisition of one or several mass spectrometer(s) and the abolition of immunochemical techniques might be the final result. For researchers, the consequences of an uncertain or even halted antibody supply also might be grave. Long-term research projects definitely need long-term assurance of antibody access of a constant quality. Even when the company openly states that the product is not available any more, it remains a difficult situation for the research projects affected.

9. 9.

   Antibodies as a subject in publications

   The final product of research is often a publication, which disseminates the knowledge described in the respective paper. Therefore, the publication system in its present form may have some influence on the antibody quality issue. Several flaws of traditional publication pathways, which have been identified already, question not only the reliability of scientific work as a whole but also the quality of antibodies described in these papers. As mentioned above, quite a few criteria are necessary to validate an antibody properly. Even in highly reputable journals and in highly cited papers, the description, characterization, and validation of research antibodies are nearly always insufficient. In addition, the generation, characterization, and validation of antibodies are widely disrespected, and hence even difficult to publish. This is in a striking contrast to the ease to publish work describing novel methods based on nondefined antibodies and irre-producible immunochemical protocols. There is not a single journal in the market, which is focused on antibody development and validation, which is definitely surprising considering the thousands of established journals today.

10. 10.

    Antibodies as a commercial product

    The development of antibodies is an expensive endeavor. However, in a commercial environment, there needs to be a return of investment in a manageable time frame. Recently, an interesting paper with the title “The Antibody Dilemma” was published, ⁴³ discussing the issue from a traditional antibody manufacturer's point of view. The authors sum up: “Antibody haste, research waste”. This seems to be caused by the manufacturers and resellers taking some shortcuts to market and by the users, who see antibodies as a convenience product, without considering any limitations and assuming no responsibility, eg, for the use of suitable negative and positive controls.

    At this point, it is important to differentiate between research antibodies for experimental diagnostic, environmental, food analytical or other purposes, and therapeutic antibodies, which are finally sold in a pharmacy. Therapeutic antibodies may generate sales in the range of billions (10 ⁹ ) of US dollars. Research antibodies sometimes do not even compensate for their development costs in the range of some thousands (10 ⁴ ) of US dollars. This is about 10 ⁵ times less! Therefore, it is no problem to perform nearly any imaginable kind of characterization and a continuous in-depth QC of therapeutic antibodies. This difference is also important when deciding how the antibodies are generated. As a rough rule of thumb, a polyclonal antibody may cost about 1,000 US$, a monoclonal one about 10,000 US$, and a recombinant antibody up to 50,000 US$, including some more sophisticated affinity maturation. If these costs are compared with the annual sales of a research antibody, it is clear why most of the commercial research antibodies are still polyclonal ones. Even a monoclonal antibody might never reach the break-even point for a research application. However, some companies acknowledge the issue and have started to implement their own quality initiatives. One company claims to have discarded about one-third of its catalog after a more thorough quality check. ¹⁷

11. 11.

    Academic sources

    Many primary antibodies have been developed by academic groups, which often try to commercialize the antibodies after the end of the project. This seems to be a good approach, since many antibodies of academic groups are of high quality and are sufficiently characterized to be considered fit for purpose. However, excellent antibodies are often sold to test kit manufacturers, which do not want to see the same antibody sold freely in the market or being accessible to anybody else. Therefore, these antibodies are not available from any commercial source. All other antibodies (or clones) of doubtful quality, which could not be used for a test kit, may be offered to antibody resellers with only minimal information about their performance. This is a negative selection process that minimizes the chance for a buyer to get a good antibody in the free market. The next problem is that these academic researchers are not willing or able to give any antibody samples to other researchers anymore, since they sold them already or their Office of Intellectual Property Administration prevents this. To make the situation even worse, this transfer often prevents the antibodies or clones being deposited in any repository (such as ATCC, ECACC, DSMZ, or others). Often this is the end of the story, after the retirement of the head of department, the merger or bankruptcy of the startup company commissioned with the marketing of the antibody; the clones are often lost forever. Not only the antibody is lost then, but also all efforts to characterize them are nullified at this moment.

This list of problem areas may not be exhaustive. It should be acknowledged that these problems persist for decades and are definitely not easy to resolve. Some researchers may even deny that they are problems at all and that research came along quite well without any further measures. The following suggestions for an antibody quality initiative should be seen as a basis for further discussions and as a resource for all, who have to do with antibodies or antibody research in some way. There are several reasons, why urgent action is required. First, the reproducibility crisis of science, in general, reached an inacceptable level. Second, the huge amount of waste of time and financial resources is not acceptable in a purely economic sense. Third, the use of antibodies of doubtful quality might harm patients and other people dependent on reliable results of diagnostic and other analytical tests. Fourth, bad antibodies and immunoassays damage the reputation of a whole analytical field and on the long term destroys the economic basis of many companies, since the irreproducibility of antibody reagents leads to evasive reactions, for example, the changeover to mass spectrometric techniques in clinical laboratories, which hope to get more reliable results then. ^{44 48} And finally, bad antibodies also cause a lot of frustration, which makes the work with antibodies a daunting experience for many scientists.

What Could be Done?

1. 1.

   Full antibody traceability needs to be guaranteed in all aspects of antibody work. A unique Antibody ID should be introduced, similar to the CAS # for chemicals, DOI for publications, or Researcher ID for scientists. Antibodies without an assigned Antibody ID should not be used anymore for research work. Today, less than 50% of all antibodies mentioned in publications are identifiable at all. ²⁰ An online Antibody Registry was started recently. Unfortunately, since the operators seem to rely on vendor information, the same antibody sold by different vendors might get several ID numbers, causing inac-ceptable duplicates and inconsistent data quality. As an immediate, but preliminary approach, the original clone number of the developers should be used.

2. 2.

   An Antibody Heritage Program should be started mainly based on all publicly funded research. As a first step, all hybridoma clones of monoclonal antibodies, which have been characterized on a defined minimum level and/or have been published, should be deposited by a nonprofit organization, such as the ATCC, which already holds a collection of 1,200 hybridomas. However, the commercialization of publicly funded antibodies might be an issue. If commercialization should be possible as today and not limited to a nonexclusive licensing model, other researchers, who are interested in a clone/antibody, might be hindered to access and use this reagent. Then, a new antibody may have to be developed by spending tax money twice. Antibodies financed by private funds are expected to generate higher revenues and profits anyway and may not show many of the problems discussed in the first part of the article. However, it might be an advantage for a company to make their antibodies more amenable for the QC of the users and avoid disappointment or bad reputation. As discussed in the article by Ascoli and Birabaharan, ⁴³ formerly, there was a symbiosis between the user of an antibody and the antibody manufacturer. Today, two extreme variants might be conceivable: Any company would be expected to perform strict and extensive QCs on any antibody they manufacture or sell. The user might use this reagent with blind confidence and without any deeper knowledge of the field. This would limit the number of targets to very few, which are commercially interesting. In addition, these antibodies would be limited to verified protocols of defined applications. Most test kits work this way. On the other hand, antibody resellers or developers might offer cheap, non-validated antibodies and indicate this fact clearly in their documents. An interesting way, which is already offered by some vendors are sets of microsamples of antibodies offered at a low price, which could be tested by the user in a specific application.

3. 3.

   Training of antibody users should be offered by biochemical, clinical, and biological societies and perhaps even by commercial antibody developers and resellers to regain knowledge, which was lost during the past few decades. Many researchers are simply not familiar with the way antibodies should be used. They are also not aware of the complexity of antibody reagents, which leads to the already mentioned carelessness and lack of acknowledgment for the development of high-quality antibodies assuming that an antibody is a biochemical standard reagent. This is particularly obvious in the biosensor field, where too many researchers do not recognize that the selectivity and, often also, the sensitivity of their biosensors are not primarily governed by their sensor platform or transducer, but by the individual antibody reagent used. ⁴⁹

4. 4.

   Funding agencies need to establish antibody quality workflows for all grant applications, which mention antibodies in an experimental context. First, at least one of the applicants should be sufficiently experienced in antibody applications. Second, the hybridoma clones funded by the project should be deposited in a nonprofit collection as early as possible. This needs to be mandatory. Third, to any antibody clone produced in these projects, an Antibody ID should be assigned. Fourth, unaware applicants should be referred to existing antibodies to avoid unnecessary repetition of work. Fifth, funding agencies should not encourage the use of poor reagents, which means to take cheap, but nonreproducible shortcuts. This program might save a lot of money for the tax payer, since antibodies will not be produced and lost in many costly cycles. Funding agencies are, similar to scientific journals, strong influencers in any field they support. They carry a fundamental responsibility for the work they fund. He who pays the piper calls the tune.

5. 5.

   Publications are of extreme importance for nearly all publicly funded researchers. Therefore, the demands of journal editors and referees are the ultimate reason of many decisions of scientists. Due to their direct influence on the scientific workflows, some significant improvements might be achieved by publishers and editors. This approach would be very cost efficient and could be introduced on short notice. All journals that publish work containing antibody experiments should establish strict rules, ie, how protocols and results should be reported. Sometimes the antibodies are seen as proprietary information from some party involved. In cases where the reagents are not disclosed, the respective work must not be published in scientific journals, because they cannot be reproduced independently. In addition, without this rule, the floodgates to fraud and exaggeration are opened without any risk for dishonest researchers and sloppy journals. The second rule that should be enforced is the inclusion of positive and negative controls in any antibody experiment. In most journals, novelty is the highest ranked criterion for acceptance. But what means novelty, if the experiment cannot be repeated or not even interpreted properly? The third rule should be that antibody work needs to be assessed by competent reviewers. Only scientists with long-term experience in the field can recognize that doubtful reagents might have been used in critical steps. Ideally, dedicated referees should supervise these sections to achieve a consistent quality level. In general, if nearly all journals would systematically refuse to publish irrepro-ducible antibody work based on noncharacterized, non-disclosed, and nondefined reagents, the situation would improve instantly. Finally, the lack of journals dedicated to antibody production, characterization, and validation is obvious. Publishers are not to blame here; they would definitely take any commercial chance they see. However, it seems to be a poor appreciation of the process of the development of excellent research antibodies. It is seen as a routine task, but not as a significant scientific accomplishment. Recently, at least a small antibody validation section was introduced in a new open access journal. ⁵⁰

6. 6.

   Sequence information of antibodies is a powerful way to avoid antibody losses ⁴² and guarantees the essentially indefinite availability of an antibody as long as the information of the sequence does not get lost and the published sequence is correct, which also cannot be taken for granted. The recombinant way to generate antibodies is an attractive approach in this context, since the sequence of any recombinant antibody is accessible very easily. However, the recombinant technology still failed to deliver a cost-effective way to make reagent antibodies, which can be easily seen in any price list of recombinant proteins. It has to be stressed that the knowledge of a sequence is good to identify and secure an antibody on the long term. ⁵¹ But antibody sequencing is no validation of the analytical performance.^52,53 It seems that antibody sequences (monoclonal and recombinant) are a safe and economic way to keep antibodies available and enable an unquestionable assignment to an Antibody ID.

7. 7.

   Make your own antibodies. This is not the worst option, because the antibodies can be optimally streamlined for the intended application. However, this is a relatively expensive approach and needs a lot of experience and time. An important advantage is the reliable supply of antibodies, which cannot be guaranteed by many resellers. In the study by Baker, ¹⁷ it was described that severe consequences can occur, if access to your former antibody stops.

8. 8.

   Validate each antibody you get in the laboratory. Do not believe the statements on the datasheet. ¹⁷ It is necessary to do the right test before any new antibody is used for research. This advice is not popular and will be ignored frequently. This is partially due to the well-known publication pressure. Therefore, many scientists think that they need to take a short cut or simply have no time for additional QC. All researchers with a longer experience with antibodies know that this attitude will take revenge.

9. 9.

   Share your knowledge. In between, several antibody suppliers and other more independent platforms offer the opportunity to rate antibodies. In general, this is a good approach to exchange valuable information, which was nearly impossible or at least extremely limited before. Unfortunately, the number of comments and ratings is still very low. Therefore, it is unclear today whether this approach is leading to a general improvement in the field.

Conclusions

Two measures seem to be most promising, to get an immediate turnaround: funding agencies need to install antibody quality regulations, which could be put into place on short term. And finally, all relevant publishers should impose clear and non-negotiable rules for the documentation of antibody experiments.

Author Contributions

Conceived the concepts: MGW. Wrote the first draft of the manuscript: MGW. Developed the structure and arguments for the paper: MGW. Made critical revisions: MGW. The author reviewed and approved of the final manuscript.