PerspectiveImmunotherapy

Harmonization of Immune Biomarker Assays for Clinical Studies

See allHide authors and affiliations

Science Translational Medicine  09 Nov 2011:
Vol. 3, Issue 108, pp. 108ps44
DOI: 10.1126/scitranslmed.3002785

Abstract

Assays that measure a patient’s immune response play an increasingly important role in the development of immunotherapies. The inherent complexity of these assays and independent protocol development between laboratories result in high data variability and poor reproducibility. Quality control through harmonization—based on integration of laboratory-specific protocols with standard operating procedures and assay performance benchmarks—is one way to overcome these limitations. Harmonization guidelines can be widely implemented to address assay performance variables. This process enables objective interpretation and comparison of data across clinical trial sites and also facilitates the identification of relevant immune biomarkers, guiding the development of new therapies.

SEARCHING FOR HARMONY

Immune cells play a major role in tumor growth control and modulation of antitumor immunity. Specifically, T cells are considered to be central to the induction of a productive antitumor immune response (1, 2). Several strategies are being pursued to treat cancer in humans through manipulation of the immune system, generally recognized as “immunotherapy.” Such efforts range from the use of vaccine and adjuvant combinations (3, 4) to adoptive transfer of ex vivo–manipulated T cells (57) and are investigated either alone or possibly combined with standard radio- or chemotherapy (8). In the past, cancer immunotherapeutic development was based on methods created for chemotherapy, which did not account for some unique characteristics of immunotherapy and therefore hampered both therapeutic and biomarker development. Recently, a new methodological framework for the development of cancer immunotherapies was created (9), which improved upon existing immunotherapy study designs (10) and provided suggestions for new clinical end points and antitumor response criteria (11). The approach also introduced scientific exchange and regulatory interactions to generally inform regulatory authorities for developing guidance documents (12). Further, a publication framework was provided for immune monitoring results (13). A part of these methodological improvements was the creation of harmonization guidelines for immunoassays to support immune biomarker development (1416). With recent advances in immunotherapy, clinical trials that use immune-modulating agents are beginning to demonstrate concrete patient benefit, including prolonged survival (1720). On the basis of these clinical successes and methodological improvements, it is likely that the number and diversity of innovative immunotherapy-based strategies will continue to grow. In this Perspective, we define the benefits of assay harmonization for immune biomarker development, raise awareness about how to access adequate harmonization programs, and suggest directions for its wider implementation in the growing immunotherapy space.

Unlike most anticancer treatments, immune-based therapies primarily target the immune system and not the tumor. Therefore, an essential component of immunotherapy trials is the study of the patient’s immune response to such treatment. This analysis is typically accomplished through various laboratory assays to measure differences in the number and activity of specific immune parameters before, during, and after treatment. This may allow the identification of immune signatures as measures of biological activity, which correlate with treatment efficacy or toxicity and may play a role in selecting patient populations and establishing surrogate end points for clinical efficacy (21, 22). Although the scientific technologies currently applied to measure immune biomarkers are straightforward, the underlying assay protocols and data-reporting procedures are heterogeneous and complicate the comparison and interpretation of the results across laboratories. To date, no reliable immunological biomarkers have been identified that can be correlated with clinical outcomes of therapy (15).

Approaches to overcome these limitations have involved the standardization of assays through the use of central facilities or the establishment of assay-specific protocols in individual laboratories. Standardization of biomarker assays enforces identical reagents and/or protocols across laboratories and consortia. This concept has been employed successfully in late-stage trials and is a common mechanism for conducting multi-institutional studies funded by a single source. However, it is impractical to implement in early clinical trials when newer and lesser-established immune assays are often applied and when broader implementation is premature or prohibitively expensive (Table 1). The inherent heterogeneity and variability associated with essentially every aspect of individual laboratory-specific protocols (e.g., reagents, instrumentation, operators, and training) precludes an objective comparison of data across sites.

Table 1.

Advantages and disadvantages of assay standardization versus assay harmonization for early immune biomarker studies

View this table:

An alternative approach that facilitates the comparability and integration of data across multiple laboratories is “assay harmonization” (15). It provides distinct advantages over standardization, particularly in early clinical trials when biomarker assays are still exploratory and of questionable relevance to clinical utility (Table 1). The harmonization process involves the participation of multiple laboratories in a consortium-based iterative testing process to identify the variables crucial for assay performance. Essential components are assay proficiency panels in which individual laboratories participate to perform parallel quality control experiments on replicate samples. Each participating laboratory uses its own reagents, instrumentation, and protocols. A central laboratory manages logistics for the proficiency panel, receives raw and analyzed data sets from each participating laboratory, and provides independent central data analysis. During initial proficiency panels, variables that impact laboratory performance are identified. Subsequent panels harmonize use of performance-stabilizing variables across laboratories (Fig. 1). The outcome of this iterative process is a set of harmonization guidelines, which can be implemented both by the panel participants and by the larger scientific community. Harmonization can be applied to all assay variables from the protocol and raw data acquisition, processing, interpretation, and reporting, so that comparison of data between laboratories becomes objective and supportive of immune biomarker research.

Fig. 1.

Harmonization plays an important role as a virtual centralized facility. The process enhances the validity of data generated by participating laboratories, increases clarity in data reporting, benchmarks the performance of participants, and provides guidance toward clinical assay validation.

CREDIT: B. STRAUCH/SCIENCE TRANSLATIONAL MEDICINE

PROFICIENCY PANELS: MODELS FOR SUCCESS

Since 2005, two consortia have conducted the largest proficiency panel experiments to address assay harmonization: the Cancer Immunotherapy Consortium (CIC), formerly the Cancer Vaccine Consortium, of the Cancer Research Institute (CRI) in the United States and the Association for Cancer Immunotherapy (CIMT) Immunoguiding Program (CIP), a working group in Europe. Each consortium includes 40 to 80 academic, pharmaceutical, biotechnology, contract research, and government laboratories and has used the multistep approach described above. Both have sponsored proficiency panels in enzyme-linked immunospot (ELISPOT), intracellular cytokine staining (ICS), and human leukocyte antigen (HLA)–multimer staining (14, 16, 23, 24). Furthermore, the CIC and CIP actively pursue the advancement of cellular reference standards, the response definition criteria for ELISPOT (25), the role of serum in T cell assays (26, 27), and gating strategies in flow cytometry–based assays (28). Networks and consortia from other fields, such as the National Institute for Allergy and Infectious Diseases, also collaborate on the harmonization panels.

As an extension of assay harmonization, the CIC and CIP, together with the Stanford University Human Immune Monitoring Center, have proposed the MIATA (Minimum Information About T cell Assays) initiative. The objective of MIATA is to establish minimum criteria for reporting immune-monitoring experiments and results in order to facilitate more objective data interpretation and meta-analysis (13, 29). Insights from these large proficiency panels indeed show variability in assay performance and data and provide support for assay harmonization as a tool to address these challenges. Initial proficiency panels showed significant interlaboratory variability in ELISPOT, HLA-multimer, and ICS assay performance (16, 23); however, implementation of harmonization guidelines resulted in substantial reduction in performance variability of the ELISPOT and HLA-multimer assays, which was further reduced with the level of implementation (16, 28) (Fig. 2). Elements supporting superior assay performance include quality-enabling laboratory infrastructure, specifically well-trained and qualified personnel, the use of defined standard operating procedures (SOPs), and the use of calibrated and maintained instrumentation (14, 16).

Fig. 2.

The effect of harmonization guideline implementation on the ability of laboratories to detect low-frequency T cell responses with the ELISPOT assay. This figure presents a summary of data from successive proficiency panel experiments where participating laboratories measured antiviral immune responses in the same donor sample, followed by central evaluation of the ability of individual laboratories to detect predefined responses. Several rounds of proficiency panels were conducted in which each round improved critical assay performance variables identified in the previous round (16). The number of participating laboratories at each stage is indicated in parentheses next to the numerical value. Overall, these data demonstrate that harmonization of the ELISPOT assay through implementation of guidelines resulted in a reduction in the inability of participating laboratories to detect low-frequency responses.

CREDIT: B. STRAUCH/SCIENCE TRANSLATIONAL MEDICINE

The panels further found that harmonization guidelines allowed individual laboratories to identify relevant variables that impacted assay performance and to improve these variables within their individual laboratory-specific SOPs without the need for assay protocol standardization (14, 16, 23, 28). These independent and harmonized data sets from multiple laboratories facilitated the establishment of assay-specific reference values for background, lower limits of detection, and replicate variation, which served as community-wide benchmarks for assay performance (25). Finally, it was observed that, in flow cytometry–based assays, gating, data analysis, and reporting methodologies played a crucial role in individual laboratory performance (14, 28, 30).

In summary, the insights from the largest international programs in the field suggest that the harmonization process is an effective mechanism for quality control of immunological assays. Although the above efforts have focused specifically on T cell assays, the principles of harmonization may be applied to assays assessing function and phenotype of other immune cell subsets, such as natural killer (NK) cells, B cells, macrophages, and myeloid-derived suppressor cells. Recent data from immunotherapy studies suggest that increases in absolute lymphocyte counts (31), elevations in levels of interferon γ–responsive cytokines, and expansion of antigen-specific infused cells (6) or vaccine-induced T cells (19, 20) may be associated with response to treatment. Given the biological complexity of most immunotherapy strategies, objective quality control in both hypothesis-testing and hypothesis-generating studies may be required to accelerate biomarker development.

VIRTUES OF HARMONY

Assay harmonization can be viewed as a virtual centralized platform that evaluates and confirms the validity of laboratory-specific reagents, standards, SOPs, analytical methods, and data-reporting procedures for participating laboratories throughout assay development and implementation. Its iterative testing process provides important and ongoing quality control checks for individual laboratories (Fig. 1), with a substantial reduction in data variability across laboratories (Fig. 2). On the basis of the increased relevance of and emphasis on biomarkers in all stages of clinical trials, it is reasonable for assay harmonization to be initiated at the preclinical or early clinical stage, particularly to support rational decision-making and accelerated development of new therapies through identification of clinically useful immune biomarkers. This may enable the defining of patient populations for therapeutic interventions or predicting toxicity. Ultimately, the clinical utility of harmonized immune assays will depend upon their correlation with clinical results. For instance, recent clinical trials have revealed a potential association between T cell activity and clinical efficacy (6, 19, 20).

Other advantages of assay harmonization are that participants share ownership in the process and that all laboratories, independent of their level of experience, benefit from participation. The fact that harmonization allows individual laboratories to use their own resources and protocols distinguishes the harmonization process from the more rigid standardization of assay protocols (Table 1), which cannot be practically adopted throughout the broader research community. The primary outcome of harmonization is that data sets generated across multiple harmonized laboratories are directly interpretable with regard to test performance. Its practical application may lead to similar improvements as the International Conference on Harmonization—Good Clinical Practice Guidelines (32) did for clinical protocols. Additionally, availability of the harmonization guidelines in peer-reviewed journals provides access for laboratories that have not participated in proficiency panels and supports larger-scale integration of the assay harmonization process in translational research.

Signs of increased awareness for the benefits of assay harmonization are the growing number of proficiency panels conducted by CIC and CIP, with 23 completed and ongoing panels and 110 participating laboratories from across several communities, as well as the growing number of related initiatives (33, 34). Translation of the insights obtained from harmonization processes into successful biomarker development will require wider acceptance across several scientific fields, such as the cancer, infectious diseases, and autoimmunity research communities.

SIGNING UP FOR HARMONIZATION

Participation of new laboratories and clinical trial sites in the harmonization process of the CIC (http://www.cancerresearch.org/consortium) and CIP (http://www.cimt.eu/workgroup/CIP) consortia is open through their websites. Both consortia are evolving their programs through the harmonization of new assays as needed in the community. Although initial harmonization panels focused on the measurement of T cell responses in peripheral blood samples, efforts could be extended to the measurement of T cell reactivity in the tumor microenvironment via other approaches, including immunohistochemistry or gene expression profiling, with outcomes that have the potential to inform disease prognosis (35) or predict effects of therapy.

Primarily for financial and logistic reasons, the role of the nonprofit-based consortia in conducting proficiency panels is limited to addressing crucial assay parameters and setting harmonization guidelines. Further progress and implementation of the harmonization concept can be achieved through an increase in scale to support broader and community-wide access to proficiency panels. Such a task will most certainly require dedicated support from funding agencies or involvement of commercial vendors. Harmonization consortia may function in this context as a resource for expertise to help succeeding organizations to acquire expertise and set up the proficiency panel process.

CONCLUDING REMARKS

The ultimate objective for assay harmonization is to accelerate therapeutic development of novel immunotherapy agents and to support the identification of relevant assays to generate biomarker data that correlate with clinical outcome. The harmonization process offers an important step forward to bridge the gap between exploratory biomarker assays when there is inherent methodological and data variability and validated assays used late in clinical development. Immunoassay harmonization will likely achieve its full potential if the research community can move toward the following goals: i) wide adoption of the harmonization process across relevant communities (such as cancer, infectious diseases, and autoimmune disease); ii) implementation of the harmonization process early in therapeutic development; iii) coverage of a wide spectrum of assays; iv) prospective application of harmonized assays in clinical trials; and v) scale-up of quality-control services, such as proficiency panels for assays that have harmonization guidelines in place. These goals require community-wide collaboration and buy in to achieve the translational potential of this approach. In the field of cancer immunotherapy, assay harmonization is an essential component of a new methodological framework that has the potential to enable accelerated clinical development and higher probability of success for new agents. Importantly, the concept of assay harmonization is not limited to T cell or other immune assays; rather, its principles may be applied to translational research across scientific disciplines.

References and Notes

  1. Acknowledgments: This Perspective is a summary based on the work of the CIMT CIP (S.H.V.D.B., C.G., C.O., M.J.P.W., C.M.B.) and the CIC of the CRI (M.K., S.J., P.R., A.H.). We are indebted to all participants of the assay harmonization proficiency panels for their participation. Funding: This work was supported in part by the CIC of the CRI and the Wallace Coulter Foundation. Competing interests: The authors declare no competing interests.
View Abstract

Navigate This Article