Skip to main content

Immune monitoring technology primer: clinical validation for predictive markers

An Erratum to this article was published on 17 November 2015

Abstract

Background

Recent biotechnological developments have resulted in increasing interest in immunology biomarkers. These biomarkers have potential clinical utility in the near future as predictors of treatment response. Hence, clinical validation of these predictive markers is critical.

Findings

The process of clinically validating a predictive biomarker is reviewed. Validation of a predictive biomarker requires quantifying the strength of a statistical interaction between marker and a treatment. Different study designs are considered.

Conclusions

Clinical validation of immunology biomarkers can be demanding both in terms of time and resources, and careful planning and study design are critical.

Findings

Description of the technology

Appropriate statistical design and data of experiments is necessary for the successful development of predictive immunology biomarkers. Predictive immune response-based biomarkers may provide “evidence about the probability of benefit or toxicity” [1] from an immunotherapy approach. This discussion will focus on biomarkers for benefit. An ideal predictive marker would allow us to perfectly separate patients into a group who will respond to the therapy, and a group who will not respond. The groups are formed based on biomarker measurements taken either before any therapy is given, or early on in the treatment regimen.

Predictive markers in the real world do not attain the ideal sketched in the previous paragraph, and clinical validation is the process to determine how these imperfect markers, if actually used in the clinic, will impact clinical outcomes. In general, this is a complicated question and it is helpful to break it down into simpler parts. First, the test needs to be analytically validated. Here, much guidance has been provided in recent years for assessing and reporting technical reproducibility [2, 3]. Although immunology markers present unique technical measurement challenges, the guidance in these papers may be helpful. Second, the analytically validated test needs to be clinically validated. For a predictive biomarker validation study in a phase III setting, a clinically relevant outcome should be used, one that reflects “survival or symptomatic status of the subject” [4], or an approved surrogate outcome that has been adequately validated. Few such surrogates exist, however. New end points need to be defined that capture immunotherapy related response patterns such as delayed responses, “progression before regression and delayed survival separation curves.” Third, an appropriate study design needs to be put in place and analyzed to validate the predictive marker. Such a study may be either prospective or use archived specimens from a clinical trial [5]. How to properly design and analyze a biomarker validation study is currently an area of active development, which we will briefly review next.

A predictive biomarker is clinically useful to the extent that there is a particular type of statistical interaction between the biomarker values and the effect of the treatment. Designing and analyzing studies to assess an interaction is more difficult than for more standard phase III clinical studies which test a main effect. But progress has recently been made [612]. For example, Fig. 1 shows the relationship between a hypothetical predictive biomarker’s values and the probability of 5-year survival, broken down by treatment groups. From this one can assess the variability in response to each treatment as a function of marker value. A similar figure appears in Janes et al. [10]. The optimal biomarker-guided treatment would be that individuals with marker value below 5 should receive treatment, and those with values above 5 should avoid the therapy. Since the improvement in 5-year survival based on optimal marker-guided therapy results from assigning those with marker value below 5 to the treatment, the distance between the “Control” and “Treatment” curves to the left of 5 characterizes the impact of the marker. Software for estimating this difference is available (TreatmentSelection, http://labs.fhcrc.org/janes/index.html).

Fig. 1
figure 1

A predictive biomarker example. The solid line shows the probability of 5-year survival under treatment; as the biomarker value increases from 0 to 10, the probability of survival decreases. The dashed line shows the probability of 5-year survival under a control; as the biomarker value increases from 0 to 10, the probability of survival increases. The optimal therapy is: treatment those with biomarker value below 5, and do not treat those with biomarker value above 5

Type of data obtained/readout

This primer has focused on biomarkers that provide a continuous response since this is likely to be the most frequent scenario for immune-oncology biomarkers. These types of biomarkers include univariate “machine readout” settings, some pathologist scoring settings, multiplex assays and high dimensional assays (e.g., RNAseq). The latter two are included in this category because the multiple dimensions must be formed into a univariate score in order for clinical decisions to be made. This is typically done using a linear combination. In addition, a cutoff point to be used for the medical treatment decision must be specified. While the linear combination is typically formed based on statistical criteria, the selection of the cutoff point should be based largely on implications for clinical outcomes (e.g., lower tolerance for False Negative considering the life threatening nature of cancer and lack of alternative treatments for advanced tumors). Other types of biomarkers provide binary or categorical results.

The readout from software such as TreatmentSelection is the change in the average probability of 5-year survival under marker-guided therapy compared to current standard of care. The “average” here is taken across the target population.

Limitations of the approach

Biostatistical methods are not yet as well developed for predictive markers as they are for other types of markers. Common examples of biomarker study designs are shown in Fig. 2. The marker strategy design compares a standard of care strategy versus a marker guided strategy, and is being used in the MINDACT trial [13]. The marker stratified design provides the most complete information about the biomarker but tends to be the most expensive. The enrichment design uses the marker for patient selection.

Fig. 2
figure 2

Different biomarker designs: Marker strategy design, marker stratified design, and enrichment design (top to bottom)

Types of samples needed and special issues pertaining to samples

In most cases, clinical validation of a predictive marker will require samples from a phase III clinical trial in which individual patients have been randomized to the therapy to which the marker is predicting response.

Level of evidence

The number of publications discussing the methodologic aspects of the process of clinical validation of predictive biomarkers has been growing in parallel with the increasing understanding of the disease biology and the mechanism of action of cancer drugs, including immunotherapy approaches.

References

  1. Dancey JE, Dobbin KK, Groshen S, Jessup JM, Hruszkewycz AH, Koehler M, et al. Guidelines for the development and incorporation of biomarker studies in early clinical trials of novel agents. Clin Cancer Res. 2010;16:1745–55.

    Article  CAS  PubMed  Google Scholar 

  2. Raunig DL, McShane LM, Pennello G, Gatsonis C, Carson PL, Voyvodic JT, et al. Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment. Stat Methods Med Res. 2014;24:27–67.

    Article  PubMed  Google Scholar 

  3. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M. Clark GM for the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics. J Natl Cancer Inst. 2005;97:1180–4.

    Article  CAS  PubMed  Google Scholar 

  4. Cook TD, DeMets DL. Introduction to Statistical Methods for Clinical Trials. Boca Raton: Chapman and Hall; 2008.

    Google Scholar 

  5. Simon RM, Paik S, Hayes DF. Use of archived specimens in evaluation of prognostic and predictive biomarkers. J Natl Cancer Inst. 2009;101:1446–52.

    Article  PubMed Central  PubMed  Google Scholar 

  6. Royston P, Sauerbrei W. A new approach to modeling interactions between treatment and continuous covariates in clinical trials using fractional polynomials. Stat Med. 2004;23:2509–25.

    Article  PubMed  Google Scholar 

  7. Bonetti M, Gelber RD. Patterns of treatment effects in subsets of patients in clinical trials. Biostatistics. 2004;5:465–81.

    Article  PubMed  Google Scholar 

  8. Taylor JM, Ankerst DP. Andridge RR Validation of biomarker-based risk prediction models. Clin Cancer Res. 2008;14:5977–83.

    Article  PubMed Central  PubMed  Google Scholar 

  9. Song X, Pepe MS. Evaluating markers for selecting a patient’s treatment. Biometrics. 2004;60:874–83.

    Article  PubMed  Google Scholar 

  10. Janes H, Brown MD, Huang Y, Pepe MS. An approach to evaluating and comparing biomarkers for patient treatment selection. Int J Biostat. 2014;10:99–121.

    PubMed Central  PubMed  Google Scholar 

  11. Janes H, Pepe MS, Bossuyt PM, Barlow WE. Measuring the performance of markers for guiding treatment decisions. Ann Intern Med. 2013;154:253–9.

    Article  Google Scholar 

  12. Janes H, Pepe MS, Huang Y. A framework for evaluating markers used to select patient treatment. Med Decis Making. 2014;34:159.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Cardoso F, Van’t Veer L, Rutgers E, Loi S, Mook S, Piccart-Gebhart MJ. Clinical application of the 70-gene profile: the MINDACT trial. J Clin Oncol. 2008;26:729–35.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Many thanks to Alessandra Cesano, Magdalena Thurin and Giuseppe Masucci for comments on the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kevin K. Dobbin.

Additional information

Competing interests

The author declares no competing interests.

An erratum to this article is available at http://dx.doi.org/10.1186/s40425-015-0100-2.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dobbin, K.K. Immune monitoring technology primer: clinical validation for predictive markers. j. immunotherapy cancer 3, 40 (2015). https://doi.org/10.1186/s40425-015-0086-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40425-015-0086-9

Keywords