Immune monitoring technology primer: clinical validation for predictive markers
© Dobbin. 2015
Received: 3 August 2015
Accepted: 6 August 2015
Published: 20 October 2015
Recent biotechnological developments have resulted in increasing interest in immunology biomarkers. These biomarkers have potential clinical utility in the near future as predictors of treatment response. Hence, clinical validation of these predictive markers is critical.
The process of clinically validating a predictive biomarker is reviewed. Validation of a predictive biomarker requires quantifying the strength of a statistical interaction between marker and a treatment. Different study designs are considered.
Clinical validation of immunology biomarkers can be demanding both in terms of time and resources, and careful planning and study design are critical.
Description of the technology
Appropriate statistical design and data of experiments is necessary for the successful development of predictive immunology biomarkers. Predictive immune response-based biomarkers may provide “evidence about the probability of benefit or toxicity”  from an immunotherapy approach. This discussion will focus on biomarkers for benefit. An ideal predictive marker would allow us to perfectly separate patients into a group who will respond to the therapy, and a group who will not respond. The groups are formed based on biomarker measurements taken either before any therapy is given, or early on in the treatment regimen.
Predictive markers in the real world do not attain the ideal sketched in the previous paragraph, and clinical validation is the process to determine how these imperfect markers, if actually used in the clinic, will impact clinical outcomes. In general, this is a complicated question and it is helpful to break it down into simpler parts. First, the test needs to be analytically validated. Here, much guidance has been provided in recent years for assessing and reporting technical reproducibility [2, 3]. Although immunology markers present unique technical measurement challenges, the guidance in these papers may be helpful. Second, the analytically validated test needs to be clinically validated. For a predictive biomarker validation study in a phase III setting, a clinically relevant outcome should be used, one that reflects “survival or symptomatic status of the subject” , or an approved surrogate outcome that has been adequately validated. Few such surrogates exist, however. New end points need to be defined that capture immunotherapy related response patterns such as delayed responses, “progression before regression and delayed survival separation curves.” Third, an appropriate study design needs to be put in place and analyzed to validate the predictive marker. Such a study may be either prospective or use archived specimens from a clinical trial . How to properly design and analyze a biomarker validation study is currently an area of active development, which we will briefly review next.
Type of data obtained/readout
This primer has focused on biomarkers that provide a continuous response since this is likely to be the most frequent scenario for immune-oncology biomarkers. These types of biomarkers include univariate “machine readout” settings, some pathologist scoring settings, multiplex assays and high dimensional assays (e.g., RNAseq). The latter two are included in this category because the multiple dimensions must be formed into a univariate score in order for clinical decisions to be made. This is typically done using a linear combination. In addition, a cutoff point to be used for the medical treatment decision must be specified. While the linear combination is typically formed based on statistical criteria, the selection of the cutoff point should be based largely on implications for clinical outcomes (e.g., lower tolerance for False Negative considering the life threatening nature of cancer and lack of alternative treatments for advanced tumors). Other types of biomarkers provide binary or categorical results.
The readout from software such as TreatmentSelection is the change in the average probability of 5-year survival under marker-guided therapy compared to current standard of care. The “average” here is taken across the target population.
Limitations of the approach
Types of samples needed and special issues pertaining to samples
In most cases, clinical validation of a predictive marker will require samples from a phase III clinical trial in which individual patients have been randomized to the therapy to which the marker is predicting response.
Level of evidence
The number of publications discussing the methodologic aspects of the process of clinical validation of predictive biomarkers has been growing in parallel with the increasing understanding of the disease biology and the mechanism of action of cancer drugs, including immunotherapy approaches.
Many thanks to Alessandra Cesano, Magdalena Thurin and Giuseppe Masucci for comments on the manuscript.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Dancey JE, Dobbin KK, Groshen S, Jessup JM, Hruszkewycz AH, Koehler M, et al. Guidelines for the development and incorporation of biomarker studies in early clinical trials of novel agents. Clin Cancer Res. 2010;16:1745–55.View ArticlePubMedGoogle Scholar
- Raunig DL, McShane LM, Pennello G, Gatsonis C, Carson PL, Voyvodic JT, et al. Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment. Stat Methods Med Res. 2014;24:27–67.View ArticlePubMedGoogle Scholar
- McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M. Clark GM for the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics. J Natl Cancer Inst. 2005;97:1180–4.View ArticlePubMedGoogle Scholar
- Cook TD, DeMets DL. Introduction to Statistical Methods for Clinical Trials. Boca Raton: Chapman and Hall; 2008.Google Scholar
- Simon RM, Paik S, Hayes DF. Use of archived specimens in evaluation of prognostic and predictive biomarkers. J Natl Cancer Inst. 2009;101:1446–52.PubMed CentralView ArticlePubMedGoogle Scholar
- Royston P, Sauerbrei W. A new approach to modeling interactions between treatment and continuous covariates in clinical trials using fractional polynomials. Stat Med. 2004;23:2509–25.View ArticlePubMedGoogle Scholar
- Bonetti M, Gelber RD. Patterns of treatment effects in subsets of patients in clinical trials. Biostatistics. 2004;5:465–81.View ArticlePubMedGoogle Scholar
- Taylor JM, Ankerst DP. Andridge RR Validation of biomarker-based risk prediction models. Clin Cancer Res. 2008;14:5977–83.PubMed CentralView ArticlePubMedGoogle Scholar
- Song X, Pepe MS. Evaluating markers for selecting a patient’s treatment. Biometrics. 2004;60:874–83.View ArticlePubMedGoogle Scholar
- Janes H, Brown MD, Huang Y, Pepe MS. An approach to evaluating and comparing biomarkers for patient treatment selection. Int J Biostat. 2014;10:99–121.PubMed CentralPubMedGoogle Scholar
- Janes H, Pepe MS, Bossuyt PM, Barlow WE. Measuring the performance of markers for guiding treatment decisions. Ann Intern Med. 2013;154:253–9.View ArticleGoogle Scholar
- Janes H, Pepe MS, Huang Y. A framework for evaluating markers used to select patient treatment. Med Decis Making. 2014;34:159.PubMed CentralView ArticlePubMedGoogle Scholar
- Cardoso F, Van’t Veer L, Rutgers E, Loi S, Mook S, Piccart-Gebhart MJ. Clinical application of the 70-gene profile: the MINDACT trial. J Clin Oncol. 2008;26:729–35.View ArticlePubMedGoogle Scholar