What evidence do we need for biomarker qualification?

DEVELOPING AN EVIDENCE FRAMEWORK FOR BIOMARKER QUALIFICATION

The history of biomarker qualification stretches back to classic statistical theory on surrogate end points (9). Since then, the discussion has evolved from initial concepts of validity criteria to more comprehensive views that account for benefit-risk relationships, the level of evidence required, the distinction of the term “qualification,” and the separate needs for analytical validation (establishing that the performance characteristics of a test, tool, or instrument are acceptable in terms of its sensitivity, specificity, accuracy, precision, and other relevant performance characteristics using a specified technical protocol) (2, 10–14).

After a 1999 FDA/NIH workshop on biomarkers and the FDA Critical Path publication (2004) that initially called for an emphasis on biomarker development (15), aspects of the biomarker qualification process have been steadily developed. Yet, progress on development of evidence criteria has been intermittent given the challenges of defining a generalized framework, which is applicable across multiple disease areas, types of biomarkers, and COUs. The factors that impact the type and extent of evidence criteria needed to qualify a biomarker are complex, unique to how the biomarker will be used, and often are not easily quantified. To date, no single-stakeholder community has been able to develop evidence criteria in isolation. Therefore, this group effort developed a framework drawing on the expertise and experience of its members while following the lead of prior efforts in using a semiquantitative approach.

One salient earlier effort to develop evidence criteria by Williams et al. (16) proposed a semiquantitative framework that aimed to “qualify biomarkers in terms of cost effectiveness using a set of principles that enable the evaluation of biomarkers even with incomplete knowledge.” This approach applies “tolerability of risk” to supportive evidence. It concludes that biomarker qualification essentially rests on the principle that the value of incremental benefits provided by the true results of the biomarker must exceed the incremental costs (defined as societal harm) of the false results with the biomarker. The practicality of this approach, however, is limited given the difficulty in measuring both benefits as well as harm.

Altar et al. (17) provided the first “evidence map” for qualification with categorical descriptions of different types of scientific evidence required for different levels of risk. The map, however, was complex and did not adequately account for COU. By contrast, a 2010 Institute of Medicine study (18) emphasized the importance of COU as part of a three-part framework, which included analytical validation, biomarker qualification, and utilization components. In that case, however, the model did not address specific decision processes or evaluation criteria. Amur et al. (8, 19) noted the difference between biomarker development within drug-specific development programs and the process outlined by the FDA’s Biomarkers Qualification Program (20). These authors emphasized the importance of the COU described in conjunction with definitions of biomarker types, although risk-based evidence criteria were not defined.

The framework proposed here synthesizes key elements of the above efforts to propose an overarching approach to evidence criteria. In particular, it applies some of the concepts of risk and benefit proposed by Williams et al. (16) to specific considerations driven by the COU and then uses these to help define an evidence map.

We applied the biomarker nomenclature outlined in the BEST (Biomarkers, Endpoints, and other Tools) glossary first published by the FDA/NIH biomarker working group in 2016 (2). For reference, categories of biomarkers are summarized in the full framework document (21). These categories largely reflect the various COUs that can inform how the biomarker may be applied in drug development and regulatory review.

Under the auspices of the Foundation for NIH (FNIH) Biomarkers Consortium, a multistakeholder group with representatives from FDA, NIH, industry, patient groups, and academia created an early draft of the framework document. A workshop cosponsored by the FNIH Biomarkers Consortium and FDA was held in April 2016 with more than 200 participants. A draft of this framework document was distributed to participants prior to the workshop, along with several case studies. The authors summarized input from the workshop and synthesized the resulting consensus full framework document (21).

THE EVIDENTIARY CRITERIA FRAMEWORK PROPOSAL

The general evidentiary criteria framework is summarized below and includes five component steps as follows: define a need statement, define a COU statement, assess benefit in light of the COU, assess risk in light of the COU, and populate an evidentiary criteria map (Fig. 1). The current framework is intended to support constructive discussions between biomarker developers needing to submit to the biomarker qualification program and regulators at the FDA, with the goal of enabling refinements of the COU as the data mature (Fig. 1).

• Download high-res image
• Open in new tab
• Download Powerpoint

LINKING BENEFIT AND RISK TO THE EVIDENCE REQUIRED

The evidence maps in this evidentiary criteria framework are inspired by, but are considerably less complex than, the map used by Altar et al. (17). The identified need and choices made in the COU section impact the overall relative level of benefit and risk, which in turn determines the level of evidence needed to evaluate the biomarker for qualification. Note that the type, quality, and amount of data required will be linked to the benefit-risk relationship through the evidentiary criteria grid relative to the need statement (Fig. 1). Thus, a candidate biomarker with a high negative consequence if incorrect and low improvement over the current standard would require a more extensive data package than a candidate biomarker with a low negative consequence of failure and a high benefit. A list of questions to help guide the amount of evidence needed is included in the full framework document (21).

Consider a favorable benefit-risk scenario (requiring a minimal level of evidence) in which a biomarker is used for stratification of patients to ensure equal distribution of biomarker-positive and biomarker-negative individuals in the different arms of a clinical trial. If the biomarker does not perform as expected, the loss consists of the resources spent on the biomarker assay and would not influence the trial outcome to the patient or patient safety. An example of this would be a predictive biomarker of drug efficacy that attempts to limit the trial to only those who are most sensitive to the drug. If the biomarker fails, the subjects in the trial would have the same risk and benefit from treatment that would be found in the absence of the biomarker.

An example of a challenging benefit-risk relationship (potential for high benefit while accompanied by high risk) is a biomarker used as a surrogate end point, which would require a high level of evidence. If the biomarker is not truly a surrogate end point in terms of predicting clinical benefit, the results of the clinical trial would be invalid, and inappropriate approval decisions would be made. This would lead to potentially ineffective drugs being marketed or patients being denied access to effective therapy. Yet, another example of a challenging COU is a safety biomarker intended to replace the current standard and be used routinely to identify organ toxicity. In both of these examples, a high level of evidence would be needed to support the biomarker’s COU. On the other hand, a safety biomarker intended to supplement existing standards to be used only in certain well-defined circumstances would be anticipated to require more moderate evidence.

As part of the evidence needed to support a biomarker’s development and COU, two critical factors include the characterization and validation of the biomarker assay and the statistical plan to ensure that the evidence generated supports the eventual use of the biomarker. For the assay, biases introduced will affect the clinical interpretation of the biomarker’s accuracy and its value as a drug development tool. Similarly, a careful evaluation of statistical approaches is critical because some assumptions/claims may be unfounded if based on inappropriate statistical analyses.

THE EVIDENTIARY CRITERIA ASSESSMENT MAP

Our current inability to quantitate the benefit, risk, or value of individual data sources prevents a strictly quantitative link from the amount of benefit and risk to the amount of evidence needed to qualify a biomarker. It is generally agreed, however, that we can categorize benefits, risks, and evidence into broad semiquantitative groups. Because regulatory science is not sufficiently advanced to provide this direct link, we have proposed the use of categorical descriptions for what constitutes a high and minimal level of evidence. An assessment of the strength of the evidence needed for a biomarker to be linked to drug development decision-making has been separated into several categories. These categories are captured in the evidentiary criteria assessment map and include elements of scientific understanding and biological performance and the types of clinical data and samples proposed to establish qualification. Although the process described herein refers to clinical qualification, clinical data can be augmented by nonclinical data to provide additional weight of evidence or to provide bridging information if clinical data are not feasible. Scientific understanding includes the biological rationale, understanding of the molecular mechanisms, and the link of the proposed biomarker to regulatory understanding of the scientific impact. Biological performance includes consistency of correlation of the biomarker changes, presence of a dose response, temporal relationship to the magnitude of the biomarker response and outcome, and sensitivity and specificity of the biomarker response. The types and amount of data/samples required include the quality of the data source and whether prospective or retrospective. To provide initial context, we have suggested example descriptions of expectations for several areas that have been discussed in this document [see Table 1 in (21)]. It is important to note that the evidentiary criteria for qualification are dependent on the COU. For a broad COU, the criteria for each area may require a high level of evidence. Alternatively, a biomarker that is used in a narrow context could be fulfilled with less evidence. The criteria will be modified as more data become available and as the COU is refined during the biomarker qualification process and the field advances as a whole.

An evidentiary criteria assessment map provides a visual representation of an evidence map for biomarker development (21). For example, with assay validation, a high level of evidence should be required for regulatory clearance or approval for marketing as a diagnostic, whereas a minimal level of evidence should be deemed “fit-for-purpose” validation with acceptable performance characteristics. The evidence map can be used as a communication tool for gaining alignment between submitters and FDA reviewers at several key milestones for a biomarker development plan: (i) initial discussions to align expectations; (ii) purposeful interim progress updates to ensure that evidence expectations have been met before proceeding further; and (iii) review evaluation to support the qualification outcome. An evidentiary criteria assessment map also can be used internally by the submitter to track the current level of evidence of the biomarker relative to the intended level of evidence to meet the qualification claim.