Table 2 NGS: Diagnostic enterprise from clinical sample through diagnosis, reporting, and data archival.

Shown is (i) suggested evidence appropriate for the development of validation standards, which could then be used to establish that evidence, and (ii) critical stakeholders who can be convened to identify existing standards, create new standards, or establish the research agenda that would underpin the needed standards. VCF, variant call format; SOP, standard operating procedures; SDOs, study delivery operations specialists; HHS, health and human services.

PhaseEvidence neededStandards and evidence-developing practices (examples)Stakeholders for standards developmentKnowledge gaps (examples)
Preanalytical:from tissue to DNA • Representative sampling
• Accurate (unbiased) extraction
• Integrity of DNA
• Documentary standard to establish SOPs for sampling
• Reference samples and interlaboratory studies to evaluate extraction and DNA integrity
• Clinical laboratories
• Professional societies
• Clinical SDOs
• Artifacts associated with extraction from archival tissue samples
Sequencing: from DNA to raw sequence data “Wet bench” • Accurate (unbiased) sequencing
• Fit-for-purpose characteristics
• Well-characterized genomic DNA reference materials
• Documentary standard describing sequencing characteristics appropriate for different clinical indications
• Standards laboratories
• Clinical laboratories
• Sequencing technology developers
• Academic laboratories developing methods
• Genome centers
• Sequencing of “difficult” regions of the genome
• Platform artifacts
• High-quality benchmark genomes
• Performance expectations (sensitivity, specificity thresholds)
Sequence bioinformatics: from raw sequence data to VCF “Dry bench” • Unbiased processing of sequence data (mapping and assembly)
• Accurate variant calling
• Accurate and unambiguous variant representation
• Interoperability of data representation
• Documentary standards describing protocols to critically evaluate processes, coupled to knowledge of technical platform idiosyncrasies
• Data representation standards
• Reference data, implementation: benchmark VCF files
• Reference software to evaluate VCF files
• Standards laboratories
• Clinical laboratories
• Sequencing technology developers
• Academic laboratories developing methods
• Genome centers
• Assembly and mapping in “difficult” regions of the genome
• Platform and algorithm artifacts
• High-quality benchmark genomes
• Performance expectations (sensitivity, specificity thresholds)
Functional variant annotation• Accuracy of variant annotation, including establishing confidence in genomic landscape of the call • Documentary standards for critical evaluation of processes, coupled to knowledge of technical platform idiosyncrasies
• Data representation standards
• Interlaboratory comparisons of annotation
• Gold standard annotation of benchmark samples
• Clinical laboratories
• Academic laboratories developing methods
• Genome centers
• Development of genome-wide “gold standard” annotations
Clinical variant knowledge base (PharmGKB, ClinVar)• Clinical scope, reliability, relevance, strength, applicability of data in knowledge base • Documentary standards of evidence for inclusion in knowledge base
• Documentary standards for critical evaluation of knowledge base contents, formatting, transaction accuracy
• Knowledge base intercomparisons
• Clinicians
• Professional societies
• Academic laboratories
• Quantitative frameworks to assess knowledge of the strengths of associations of variants and disease
• Quantitative framework to assess knowledge base curation/accuracy
Clinical interpretation • Accurate interpretation of variants, including incidental findings and classification of pathogenicity
• Accurate clinical findings
• Documentary standards describing best practices and methods of critical evaluation
• Adjudicated benchmark case studies for interlaboratory comparisons of clinical interpretation
• Clinicians
• Professional societies
• Academic laboratories
• Payers
• Quantitative framework to predict performance of clinical interpretation
Reporting• Accurate and clear reporting of results to clinician in a standard format • Documentary standards describing reporting guidelines
• Interlaboratory comparisons and evaluations of reporting
• Professional societies
• Clinicians
• Genetic counselors
• Clinical laboratories
• Payers
• Communicate confidence in findings
EHR archival• Accurate and interoperable representation of WGS, WES test results • Data representation standards
• Documentary standards describing data representation
• Compliance test software to evaluate EHR formatting
• Reference implementations
• Payers
• Professional societies
• HHS
• Standards bodies
• No interoperable EHR standards in common practice