《自然·生物技术》刊发国际组学数据质量控制联盟最新研究成果.pdf

FOCUS | CoMMent FOCUS | comment MAQC and the era of genomic medicine The MicroArray Quality Control consortium—a 16-year international effort led by the FDA and involving hundreds of scientists from academia, industry and government—helped make genomic medicine a reality. Marc Salit and Janet Woodcock N ature Biotechnology readers are no strangers to application, innovation and critical consideration of genomic and genome-scale measurements. This is in part because of the work of scientists involved in the MicroArray Quality Control (MAQC) project. Over its 16-year history—the project was initiated in 2005 and is completing its fourth phase this month (Fig. 1)—MAQC has grown into an enduring collaboration and scientific consortium of researchers from academia, industry and government working to develop the technical basis for assessing ‘fitness for purpose’ of genomic assays for regulatory and clinical applications. MAQC studies, experiences and results have contributed directly to today’s regulatory science landscape, starting with the publication of foundational regulatory guidance and culminating in routine regulatory acceptance. Today, genomic data, with the confidence lent by the MAQC efforts, underpins many drug and device applications in the precision medicine arena. Genomic measurements MAQC did some powerful things to make us better at genomic measurements. One of the most notable was to socialize the idea that our genome-scale tools are measurement devices whose results call for the same critical consideration as those of any lab tool. In the core MAQC work of measuring a shared set of samples on multiple different platforms, MAQC brought classic metrology wisdom to genomics: “The person with one thermometer always knows what temperature it is; the person with two thermometers never knows what temperature it is.” The consortium was originally formed to assess the influence of the measurement process on results from DNA microarrays, with the hope of collectively developing rational microarray quality control. This was in part a response to a burgeoning literature inspired by Maggie Cam’s canonical DNA microarray gene expression comparison1. The Cam lab’s study measuring the same samples on three platforms and comparing the gene lists in a Venn diagram was “… the microarray comparison that launched MAQC projects MAQC-I (2006) • Provide quality control tools to the microarray community • Develop guidelines for microarray data analysis • Establish quality control metrics and thresholds MAQC-II (2010) • Develop best practices for development and validation of predictive models • Assess capabilities and limitations of various data analysis methods for microarray-based predictive models 2005–2007 • Experience with “Guidance to Industry on Pharmacogenomics Data Submission” helps guide and inspire MAQC • Voluntary Genomics Data Submission/ Voluntary eXploratory Data Submission (VGDS/VXDS) MAQC-III/SEQC (2014) • Assess technical performance of NGS and bioinformatics strategies • Evaluate the utility of these technologies in clinical and safety assessments 2007–2011 • Pharmacogenomics Data Submission — Companion Guidance • ICH E15 — Definitions for genomic biomarkers • ICH E16 — Guidance for biomarker qualification MAQC-IV/SEQC-II (2021) • Develop standard analysis protocols and quality control metrics for next-generation sequencing data in regulatory science research and precision medicine 2018 • Guidance on use of public human genetic variant databases to support clinical validity • Guidance on design, development and analytical validation of next-generation sequencing 2010 onward • MAQC results increasingly support regulatory submissions and consultations that include genomic data Regulatory science advances Fig. 1 | The four project phases of the MAQC consortium (top) and advances in related genomic regulatory science (bottom). Phase I, issues and challenges of applying DNA microarrays (2005– 2006). Phase II: genome-wide association studies and machine learning (2006–2010). Phase III: RNA sequencing in clinical use and regulatory application (2010–2016). Phase IV: quality control metrics and reference materials, reference datasets for next-generation sequencing in regulatory settings and precision medicine (2016–2021). Regulatory science highlights include guidances, new regulatory capacities, standardized vocabularies, and the increasing trend of consultations and submissions as genomics translates to application. 1,000 microarray comparisons” (Chris Cheadle, personal communication, 2006). MAQC sought to both develop a systematic approach to assessing and establishing the trustworthiness of results and to identify practices that best yielded such results. Analytical validation2 is typically based on a set of well-characterized samples representative of those under study. MAQC stepped into the maw of what remains (nearly) out of reach: samples authoritatively characterized at the genome scale (still practically out of reach for quantitative properties, such as gene expression, but becoming more accessible for sequence3). The innovative ‘titration’ mixture design (MAQC I) and adoption of spike-in controls (MAQC II) helped to develop quantitative performance metrics where none was Nature Biotechnology | www.nature.com/naturebiotechnology intrinsically accessible. These performance metrics offered some ways to ‘benchmark’ genome-scale measurements that allow us all to do better genome-scale biology, to better consider hypothesis testing, and to develop better hypotheses and chase fewer (costly) wild geese. Genomic data in the clinic and at the regulator It is noteworthy that when MAQC was initiated, genomic and genome-scale measurements were novel to regulated applications, which relied heavily on analytical validation as the basis for regulatory approval and oversight. Similar validation of genomics remains out of reach, but MAQC developed systematic ways to develop evidence CoMMent | FOCUS comment | FOCUS supporting claims of validity. MAQC-III (Sequencing Quality Control, or SEQC) saw innovation in systematic evaluation of sequence bioinformatics, a critical element of genomic measurement science. This work foregrounded the challenge of demonstrating performance of computational methods in clinical and regulated applications of RNA and DNA sequencing and has seen application in establishing clinical sequencing pipelines. The work of MAQC was an element in the continuing advance of quantitative genome-scale measurements, now routinely deployed in scientific inquiry with single-cell resolution. A particular power of MAQC came from its inclusion of scientific leaders from all key sectors, with strong representation by commercial innovators and service providers, academia, public sector organizations, biotech and pharma. The convening power of the US Food and Drug Administration (FDA) energized the community with the hope and purpose of seeing the technologies reach scale in the regulated applications that touch our lives through medical advances. The substantial impacts of the 16 years of MAQC work have led to better experiment designs, more objective and sensitive ways to detect differences among conditions through better practices and advances in biostatistics, ongoing cross-sector community collaboration, and a rich substrate of FAIR (findable, accessible, interoperable, reusable) data that have been the basis of advances beyond the MAQC community. And all this work has arisen in a timely manner matching the advances and scaled deployment of science and technology, beginning a scant few years after the first publications of the human genome and an active presence at the time of wide-scale adoption of next-generation sequencing as a quantitative tool. MAQC stands as an example of an effective sociological approach to the scale-up of biological science in the early genomics era. It is an example of the FDA working closely with its stakeholders to establish not only the scientific basis for assessing safety and efficacy, but also expectations of best practices and fitness to purpose of regulatory submissions. A regulatory transformation The success of MAQC is evident in the transformation of product data packages submitted to the FDA. Industry went from being wary of including genomic data in data packages to recognizing that the agency now leads the world in terms of understanding, and nuanced interpretation, of genomic measurements, bringing a scientifically rigorous and fit-for-purpose consideration of quality and reliability. The FDA now routinely receives genomic data in medical product applications, and drug approvals increasingly include genotypes in indications on product labels. FDA decisions frequently affect individual or public health and may have major economic consequences. Data used for regulatory decision-making cannot just be interesting or innovative: they must be actionable. That means reliable, reproducible and interpretable. Translation of new research findings and technologies into actionable information has historically been a chaotic process that can stretch over decades. The regulatory-science ecosystem of MAQC laid out strong commercial incentive to do the rigorous work needed to translate innovation to practice. The public–private– academic project had full partnership from a wide array of stakeholders whose motives transcended traditional academic incentives of novelty with their drive to create robust, reliable, reproducible products that could both be subject to regulatory oversight and be deployed, maintained and productive in clinical service. This formula shows that translational science can move fast-moving basic research fields quickly into application, even in the face of technical challenges, given leadership and collaboration. MAQC’s production of high-quality science and publications has provided a substrate both for regulatory use and in the clinic, moving the entire field forward and enabling widespread adoption of the new technologies. Looking forward As scientific discovery and technologies continue to accelerate, both the objectives and the success of MAQC can serve as a model for collaborations in new fields focused on translation, particularly the application of measurement science. FDA involvement and leadership were key to MAQC’s success and will likely be critical for future endeavors. There are ongoing consortia in fields as disparate as patient-reported outcomes4 and imaging technologies5. Many of these efforts are focused on developing reliable, reproducible, interpretable measurement technologies that can be used confidently in regulatory decisions. Emerging techniques, such as microphysiological systems or ‘organs on a chip’, may require the same collaborative scientific approach to enable wide utility. The current robust incorporation of genomic technologies into medical product development and regulation demonstrates how regulatory science can facilitate rapid adoption of new technologies. MAQC has been a major translational success story. Hopefully, its success will be replicated in more research areas and the lessons learned will help overcome barriers to translation. ❐ Marc Salit1,2 ✉ and Janet Woodcock3 Departments of Bioengineering and Pathology, Stanford University, Stanford, CA, USA. 2SLAC National Accelerator Laboratory, Menlo Park, CA, USA. 3Office of the Commissioner, US Food and Drug Administration, Silver Spring, MD, USA. ✉e-mail: msalit@stanford.edu 1 Published: xx xx xxxx https://doi.org/10.1038/s41587-021-01050-y References 1. Tan, P. K. et al. Nucleic Acids Res. 31, 5676–5684 (2003). 2. Taylor, J. K. Anal. Chem. 55, 600A–608A (1983). 3. Zook, J. M. et al. Nat. Biotechnol. 37, 561–566 (2019). 4. Critical Path Institute. Patient-Reported Outcome Consortium https://c-path.org/programs/proc/ (accessed 1 August 2021). 5. Quantitative Imaging Biomarkers Alliance. https://www.rsna.org/ research/quantitative-imaging-biomarkers-alliance (accessed 2 August 2021). Competing interests The authors declare no competing interests. Nature Biotechnology | www.nature.com/naturebiotechnology