METADATA last updated: 2026-03-20 RT initial conversion file_name: 2020-12-22_Complaint Letter to FDA regarding Reference Panel - IQA-Request-Hyman-Phelps-McNamara.md file_date: 2020-12-22 title: Complaint Letter to FDA regarding Reference Panel - IQA-Request-Hyman-Phelps-McNamara category: regulatory subcategory: fda-policy tags: source_file_type: pdf xfile_type: NA gfile_url: NA xfile_github_download_url: NA pdf_gdrive_url: https://drive.google.com/file/d/1VJGWxjF7-l8KlYTB_pCKoSIhtiLTeEjW pdf_github_url: https://github.com/FocusOnFoundationsNonprofit/floodlamp-archive/blob/main/regulatory/fda-policy/2020-12-22_Complaint%20Letter%20to%20FDA%20regarding%20Reference%20Panel%20-%20IQA-Request-Hyman-Phelps-McNamara.pdf conversion_input_file_type: pdf conversion: claude opus 4.6 license: Public Domain tokens: 2717 words: 1984 notes: converted using prompt file - not checked summary_short: A December 2020 Information Quality Act complaint filed by law firm Hyman, Phelps & McNamara on behalf of unnamed EUA test developer clients, challenging the accuracy of the FDA SARS-CoV-2 Reference Panel Comparative Data published on the FDA website. The letter argues that flaws in the reference panel protocol—particularly variable performance of heat-inactivated virus in negative clinical matrix—produced unreliable limit-of-detection results, and requests that FDA remove the data, issue a corrective public statement, and make the panel's validation data publicly available. CONTENT Hyman, Phelps & McNamara, P.C. 700 Thirteenth Street, N.W. Suite 1200 Washington, D.C. 20005-5929 (202) 737-5600 www.hpm.com December 22, 2020 By Email Office of the Ombudsman U.S. Food and Drug Administration 10903 New Hampshire Avenue WO 32, Room 4260 Silver Spring, Maryland 20993 OMBUDS@OC.FDA.HHS.gov Re: Information Quality Act Complaint Dear Office of the Ombudsman: We write on behalf of clients adversely affected by FDA's publication on its website of information that does not meet the statutory, Office of Management and Budget (OMB), Department of Health and Human Services (HHS), or FDA guidelines under the Information Quality Act, Pub. L. No. 106-554 (2000). We ask that the inaccurate and misleading information be immediately removed until such time as FDA is able to present objective, accurate, and useful information to the public. The inaccurate and misleading information we are challenging is contained on FDA's website, under the heading [SARS-CoV-2 Reference Panel Comparative Data](https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-reference-panel-comparative-data). The website lists what purport to be comparative sensitivity data for FDA-authorized SARS-CoV-2 assays. For the reasons explained below, the data contained on this website are unreliable and must be removed. We understand that FDA published these data on its [website](https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-reference-panel-comparative-data) on or about September 15, 2020. The very first statement by the Agency on the webpage is the claim that "[t]he FDA SARS-CoV-2 Reference Panel allows for a more precise comparison of the analytical performance of different molecular in vitro diagnostic (IVD) assays intended to detect SARS-CoV-2." Unfortunately, due to flaws in the Agency's Reference Panel program, this statement is incorrect, and the results published on the website are misleading to the public. Instead of allowing for "more precise comparison[s]," the data generated by test manufacturers using the FDA Reference Panel in accordance with FDA's protocol have yielded spurious results. According to FDA's website, the Agency developed a Reference Panel Program in order to compare the sensitivity of EUA-authorized assays for SARS-CoV-2 through testing against standardized reference material produced by FDA. We understand that FDA sent a specified number of reference samples (the Reference Panel) to the test manufacturers together with a protocol for testing these samples with their EUA assays to determine the "limit of detection" (which correlates with analytical test sensitivity). We further understand that the Reference Panel comprises "heat-inactivated virus" at predetermined concentrations. Finally, we understand that FDA's protocol instructed laboratories to first dilute the FDA reference samples by pre-mixing them with "Negative Clinical Matrix" (i.e., viral transport media combined with a clinical sample from a patient who tested negative). FDA's use of the reference panel is predicated on the assumption that FDA's virus preparation can serve as a surrogate for actual clinical samples from patients with SARS-CoV-2 infection (which would contain live encapsulated virus). FDA has provided no data showing that its reference panel is a valid surrogate (in contrast to FDA's extensive validation requirements for companies seeking to use surrogate markers). In numerous instances, the limit of detection (LOD) listed on FDA's website, based on the use of FDA's reference panel, is significantly different from the established LOD in the EUA for the same test. Tests that previously were established based on clinical studies to have high sensitivity are listed as having performed poorly with FDA's reference panel. There is also a wide range of results for this supposedly key determinant of performance, with listed LoDs varying more than ten-fold, even though all of these tests have received EUA authorization. We understand that several laboratories have performed experiments on the reference panel material to investigate the source of the discrepant results. We further understand that these laboratories observed variable performance of the reference panel material when placed into the negative clinical matrix as instructed in the protocol provided by FDA. The laboratories believe that the variability combined with the protocol design (which locks a laboratory into a particular performance range based on an initial 'range finding' experiment) is likely the cause of the suppressed sensitivity observed in the panel results, and that the degree of variability is likely to be variable across different tests and laboratories as the timing and handling steps in the protocol are not specified in sufficient detail. In short, the reason for these high LoDs lies not in the EUA assays, but with flaws in FDA's reference panel and associated protocol, specifically the apparent variability in performance of the reference panel when placed in negative clinical matrix. The results here stand in contrast with reproducible, accurate performance when testing real clinical specimens. We understand that FDA personnel have been informed of the problems with the reference panel program but have failed to take remedial measures, and the erroneous information remains on FDA's website. We understand that several companies notified FDA of these significant problems, but the agency failed to act. Indeed, even after being notified that the data were flawed, FDA has continued to recommend that these data be used to compare the sensitivity of different assays and to help select SARS-CoV-2 assays for both diagnostic and developmental purposes. See, e.g., FDA, [Antigen Template for Test Developers](https://www.fda.gov/media/137907/download), at 15 (Oct. 26, 2020 ed.) ("We recommend only using an EUA test with high sensitivity and reverse transcription polymerase chain reaction (RT-PCR) … as the comparator method. The comparator method should be one of the more sensitive RT-PCR assays authorized by FDA. We encourage you to review the results from the FDA SARS-CoV-2 Reference Panel available here when selecting your comparator method."). In addition to the direct harm to our clients of having their tests being inaccurately presented as having a low sensitivity, health care practitioners and the public are being misled about critical information regarding testing for a pandemic that represents the most significant public health crisis in a century. A health care provider or a patient may unnecessarily avoid a well-characterized, validated, and FDA-authorized test kit due to its relative placement in the reference panel results table on the FDA website. At a time when access to testing is critical to the U.S. response to COVID-19, sowing public doubt about the performance of test kits that have undergone extensive analytical and clinical validation by test manufacturers, and review and authorization by the Agency, is unnecessary and contrary to public health. The 2000 Information Quality Act, sometimes referred to as the Data Quality Act, directed the Office of Management and Budget (OMB) to issue guidance to federal agencies designed to ensure the "quality, objectivity, utility, and integrity" of information disseminated to the public. Recognizing that "prudent decision making depends on reliable, high-quality information," OMB recently issued, "Improving Implementation of the Information Quality Act," a memorandum to reinforce, clarify, and interpret agency responsibilities with regard to responsibilities under the Information Quality Act (IQA) (hereafter the "Memorandum").[^1] The Memorandum explains that federal agencies have "three core responsibilities" under the IQA: 1. Agencies must embrace a basic standard of quality and consider quality in their information dissemination practices. 2. Agencies must develop information quality assurance procedures that are applied before disseminating information. 3. Agencies must develop an administrative mechanism for affected parties to request that agencies correct information of inadequate quality, with an appeal process and annual reports to OMB. In response to the Memorandum, agencies were required to update their guidelines within 90 days. While FDA does not appear to have updated its guidelines recently, according to the agency, FDA's "goal has been and remains to ensure that all the information [it] disseminate[s] meets the high standards of quality (including objectivity, utility, and integrity) described in the OMB and HHS Guidelines."[^2] FDA "recognizes that public access to high quality information is critical" to its mission and, as a result, the agency "reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance, and dissemination." Those high standards are particularly applicable here, when the dissemination of information erroneously indicating that tests are not sensitive can lead to avoidance of assays that do, in fact, have excellent clinical performance. The data disseminated by FDA pursuant to the Reference Panel program falls well short of these standards and is contrary the agency's stated goal of disseminating only information that "will be useful to the public." FDA's failure is particularly egregious because the data in question constitute "influential scientific [and] statistical information." As FDA recognizes, "OMB Guidelines . . . apply special quality standards to the dissemination of information that is considered . . . 'influential scientific, financial, or statistical information,'" defined by OMB as information that the agency reasonably determines "'will have or does have a clear and substantial impact on important public policies or important private sector decisions' (67 FR 8452; February 22, 2002)." See also Memorandum at 3 ("subset of agency information" that is 'influential scientific, financial, or statistical information' . . . is held to higher quality standards."). FDA plainly intends for this data to be influential. According to FDA's website: > "Recognizing the value to healthcare professionals, laboratories, and patients in understanding the relative performance of NAATs for SARS-CoV-2 . . . FDA is . . . providing a tool for a comparative analysis of the performance of different tests. Such comparison has shown to be useful to healthcare providers and laboratories using these tests." See also, e.g., FDA, Antigen Template for Test Developers (referenced above). Moreover, this particular scientific and statistical information will have a clear and substantial impact on public and private sector decisions relating to a global pandemic that has already killed more than 300,000 Americans and caused enormous and social economic harm. FDA's report clearly qualifies as "influential." FDA's IQA violations are not confined to the substantive flaws with the data presented. FDA has also violated the IQA's mandate that influential data be transparent: "If information that meets the criteria for influential information is disseminated, the OMB Guidelines provide that it must meet certain higher standards of transparency and methods to facilitate the reproducibility of information by qualified third parties." Yet, FDA has provided no public information regarding the development of the Reference Panel or its protocol for preparation of the heat-inactivated virus or the basis for the Agency's belief that its heat-inactivated virus "Reference Panel" accurately measures IVD assay sensitivity for detecting SARS-CoV-2 for all assays. Nor has FDA provided information that would allow anyone else to reproduce the data. The public, including our clients, has no way of evaluating whether FDA adequately validated either the panel or its methodology. As a result, FDA has failed to comply with its own commitment to being transparent. For all of these reasons, we respectfully request that FDA immediately: 1. Remove from its website the SARS-CoV-2 Reference Panel Comparative Data until such time as all the information it contains is accurate; 2. Issue a public statement explaining that FDA is suspending publication of the data because of concerns regarding the accuracy of the data, and stating that the data should not be used to compare sensitivity among assays; and 3. Make data sufficient to demonstrate the validity of the reference panel and protocol publicly available. Additionally, we ask that FDA refrain from publishing data from laboratories generated based on any future FDA reference panel without prior notice to, and consent of, the participating laboratories. Thank you for your prompt attention to this serious public health issue. We are available to meet with you, your respective staffs, and your counsel regarding these issues at your earliest possible opportunity. Respectfully submitted, /s James P. Ellison /s Jeffrey N. Gibbs /s Gail H. Javitt /s Michael D. Shumsky cc: Stacy Amin Abiy Desta Jeff Shuren Tim Stenzel [^1]: OMB, Memorandum, Improving Implementation of the Information Quality Act (Apr. 24, 2019), https://www.whitehouse.gov/wp-content/uploads/2019/04/M-19-15.pdf. [^2]: HHS Office of the Assistant Secretary for Planning and Evaluation, HHS Guidelines for Ensuring and Maximizing the Quality, Objectivity, and Integrity of Information Disseminated to the Public (Oct. 1, 2002), https://aspe.hhs.gov/report/hhs-guidelines-ensuring-and-maximizing-quality-objectivity-utility-and-integrity-information-disseminated-public.