METADATA
last updated: 2026-03-04 by BA
file_name: 2023-09-23_FDA Response Letter - To Compliant regarding FDA Reference Panel from Dec 2020.md
file_date: 2023-09-23
title: FDA Response Letter - To Compliant regarding FDA Reference Panel from Dec 2020
category: regulatory
subcategory: fda-policy
tags: 
source_file_type: pdf
xfile_type: NA
gfile_url: NA
xfile_github_download_url: NA
pdf_gdrive_url: https://drive.google.com/file/d/12TN_DQShtSnmCa-hJFLGRq-n2W2JJICu
pdf_github_url: https://github.com/FocusOnFoundationsNonprofit/floodlamp-archive/blob/main/regulatory/fda-policy/2023-09-23_FDA%20Response%20Letter%20-%20To%20Compliant%20regarding%20FDA%20Reference%20Panel%20from%20Dec%202020.pdf
conversion_input_file_type: pdf
conversion: megaparse
license: Public Domain
tokens: 4124
words: 3051
notes: Unsure about license
summary_short: The FDA letter dated September 29, 2023 responds to an Information Quality Act request from a law firm seeking removal of the FDA SARS-CoV-2 Reference Panel Comparative Data, claiming it was inaccurate and harmed EUA test developers. FDA explains the reference panel and protocol were scientifically sound for comparing relative limits of detection across assays, that participation and public posting were conditions of EUA authorization, and that variability in posted results supports the panel’s validity rather than a systemic flaw. It notes FDA later stopped distributing the reference panel when materials reached effective expiration and removed the comparative data from its website as outdated, denying requests for a public statement blaming accuracy concerns or for additional validation data publication.


CONTENT

September 29, 2023

Via Email

James P. Ellison 
Jeffrey N. Gibbs 
Gail H. Javitt 
Michael D. Shumsky 
Hyman Phelps & McNamara, P.C. 
700 13th Street NW, Suite 1200 
Washington, DC 20005

Dear Mr. Ellison, Mr. Gibbs, Ms. Javitt, and Mr. Shumsky:

This letter responds to your letter dated December 22, 2020, requesting that the U.S. Food and Drug Administration (FDA) remove FDA SARS-CoV-2 Reference Panel Comparative Data for COVID-19 molecular diagnostic assays contained on FDA’s website. Your request is based on your allegation that the data are inaccurate and misleading and, as a result, do not meet the statutory, Office of Management and Budget (OMB), Department of Health and Human Services (HHS), and FDA guidelines under the Information Quality Act (IQA), Pub. L. No. 106-554 (2000).

For the reasons set forth below, part of your request is now moot and we are denying the other parts of your request.


## I. Introduction
You assert that you are writing on behalf of unidentified clients who hold Emergency Use Authorization (EUA) for their tests and have participated in the Reference Panel program, which is a condition of authorization. You claim that your clients generated data for their tests using the FDA reference panel materials and instructions, and that the reference panel data generated by your clients do not correlate with the Limit of Detection (LoD) they previously established for their tests. You allege that your clients have been subject to “direct harm” because their tests are “being inaccurately presented as having a low sensitivity.” You also allege that health care practitioners and the public are being misled about critical information regarding these diagnostic assays. As a result of these alleged inadequacies, you contend that FDA’s disclosure of the FDA SARS-CoV-2 Reference Panel data violates the 2000 IQA.

Your letter requests that FDA (1) remove the SARS-CoV-2 Reference Panel Comparative Data from its website “until such time as all the information it contains is accurate,” (2) issue a public statement explaining that the data was removed “because of concerns regarding the accuracy of the data,” and (3) publish the data demonstrating “the validity of the reference panel and protocol.”

As discussed below, we have determined that the FDA SARS-CoV-2 Reference Panel and protocol instructions, which are used to generate the SARS-CoV-2 Reference Panel Comparative Data, are scientifically sound for their intended purpose of providing a relative LoD that can be used to establish a comparison of analytical test performances. Accordingly, we are denying your request.

We note that FDA stopped use of the FDA SARS-CoV-2 Reference Panel when it determined the reference panel materials it was providing test developers had, in effect, reached their expiration date. FDA has also removed the SARS-CoV-2 Reference Panel Comparative Data from its website as part of its regular review and updating of COVID-related information because that data has become outdated.


## II. Information Quality Act
In 2002, the Office of Management and Budget issued Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies (Guidelines). As you note in your letter, the principles and core values underlying these Guidelines were updated and reinforced recently through the OMB Memorandum, Improving Implementation of the Information Quality Act (Apr. 24, 2019).1 FDA issued its agency-specific guidelines on September 30, 2002 (FDA Guidelines).2 You stated that “FDA does not appear to have updated its guidelines recently” in response to the OMB 2019 Memorandum. Although the updated guidelines have not yet been reflected on the HHS website, FDA has implemented the guidelines and complies with the latest guidelines. FDA’s practices are consistent with the Memorandum.


## III. Reference Panel and Protocol
The Reference Panel and its protocol provide both the material and procedure to be used to evaluate the analytical sensitivity or relative LoD of an assay used to detect SARS-CoV-2.

### A. Limit of Detection for SARS-CoV-2 Tests
Establishing the analytical sensitivity of a test, often referred to as the LoD (i.e., the lowest quantity or concentration of a component that can be reliably detected with a given assay in at

1 OMB, Memorandum, Improving Implementation of the Information Quality Act (Apr. 24, 2019)1, https://www.whitehouse.gov/wp-content/uploads/2019/04/M-19-15.pdf. 2 HHS Office of the Assistant Secretary for Planning and Evaluation, HHS Guidelines for Ensuring and Maximizing the Quality, Objectivity, and Integrity of Information Disseminated to the Public (HHS Guidelines) (Oct. 1, 2002), https://aspe.hhs.gov/report/hhs-guidelines-ensuring-and-maximizing-quality- objectivity-utility-and-integrity-information-disseminated-public.

U.S. Food & Drug Administration 10903 New Hampshire Avenue Silver Spring, MD 20993 FDA.GOV least 19/20 replicates), is a standard analytical procedure. For SARS-CoV-2 molecular tests, FDA has provided an EUA template with recommendations for determining the LoD.3 The template is updated as appropriate as we learn more about the COVID-19 disease and gain experience with the EUA process for the various types of COVID-19 tests.

The EUA template also provides recommendations to establish an assay clinical sensitivity. Clinical sensitivity is the probability that the test will identify as positive a clinical specimen that has been identified with a reference method. Clinical specimens are the preferred material for determining performance. Since natural clinical specimens were not available to test developers in the early phases of the COVID-19 pandemic, FDA authorized tests based on available data from contrived samples generated from a range of SARS-CoV-2 material sources for analytical and clinical performance evaluation. “Contrived” means that developers could “spike” some specified materials (e.g., viral RNA or inactivated virus) into a clinical matrix (e.g., BAL fluid, sputum, or nasopharyngeal swab). However, this approach is less likely than use of natural patient specimens to accurately characterize test performance.

The LoD, which is required in the labeling for each EUA-authorized test, was therefore based on the material and methodology used by the test developer, and these vary across the hundreds of EUA-authorized tests. As of the end of December 2020, when you had sent your letter, FDA had issued EUAs to over 230 nucleic acid-based tests (NATs) for SARS-CoV-2. Many of these tests (59) were developed and EUA-authorized based only on contrived specimens in the early phases of the pandemic, before clinical specimens were available to the test developers. As clinical specimens became available, FDA recommended that developers obtain and use patient specimens to validate their tests. In addition, FDA recommended evaluating the performance of the test with a comparator test by using clinical specimens. For the analytical performance, a quantified known positive clinical specimen, as determined by an EUA-authorized test, could also be used to create dilutions in clinical matrix for LoD determination. As of July 14, 2023, 277 NATs for SARS-CoV-2 were authorized for emergency use.

Since the LoD of the 230-plus molecular tests was determined using samples spiked with different types of materials, the performance of these tests, as reflected in the labeling originally authorized under the EUAs, cannot be directly compared. In addition, when clinical samples became available, different developers did not utilize the same samples and therefore results obtained with clinical specimens cannot be directly compared across assays either. Use of the same reference material across test developers is thus critical to allow a determination and direct comparison of the relative LoD of NATs for SARS-CoV-2.

For the analytical sensitivities of different tests to be compared, the clinical matrix must be spiked with the same material that has the different targets at the ratios found in the original virus. The material must be the same for different labs and be stable during transport and under the conditions stored prior to testing. Panels developed at the FDA laboratory produced the amount needed to fulfill testing for hundreds of assays, and to achieve consistency of the viral titer, sample composition, and targets present. For these reasons, FDA supplied the FDA SARS-CoV-2 Reference Panel with recommended handling procedures so that each lab would be assured that the material remained functional. Results of the testing to determine LoD were then checked with the blinded panel. This is a robust and efficient procedure to compare analytical sensitivities of a large number of different assays.

Recognizing that the availability of well-characterized reference reagents that all molecular tests can be compared against is critical to performance standardization of EUA-authorized assays during a pandemic, FDA had included in the EUA Letters of Authorization for every molecular-based test a requirement that the test developer would evaluate its test with an FDA- recommended reference material when such material was made available, and would update the test’s labeling to include the results of that additional data. This requirement is set forth in the following Condition of Authorization:

“You will evaluate the analytical limit of detection and assess traceability* of your product with any FDA-recommended reference material(s). After submission to and concurrence with the data by FDA, you will update your labeling to reflect the additional testing. Such labeling updates will be made in consultation with, and require concurrence of, DMD/OHT7- OIR/OPEQ/CDRH.” * Traceability refers to tracing analytical sensitivity/reactivity back to an FDA- recommended reference material.”

Thus, from the beginning, test developers submitting an EUA request for a SARS-CoV-2 NAT knew not only that they would be required to participate in testing with an FDA-recommended reference material but also that the data from that testing would be made publicly available for their EUA-authorized test. Data from evaluation of tests using the same reference panel provide more accurate information on the relative performance of different tests, allowing comparative studies that will give the FDA, professionals, laboratories, and patients a better understanding of the relative sensitivity of the various tests. The FDA SARS-CoV-2 Reference Panel also includes an additional coronavirus allowing the evaluation of cross- reactivity to MERS-CoV.

### B. Development and Validation of the Reference Panel
FDA began distributing the FDA SARS-CoV-2 Reference Panel in May 2020. To develop the FDA SARS-CoV-2 Reference Panel, FDA obtained live virus in February 2020. The SARS- CoV-2 strain used in this panel was cultivated, heat-inactivated, sequenced, and genetically characterized by the Center for Biologics Evaluation and Research (CBER) to produce reference reagents.

FDA appropriately validated the Reference Panel and protocol. In developing, producing, and characterizing the Reference Panel for SARS-CoV-2, CDRH/CBER followed the same scientific approach, methods, and principles used in the development and production of reference reagents for Zika virus tests, which are described in detail in the published articles “Production and characterization of Zika virus RNA reference reagents as a response to a public health emergency,” Transfusion, volume 58, September 2018, and “A Zika Reference Panel for Molecular-Based Diagnostic Devices as a US Food and Drug Administration Response Tool to a Public Health Emergency,” The Journal of Molecular Diagnostics, Vol. 21, No. 6, November 2019. The value of reference materials containing heat-inactivated SARS- CoV-2 virus has also been recognized in international studies; see, e.g., “RNA reference materials with defined viral RNA loads of SARS-CoV-2—A useful tool towards a better PCR assay harmonization,” PLoS ONE 17(1): e0262656 (January 20, 2022) available at https://doi.org/10.1371/journal.pone.0262656.

For Zika virus, CDRH used a variation of the CBER-developed reference reagents to create a reference panel for Zika virus tests suitable to in vitro diagnostics developers. CDRH created the reference panel known as Zika FDA-RP using dilutions of the culture media virus stocks in defibrinated human plasma. The stocks were diluted to various concentrations of NAT- detectable units (NDU), or NDU/mL. A single NDU is the minimum level of target that will result in a positive PCR result, which is not interchangeable with viral copy number/mL. The methodology for production of reagents and preparation of the reference panel, as well as the designs for the LoD range finding study, LoD confirmation study, and LoD blinded validation, are described in detail in the above-referenced article, “A Zika Reference Panel for Molecular- Based Diagnostic Devices as a US Food and Drug Administration Response Tool to a Public Health Emergency.” Those details will not be repeated here but can be found in the attached article. Articles are accepted for publication in The Journal of Molecular Diagnostics only after external scientific review. The FDA reference panel for Zika virus tests, developed and studied as described in this published article, was used to evaluate the performance of Zika virus diagnostic assays before they received an EUA.

Although the details about the preparation of the reference reagents, and the production and validation of the FDA SARS-CoV-2 Reference Panel, have not yet been published4, FDA followed the same scientific approach and methodology that was described in detail in the peer-reviewed, published articles describing the Zika FDA-RP, which was used successfully to evaluate Zika virus assays prior to granting an EUA. In addition, to further evaluate both the material and the protocol, FDA conducted a pilot study with several commercial manufacturers and laboratories prior to sending the FDA SARS-CoV-2 Reference Panel to all developers of EUA-authorized tests. The results of this pilot study supported the quality and utility of this specific FDA SARS-CoV-2 Reference Panel as well as the clarity of its protocol, providing additional external confirmation of the scientific validity of the FDA SARS-CoV-2 Reference Panel prior to distributing it to developers of EUA-authorized tests.

Furthermore, results obtained using the FDA SARS-CoV-2 Reference Panel show a wide distribution of values that further support that the reference panel was of appropriate quality for its use at the time the IQA complaint was submitted (12/22/2020). If the design of the Reference Panel were flawed, it should have resulted in all devices showing high LoD values with no expected dispersion.

4 FDA is developing a manuscript with information on the SARS-CoV-2 Reference Panel, but it has not been internally reviewed and cleared.

You have not provided any data or evidence to support your claim that the Reference Panel or protocol were flawed, or that use of the Reference Panel or protocol led to the FDA SARS- CoV-2 Reference Panel Comparative Data being unreliable. In contrast, existing data and information supports that the development, validation and use of the Reference Panel was appropriate for its stated purpose.


## IV. The FDA SARS-CoV-2 Reference Panel Comparative Data Is Not “Influential”
In your letter, you assert that the FDA SARS-CoV-2 Reference Panel Comparative Data constitute “influential” information as that term is used in the context of the IQA. The OMB Guidelines state that “influential” “means that the agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions.” As explicitly contemplated by the OMB’s Guidelines, FDA’s Guidelines further explain this term in light of FDA’s areas of responsibility, stating:

For purposes of this guidance, influential information is defined as disseminated information that results from or is used in support of agency actions that are expected to have an annual effect on the economy of $100 million or more or will adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or tribal governments or communities. It should be noted that the definition applies to “information” itself, not to decisions that the information may support. Even if a decision or action by FDA is itself very important, a particular piece of information supporting it may or may not be “influential.”

In asserting that the information is “influential,” you base this on your view that FDA intends it to be influential because FDA has said that this type of comparison information has been “shown to be useful to healthcare providers and laboratories using these tests.” When a person or firm is deciding which authorized COVID molecular test to use, the comparative performance information can be useful. To the extent such information is considered, however, it is one of a wide range of considerations. The SARS-CoV-2 Reference Panel Comparative Data can help people understand the relative performance of authorized molecular tests, but relative performance is, and should be, just one consideration along with many other considerations, such as the indication a test is authorized for, test availability, test throughput, and cost. FDA is not aware of any information that shows, or even suggests, that healthcare providers and labs rely solely or even heavily on the posted relative performance information and, even if they did, they would merely be choosing one FDA-reviewed and authorized molecular test over another. In short, although this information can be useful, it is not “influential” as that term is used in the OMB and FDA Guidelines.


## V. Conclusion
Your request that FDA remove the SARS-CoV-2 Reference Panel Comparative Data from its website “until such time as all the information it contains is accurate,” is moot because FDA subsequently removed that information as part of its regular review and updating of COVID-

U.S. Food & Drug Administration 10903 New Hampshire Avenue Silver Spring, MD 20993 FDA.GOV

related information because it had become outdated. For the reasons stated above, FDA denies your request that it issue a public statement explaining that the data was removed “because of concerns regarding the accuracy of the data” and that FDA publish data demonstrating the validity of the Reference Panel and protocol.

Thank you for your interest in the quality of information disseminated by FDA. If you do not agree with FDA’s decision about your complaint (including any corrective action), you may send a request for reconsideration within 30 days of receipt of our decision. You may use any of the Procedures for Submitting Complaints described in the FDA specific guidelines contained in the HHS Information Quality Guidelines available at: https://aspe.hhs.gov/reports/hhs-guidelines-ensuring-maximizing-quality-objectivity-utility- integrity-information-disseminated. A request for reconsideration should state the reasons why you believe the response is inadequate, should be designated as an “Information Quality Appeal,” and sent to the following address:

Food and Drug Administration Office of Ombudsman 10903 New Hampshire Avenue WO Building 32, Room 4260 Silver Spring, MD 29993 Email: Ombuds@OC.FDA.gov

A request for reconsideration should include a copy of your original request and the Agency’s decision. The Agency will respond to all requests for appeals within the time frame specified in the procedure you use. Where a procedure does not specify a time frame for a response to your appeal, we will respond in a timely manner, in accordance with OMB and HHS Guidelines.

Sincerely,
Ellen J. Flannery 
Deputy Center Director for Policy 
Director, Office of Policy 
Center for Devices and Radiological Health

cc: Jeffrey Shuren, M.D., J.D. 
Timothy Stenzel, M.D., Ph.D. 
Mark Raza, J.D. 
Laurie Lenkel