METADATA last updated: 2026-03-04 by BA file_name: 2020-08-04_FDA Policy IVD - Sensitivity and Specificity Study Outline FULL STUDY.md file_date: 2020-08-04 title: FDA Policy IVD - Sensitivity and Specificity Study Outline FULL STUDY category: regulatory subcategory: fda-policy tags: source_file_type: docx xfile_type: docx gfile_url: https://docs.google.com/document/d/1SqJHxNoWQvg1EIY-Dr2ngahed4UIY2Oo xfile_github_download_url: https://raw.githubusercontent.com/FocusOnFoundationsNonprofit/floodlamp-archive/main/regulatory/fda-policy/2020-08-04_FDA%20Policy%20IVD%20-%20Sensitivity%20and%20Specificity%20Study%20Outline%20FULL%20STUDY.docx pdf_gdrive_url: https://drive.google.com/file/d/15aOYfuHk4JV44odELrt66CvA_carCuPm pdf_github_url: https://github.com/FocusOnFoundationsNonprofit/floodlamp-archive/blob/main/regulatory/fda-policy/2020-08-04_FDA%20Policy%20IVD%20-%20Sensitivity%20and%20Specificity%20Study%20Outline%20FULL%20STUDY.pdf conversion_input_file_type: docx conversion: pandoc license: Public Domain tokens: 2684 words: 1730 notes: has what looks like html or xml tags and non-pipe table format summary_short: The FDA Sensitivity and Specificity Study Outline (Full Study) is a step-by-step protocol for evaluating SARS-CoV-2 test performance using the FDA Verification Panel, including LoD determination/corroboration, dilution schemes, replicate testing, and blinded controls (T2–T5). It standardizes how developers generate and report Ct-based sensitivity/specificity results—plus a required Ct vs. log10 concentration plot—so FDA can compare new and previously authorized COVID-19 assays under consistent conditions. CONTENT ***INTERNAL TITLE:*** FDA Sensitivity and Specificity Study Outline – FULL STUDY ## Purpose: To provide the procedure to be used to evaluate **COVID-19** device sensitivity and specificity. Besides establishing an LoD for the device using the **COVID-19** strain supplied for this study, the study will enable FDA to compare sensitivity and specificity of new and previously authorized devices. ## Materials Provided: - FDA Verification Panel (FDA SARS-CoV-2 Reference Panel)– contains one heat-inactivated **COVID-19** strain and one heat-inactivated **MERS-CoV** in cell culture media obtained through collaboration with Center for Biologics Evaluation and Research (CBER). Panel contents: - T1 – concentrated stock 1 (USA-WA1/2020: NR-52281, concentration ~**1.8x10^8^** RNA NAAT detectable units/ mL) - T2 – control 1 (blinded testing) - T3 – control 2 (blinded testing) - T4 – control 3 (blinded testing) - T5– control 4 (blinded testing) *Note*: The controls in the validation panel were prepared to generate specific RNA NAAT detectable units/mL. These units may not be equivalent to units provided in copies/mL. *Note 2*: Please store the stock at -80°C until use. Prior to use, please thaw the stock at room temperature, prepare dilutions, and test the samples the same day. *Note 3*: Both vials of T1 are Sars-CoV-2. One vial of T1 is provided for the LoD determination study and another one for the LoD corroboration study. MERS-CoV is part of the unknowns. **Do not store diluted reference material over-night, or freeze-thaw the stock since it will produce a decrease in performance due to degradation.** ## Materials Not Provided: - Negative clinical matrix of choice *Note*: If several specimen types are claimed, please choose NP clinical matrix. If NP is not a specimen type claimed by your device, please contact the FDA. *Note 2*: For devices authorized for use with dry swab specimens only, please adjust the protocol for your assay as follows: -dilution series could be done using NP swabs collected from individuals negative for COVID-19 diluted in saline or other diluent appropriate for analytical testing for your particular device -prepare each dilution so there is sufficient volume to perform replicates as per FDA’s protocol below -test each dilution following the number of replicates requested by the FDA’s protocol below  For devices authorized for use with dry swab specimens only, prepare mock swabs by pipetting 50 μL of each diluted virus stock onto a swab, let the swab dry for a minimum of 20 minutes, and test the swab following the Instructions for Use for the device. Please provide a detail description of the workflow of your dry swab assay (including input volumes at each step). - Device to be evaluated may contain the following: - Real-time RT-PCR Primer and Probe Set - Real-time RT-PCR Control Set - Mastermix - Extraction method authorized (or to be authorized) with the new device - Instrument(s) to be authorized with the device *Note*: If you determine that one or more instrument(s) has/have a different LoD please select the extraction/ instrument combination with the worst LoD (anything that is within 1 to 2x of the established LoD in a side-by-side comparison is considered comparable). ## Study Outline for Negatives: To ensure that your matrix of choice will not interfere with the results produced by the reference material, you should first demonstrate that unspiked matrices generate valid *negative results* for all virus-specific primer and probe (i.e., are positive for your internal control and negative for SARS-CoV-2 targets). To this end, unspiked clinical matrix should be extracted 20 times. Each extracted nucleic acid matrix should be tested once by your assay before using the matrix for spiking as explained below. ## Study Outline for Sensitivity using T1: ### 1. Preparation of spiked NP samples for extraction: - To start, please prepare a 1:10 dilution of the provided material T1. For example: - Pipette 0.9 mL of NP matrix (swab eluted in VTM) into 1 vial labeled T1.1. - From the FDA Verification Panel stored at -80°C, remove T1 and thaw it at room temperature (RT), vortex and briefly centrifuge. - Transfer 0.1 mL of T1 to one of the vials -labeled T1.1- containing the 0.9 mL of clinical matrix. Total volume should now be 1 mL and the concentration in the order of ~1.8x10^7^ RNA NAAT detectable units/ mL. - Place lid on vial and vortex to mix. - Centrifuge briefly to prevent aerosol upon opening the tube. ### 2. Range-finding: - **For the NP clinical matrix** spiked with T1 as described in 1: - Prepare a 10-fold dilution series in matrix – 7 dilutions total for each (1.8x10^6^ down to 1.8x10^0^) in addition to T1.1 prepared in step 1. Change tips between dilutions. We suggest pipetting 4.5 mL of normal matrix into 7 additional vials labeled T1.2-T1.8. Transfer 0.5 mL of the T1.1 to the T1.2 vial, place lid on vial and vortex to mix. Centrifuge briefly to prevent aerosol upon opening the tube. Proceed with 1:10 dilutions until T1.8 is generated. - Extract T1.2-T1.8 three times each by the method authorized (or to be authorized) with the new device. - Test each of the extracted T1.2-T1.8 replicates with your assay. ### 3. Sensitivity Confirmation: - For each dilution series of virus T1 in clinical matrix, identify the lowest concentration for which all three replicates generate positive results. This is the target dilution for confirmation for each virus. - For the next testing step (confirmatory testing) you should test the target dilution(s) identified in the previous step plus one 3-fold dilution above and below the identified target dilution. Additional dilutions need to be made as applicable until at least 1of the 20 samples generate a negative result. If in the confirmatory testing none of the levels reaches at least 95% positivity, you will need to go further up in the dilution scheme until you can show at least 95% with 20 replicates (19/20 positive results). - Using the same extraction method used in the range finding study that has been authorized (or to be authorized) with the device: - For the targeted dilutions for confirmation, if the confirmation is performed the same day of the range finding, 17 additional replicates should be tested from extraction to amplification/detection. Please add any triplicates from the range finding to the 17 replicates of the confirmatory study as applicable. If confirmation is performed on a following day, 20 replicates should be tested from extraction to amplification/detection for the targeted dilution to limit variability in Ct values. - For the two dilutions bracketing the targeted level for confirmation, 20 replicates should be tested from extraction to amplification/detection. - Test each individually extracted nucleic acid sample once by your assay. ### 4. Graphical Summary: - Please provide a plot of Mean Ct (with error bars) versus log~10~ (RNA NAAT detectable units/ mL). **Study Outline for Blinded Controls using T2-T5**: - Prepare a 1:10 dilution and a 1:100 dilution of provided material T2 to T5 in NP matrix as shown in Figure 1 and test the dilutions in red. Figure 1. Panel Dilution Scheme _Diagram showing the panel dilution scheme: serial dilution of SARS-CoV-2 reference material into multiple tubes at decreasing concentrations for limit of detection determination and confirmation._ For 10mL final volume: - Pipette 9 mL of normal human matrix into eight vials; designate two vials for each blinded control T2 (T2 and T2.1), T3 (T3 and T3.1), T4 (T4 and T4.1), and T5 (T5 and T5.1). - From the FDA Verification Panels stored at -80°C, remove T2-T5 and thaw them at RT, vortex and briefly centrifuge. - Transfer 1 mL of each of T2-T5 to its designated vial containing clinical matrix. Total volume should now be 10 mL in each vial. - Place lid on vial and vortex to mix. - Centrifuge briefly to prevent aerosol upon opening the tube. - Use these final T3 and T4 (all 1:10 dilutions) dilution for testing. - For T2, T3, T4 and T5 (all 1:10 dilutions), transfer 1 mL of T2 into the vial labeled T2.1., 1 mL of T3 into the vial labeled T3.1, 1 mL of T4 into the vial labeled T4.1, and 1 mL of T5 into the vial labeled T5.1 to generate the 1:100 dilution. Use these final T2.1, T3.1, T4.1 and T5.1 (all 1:100) dilutions for testing. - Place lid on vial and vortex to mix. - Centrifuge briefly this vial to prevent aerosol upon opening the tube. - For each spiked matrix with T2.1 (1:100), T3 (1:10), T3.1(1:100), T4 (1:10), T4.1 (1:100) and T5.1 (1:100): - Extract each control at least five times using the extraction method authorized (or to be authorized) with the new device. You will have half of the preparation unused in this case, - Test the diluted specimens with your assay. - For each Ct value obtained for the blinded controls (30 values in total), please provide an estimated value of RNA NAT detectable units/ mL by comparing the Ct values of the controls to the corresponding 10-fold dilution with known concentrations in units/ mL used to construct the graph above. *Note*: Please do not include the dilution factor in your estimation; provide the NAAT detectable units as measured in the sample tested. *Note* 2: For a more robust study, please extract and test 10 times T2.1, T3, T3.1, T4, T4.1, and T5.1. This will result in 60 blinded testing values in total. In this case, you will use all the volume prepared above. *Note 3*: Please run all the unknowns on the same day. *Note 4:* Please provide individual Ct values, average Ct values, error of the mean Ct values, and number of positive hits over total replicates in each step of the study. *Summary of the vials provided:* | Tube # | Volume (mL) | Number of tubes | Dilution to be tested | | --- | --- | --- | --- | | 1 | 0.6 | 2 | Serial dilution: one vial is for LoD determination and another one for LoD confirmation | | 2 | 1.5 | 1 | 1:100 | | 3 | 1.5 | 1 | 1:10 and 1:100 | | 4 | 1.5 | 1 | 1:10 and 1:100 | | 5 | 1.5 | 1 | 1:100 | || *Note: The material cannot be used for other purposes than the ones specified in this protocol.* For additional questions, please contact Mayra Garcia, Ph.D., M.B.A., at (240) 402-7213 or at [Mayra.Garcia@fda.hhs.gov](mailto:Mayra.Garcia@fda.hhs.gov)