Abstract
Background: Rapid antigen tests (RATs) were crucial during the COVID-19 pandemic. Information provided by the test manufacturer in product package inserts, also known as instructions for use (IFUs), is often the only data available to clinicians, public health professionals, and individuals on the diagnostic accuracy of these tests. We aimed to assess whether manufacturer IFU accuracy data aligned with evidence from independent research.
Methods: We searched company websites for package inserts for RATs that were included in the July 2022 update of the Cochrane meta-analysis of SARS-CoV-2 RATs, which served as a benchmark for research evidence. We fitted bivariate hierarchical models to obtain absolute differences in sensitivity and specificity between IFU and Cochrane Review estimates for each test, as well as overall combined differences.
Findings: We found 22 (100%) of 22 IFUs of the RATs included in the Cochrane Review. IFUs for 12 (55%) of 22 RATs reported statistically significantly higher sensitivity estimates than the Cochrane Review, and none reported lower estimates. The mean difference between IFU and Cochrane Review sensitivity estimates across tests was 12·0% (95% CI 7·5–16·6). IFUs in three (14%) of 22 diagnostic tests had significantly higher specificity estimates than the Cochrane Review and two (9%) of 22 had lower estimates. The mean difference between IFU and Cochrane Review specificity estimates across tests was 0·3% (95% CI 0·1–0·5). If 100 people with SARS-CoV-2 infection were tested with each of the tests in this study, on average 12 fewer people would be correctly diagnosed than is suggested by the package inserts.
Interpretation: Health professionals and the public should be aware that package inserts for SARS-CoV-2 RATs might provide an overly optimistic picture of the sensitivity of a test. Regulatory bodies should strengthen their requirements for the reporting of diagnostic accuracy data in package inserts and policy makers should demand independent validation data for decision making.
Funding: None.
Methods: We searched company websites for package inserts for RATs that were included in the July 2022 update of the Cochrane meta-analysis of SARS-CoV-2 RATs, which served as a benchmark for research evidence. We fitted bivariate hierarchical models to obtain absolute differences in sensitivity and specificity between IFU and Cochrane Review estimates for each test, as well as overall combined differences.
Findings: We found 22 (100%) of 22 IFUs of the RATs included in the Cochrane Review. IFUs for 12 (55%) of 22 RATs reported statistically significantly higher sensitivity estimates than the Cochrane Review, and none reported lower estimates. The mean difference between IFU and Cochrane Review sensitivity estimates across tests was 12·0% (95% CI 7·5–16·6). IFUs in three (14%) of 22 diagnostic tests had significantly higher specificity estimates than the Cochrane Review and two (9%) of 22 had lower estimates. The mean difference between IFU and Cochrane Review specificity estimates across tests was 0·3% (95% CI 0·1–0·5). If 100 people with SARS-CoV-2 infection were tested with each of the tests in this study, on average 12 fewer people would be correctly diagnosed than is suggested by the package inserts.
Interpretation: Health professionals and the public should be aware that package inserts for SARS-CoV-2 RATs might provide an overly optimistic picture of the sensitivity of a test. Regulatory bodies should strengthen their requirements for the reporting of diagnostic accuracy data in package inserts and policy makers should demand independent validation data for decision making.
Funding: None.
| Original language | English |
|---|---|
| Journal | The Lancet Microbe |
| Early online date | 13 Oct 2023 |
| DOIs | |
| Publication status | E-pub ahead of print - 13 Oct 2023 |