Methodological problems of SARS-CoV-2 rapid point-of-care tests when used in mass testing

The aim of the current study is to perform model calculations on the possible use of SARS-CoV-2-rapid point-of-care tests as mass tests, using the quality criteria extracted from evidence-based research as an example for the Federal Republic of Germany. In addition to illustrating the problem of false positive test results, these calculations are used to examine their possible influence on the 7-day incidence. For a substantial period of time, this parameter formed the decisive basis for decisions on measures to protect the population in the wake of the COVID pandemic, which were taken by the government. Primarily, model calculations were performed for a base model of 1,000,000 SARS-CoV-2-rapid point-of-care tests per week using various sensitivities and specificities reported in the literature, followed by sequential testing of the test positives obtained by a SARS-CoV-2 PCR test. Furthermore, a calculation was performed for an actual maximum model based on self-test contingents by the German Federal Ministry of Health. Assuming a number of 1,000,000 tests per week at a prevalence of 0.5%, a high number of false positive test results, a low positive predictive value, a high negative predictive value, and an increase in the 7-day incidence due to the additional antigen rapid tests of approx. 5/100,000 were obtained. A previous maximum calculation based on contingent numbers for self-tests given by the German Federal Ministry of Health even showed an additional possible influence on the 7-day incidence of 84.6/100,000. The model calculations refer in each case to representative population samples that would have to be drawn if the successive results were comparable which should be given, as far-reaching actions were based on this parameter. The additionally performed SARS-CoV-2-rapid point-of-care tests increase the 7-day incidence in a clear way depending on the number of tests and clearly show their dependence on the respective number of tests. SARS-CoV-2-rapid point-of-care tests as well as the SARS-CoV-2-PCR test method should both be used exclusively in the presence of corresponding respiratory symptoms and not in symptom-free persons.


Supplementary Material S2 Additional explanations regarding incidence values A) Explanation for lower incidence values than predicted:
We have developed a mathematical model which has predicted higher values than are reported at the end of June 2021. The reason for this discrepancy is most likely that several shortcomings of both the actual PCR test procedures and the way the data are processed by the Robert Koch Institute (RKI) could not be incorporated in our model calculations. In detail: Problem 1: There is no standardization of the PCR test procedure carried out in the 200 plus laboratories, neither is there a uniform cycle threshold Ct value to render a test positive. This will result in a large variability in the test results, with the most prominent effect on the false positive rate and the positive predictive value PPV. A good illustration of this state of affairs is found in a publication by the RKI in which the set of average positive test value rates per calendar week for each laboratory is visualized in a boxplot, as shown in Figure S1: Figure S1. Fraction of SARS-CoV-2 PCR-tests, calendar weeks 12/2020 to 5/2021, own graphical representation, source: Epidemiologisches Bulletin 6/2021, RKI, 11th February 2021.
The most striking feature for all calendar weeks is that the distribution of percentages is asymmetric for all calendar weeks. The asymmetry is the more pronounced the lower the incidence (as e.g. in CW 21 to 41). At the time of developing our mathematical model (March 2021), the incidence has been high, as of June 2021 the situation is similar to CW 21 to 41 in 2020.
Based on this, two conclusions are possible: 1) It appears unlikely that the distribution of positive cases is really asymmetrically distributed.
Rather, the most likely reason is the large variance in the tests which are not standardized. This corresponds to industrial production processes with a large process variance which also typically produce asymmetric distributions due to small defects, whereas stable processes tend to result in narrower normal distributions.
2) The degree of asymmetry influences the mathematical model in a significant way, since the averages are dependent on the asymmetry. This was not accounted for in our model since the data supplied by the authorities are not sufficient to do this.
Therefore, we argue that the simplification of the model using averages instead of full distributions has led to the too high prediction of the minimum attainable incidence number. Another possible reason is the simplification of using the average values for Germany rather than the values for the 16 federal states. Figure S2 shows the striking fact that at a time for which the incidence is low there is a high systematic spread of the incidence: Figure S2. Incidence per federal state on June 29 th , 2021, own graphical representation, Source: Daily situation report RKI of June 25 th , 2021.
At the end of June there is a factor of approximately 5 scatter of the incidences among the 16 federal states. Therefore, the geographic variation of the measured incidence is another factor which can have an impact on the prediction of the minimum possible incidence.

Problem 2:
The model calculations refer in each case to additional representative population samples that would have to be drawn if the results were comparable. This would be the correct epidemiological procedure. However, the testing strategy behind the test results is unclear. Due to government regulations, massive testing had to take place in settings where low prevalence was expected (schools, workplace, etc.). Testing in environments with lower prevalence of the disease is expected to have a smaller additional impact on the 7-day incidence (see Table S1). However, this does not weaken our conclusions, which refer to the standardized procedure of testing representative population samples from a population with a defined prevalence. One could argue that the government's testing strategy would then probably underestimate the actual 7-day incidence. Here, however, the discussion arises again with regard to the clinical significance of a representative 7-day incidence, which is to a considerable extent based on a SARS-CoV-2 PCR test with Ct values that are clearly too high, so that this parameter is not able to reflect the actual incidence of infection. Table S1. Additional increases in 7-day incidence per 100,000 depends on different prevalences of COVID-19. Maximum calculations using the contingent data for self-tests from the German Federal Ministry of Health are performed using 16,500,000 tests per week and sequential SARS-CoV-2 PCR testing.

B) Positive Proof of an incidence increase by introduction of POC rapid antigen tests:
Although the accuracy of the mathematical model presented is not as high as expected (for the reasons explained above), the qualitative prediction is correct as shown by recent data published by the RKI. These constitute proof that as predicted the introduction of rapid antigen tests as documented in the weekly reports of the RKI lead to an increase of positive PCR tests. Figure S3 taken from the weekly report dated June 1 st , 2021, shows a clear increase in the number of positive PCR tests at the onset of rapid antigen tests at schools: Figure S3. Percentage of positive PCR tests for different age groups as a function of calendar week. Own graphical representation, source: Laboratory-based surveillance of SARS-CoV-2, weekly report of 01.06.2021.
It is clear that as of calendar week 14 the percentage of positive tests for the age group 5 to 14 jumps from about 12% to approximately 19% and has remained higher than the values for all other cohorts, which are not subject to regular rapid antigen tests.