From the outset of the COVID-19 outbreak, testing for SARS-CoV-2 has been a priority. Initially just having a test for COVID-19 was a scientific success, enabled by the publication of the first viral genomic sequence in January 2020. As the pandemic has run its course many aspects of testing have been investigated. Here we highlight some of the key testing events that have taken place, and their outcomes.

Mass Covid Testing

Vo’, Italy – Mass testing in this Italian town was triggered by the reporting of the first COVID-19 death. With a population of 3275, Vo managed to test 86% in a first testing round, and 72% in a second round 2 weeks later. 2.6% were RT-PCR positive in the first round of tests, reducing to 1.2% in the second round. The organiser stated that ‘early identification of clusters and timely isolation of people testing positive suppressed transmission and curbed the epidemic’.

Wuhan – This COVID-19 mass testing event was orchestrated after the initial reactions to SARS-CoV-2 had been implemented and a stringent enforced lockdown had already successfully limited COVID-19 cases. In the last 2 weeks of May 2020 the nearly 10 million residents of Wuhan over 6 years old were tested using RT-PCR; pooled sensitivity was 73%, but no information provided for specificity. Asymptomatic cases reported numbered 300, with no new symptomatic cases found. Contact tracing of the 300 infected showed no cases of transmission. Pooling of five samples at a time were used to increase efficiency in some 23% of the samples. If a pooled sample was positive, all samples within the pool were re-tested as single samples.

Luxembourg – Here, RT-PCR was performed on pooled samples. Reportedly, sensitivity and specificity of the test protocol was 100%. Overall, 49% of the residential population were tested as well as 22% of cross border workers, totalling over half a million people. 850 positive cases were found and a further 249 identified through contact tracing. The study indicated that asymptomatic carriers are at least as infectious as symptomatic patients. It was reported that containment of future outbreaks will critically depend on early testing in sectors and geographical regions. Higher participation rates must be achieved, using targeted incentives and invitations.

COVID-19 Testing Site

 

Slovakia – 20000 medical and 40000 non-medical staff were used to administer 3 rounds of testing at the end of October 2020. The first round was a pilot and high prevalence areas were re-tested in the final round. In all, 5 million tests were performed, covering over 80% of the population. Samples taken by medical professionals used the SD Biosensor lateral flow rapid test, approved by WHO with a sensitivity of 95.5% (70-90% in the study) and specificity 99.2%. No confirmatory RT-PCR tests were perfomed. Over 50000 SARS-CoV-2 cases were positively identified, approximately 4% in the pilot, 1% in round 1 and 0.6% in round 2.

Qingdao, China – The events leading to the mass testing in Qingdao came from three people testing positive using RT-PCR, leading back to 2 asymptomatic dock workers whose transmission path couldn’t be followed further. Over 3 weeks nearly 11 million individuals over 5 years old were tested. To cover this many people pooling was used. The result was the identification of 12 positive cases. Public transport was only available to those with evidence of a negative test. No information was published on the specificity and sensitivity of the RT-PCR test.

Liverpool, UK – The objective was ‘to demonstrate massive asymptomatic cases can help identify far more cases and break the chain of transmission.’ Between the 6th and 26th November 108304 lateral flow tests (LFT) were carried out. Most tests were self-administered (military supervised) lateral flow tests from Innova. Some tests were repeated for quality assessment purposes with samples collected professionally and confirmed using PCR testing. Test sensitivity was substantially reduced depending on who carried out the tests; 78% when used by trained professionals, but only 58% when used by self-trained staff. Specificity was reportedly 99.93%, meaning false positives would be expected to be rare. Key workers, health and social care staff, school staff, and children aged 11 and over were targeted, but anyone could get tested, preferably at least twice within two weeks.

 

Table: Summary of SARS-CoV-2 mass testing events.

COVID Test Types

The variety of tests for SARS-CoV-2 are relatively well known thanks to media reporting.

The most sensitive is the RT-PCR test, we have described this in previous articles. RT-PCR uses engineered primers to amplify fragments of viral RNA present in the specimen and requires a laboratory. Sensitivity is high with figures ranging between 71-98%. However, this variation depends on multiple factors, including the diligence of sample collection, test type, swab type and time for which the sample is left in storage. The RT-PCR based mass testing events in China, taking place in a background of a small number of cases in the community, were seemingly effective in ending the outbreaks. 

Lateral flow tests to detect SARS-CoV-2 antigen came to the market more recently than RT-PCR tests, due to the fundamental timelines inherent in developing a new test of this type. However,  the technology has been around for years and is used in all manner of diagnostic tests, the most well known being pregnancy tests. Lateral flow tests  are relatively cheap to produce and can be tested on site with a result time varying between 5-30 minutes. This saving of time and money makes them an attractive option. Theoretically, they are extremely simple to perform and have the potential to be used by untrained individuals in any location, if the test protocol and instructions for use are simple enough. For SARS-CoV-2 specifically, some tests have now been approved by the US FDA for home-testing (Ellume and BINAX).

COVID-19 negative Lateral Flow test result

 

Lateral flow tests can also be used to identify human anti-SARS-CoV-2 antibodies present in samples, sometimes known as antibody tests or serology tests. However, these can not be used to determine if an individual has a current SARS-CoV-2 infection, only if the individual has ever had an infection. These tests have potential to be used to identify individuals who have already been infected and built up some immunity, and could therefore be a lower priority for vaccination. However, this would be a controversial use of such tests and no country has thus far decided to do this, despite the current context of a shortage of vaccine doses and an urgent drive to protect as many vulnerable people as possible. 

Mass testing analysis

The success of mass testing events is challenging to ascertain; the testing events are ambitious and complex operations featuring innumerable variables. In particular, where mass testing events are carried out alongside a “lockdown”, it is difficult to separate the results of the two and thus understand which element was responsible for any subsequent reduction in infections. It can be said that each mass testing event, in coordination with a lockdown, appeared to reduce numbers of positive covid tests, at least in the short-term.

Overall, RT-PCR is accurate but time consuming and costly. Pooling testing is a cost effective method of testing large numbers using RT-PCR. Lateral flow tests are cheap to produce and give fast results but lack accuracy. Multiple tests performed on the same individuals at different time points could help improve the accuracy. 

The experience of self-administered tests in Liverpool versus the professionally administered tests carried out China and Slovakia highlights the importance of medical expertise. Samples collected by trained professionals provide more accurate results than samples gathered by individuals. The new FDA approved Ellume and Abbott Binax tests are the first home test kits available, reporting sensitivity and specificity of 92% and 100%, respectively, and results within 20 minutes. But their use comes with a caveat in the form of a “Telehealth Proctor”. At present even the most user friendly tests available require supervision for best results; it remains to be seen whether they can be simplified to the extent that they can be performed effectively without this. 

The lateral flow tests used in Liverpool, were found to only confirm half of all COVID-19 cases identified by RT-PCR tests. It has been suggested this is understandable in low viral load cases, which may be less infectious, but it also missed 3 of 10 in higher viral load cases (based on Ct values below 25). These tests have been strongly criticised and called unfit for purpose by some,  but from a public health perspective it can be argued it is better to find 50% of cases rather than none. The UK government has announced more mass testing in COVID-19 high prevalence areas and it remains to be seen how it will modify its approach using the learning points from the Liverpool testing. 

Mass testing is an important tool in defeating SARS-CoV-2. The logistical challenges involved are exceedingly complex (and expensive) and few governments have thus far attempted them. If sensitive and user-friendly lateral flow tests can be developed we expect to see great demand for public and private sector test settings such as schools, airports, universities, care homes and cruise ships.  

Logical Biological provides swabs/serum/plasma for use in test development

At Logical Biological we supply nasopharyngeal/oropharyngeal/nasal SARS-CoV-2, FluA and Flu B swabs available in UTM, Inactivating TM, saline or dry frozen. Negative swabs and COVID-19 / pre-COVID saliva are also available. Typically, our swabs are provided together a Ct value measured from a ‘companion swab’ taken simultaneously. We also provide serum and plasma samples from individuals infected with SARS-CoV-2 and other respiratory infectious diseases.

Swabs table
Table: Products available from Logical Biological

We are in the midst of the most significant global pandemic since the 1918-19 Influenza pandemic, over 100 years ago. The 2020 pandemic has been caused by a Coronavirus, named SARS-CoV-2, which confers a severe respiratory illness (COVID-19) on a proportion of those infected. The virus is readily transmissible from human to human with many of those infected showing no or mild symptoms, meaning it is hard to know who has been infected.

The virus has resulted in severe economic impacts because many countries have adopted “lockdown” policies in order to limit its spread by limiting interaction between individuals. One idea for mitigating some of the economic impact has been to identify and liberate from lockdown those individuals who have already been infected by the virus and may therefore be immune from future infections. As yet, it is not known for sure if previous infection by the virus renders individuals immune from future viral challenge, nor how long such immunity would last for. However, for this idea to be viable, diagnostic tests that can identify the SARS-CoV-2 antibodies in the human blood are required. Such tests have been made by innumerable manufacturers but they vary greatly in their performance.

 

At what stage are antibodies exhibited?

To understand the value of SARS-CoV-2 antibody tests we need to know who exhibits what antibodies, and when. It is assumed that the vast majority of individuals will have a detectable antibody response, regardless of whether or not they are symptomatic. The below data suggests IgG and IgM antibodies are detectable 1-3 weeks after SARS-CoV-2 symptom onset.


Detection of IgG, IgM and Neutralising (NT) antibodies over time since symptom onset.
Taken from a pre-print published online by Borremans et al., (2020)

 

Why 100% test specificity is essential for SARS-CoV-2

Logical Biological received some CE-marked SARS-CoV-2 IgG and IgM antibody tests a few weeks ago (we won’t name the manufacturer). In the pack insert these tests reported 97% specificity, which sounds high, but what does it mean in the context of the proportion of people who actually have been infected in the population? Specificity can be defined as the number of true negative cases that actually return a negative test result.

As can be seen in the tables below, if 1 person is selected at random and tests positive using a 97% specificity test when the prevalence within the population is low, it is much more likely that a positive test result is a false positive than a true positive.

Even with a 99.6% specificity test, such as those now available in a leading UK high street pharmacist, there is a clear threat of false positives, the proportion of false positives to actual positives reducing as the true number of infected within the population increases. One serology study, performed in Santa Clara, California, received criticism that the specificity of the test was too low, at only 99.5%. Only with 100% can we be fully confident that the positive test result is a true positive. Fortunately, such tests are now becoming available, such as one developed by Ortho Clinical Diagnostics.

The tables below consider a random selection of the population. Individuals may feel more confident about a positive result they receive if they have also shown the classical symptoms.

97% Specificity

% of population prev. infected Test Specificity True positives (per 1000) Expected False positives (per 1000)
0.1% 97.0% 1 30
1% 97.0% 10 30
10% 97.0% 100 30
100% 97.0% 1000 30

 

99.6% Specificity

% of population prev. infected Test Specificity True positives (per 1000) Expected False positives (per 1000)
0.1% 99.6% 1 4
1% 99.6% 10 4
10% 99.6% 100 4
100% 99.6% 1000 4

 

100% Specificity

% of population prev. infected Test Specificity True positives (per 1000) Expected False positives (per 1000)
0.1% 100.0% 1 0
1% 100.0% 10 0
10% 100.0% 100 0
100% 100.0% 1000 0

 

Sensitivity

Sensitivity of a test can be defined as the proportion of those genuinely bearing a marker (such as SARS-CoV-2 antibodies) who test positive for it. In the context of SARS-CoV-2 antibody tests, results from low sensitivity tests are less “dangerous” than results from low specificity tests. Low specificity tests could result in a situation where susceptible individuals who have wrongly tested positive, believing themselves to be both immune and non-infective, stop taking precautions to protect themselves and others, leading to further infections. On the other hand, low sensitivity tests would likely result in previously-infected individuals who have wrongly tested negative continuing to be cautious and since they have already had the infection would not be able to contribute to further spread in any case.


Table shows theoretical results of a 97% sensitivity test

% of population prev. infected Test sensitivity True positives (per 1000) Expected positives based on 97% sensitivity
0.1% 97.0% 1 0.97
1% 97.0% 10 9.7
10% 97.0% 100 97
100% 97.0% 1000 970

 

 

How many people have actually been infected by SARS-CoV-2?

One of the countries most affected by the pandemic has been Spain. The government of Spain has recently performed a serological study (results published on 13th May 2020) where they assessed the blood of 70,000 individuals. The most affected province showed 14.2% positive tests whereas the least affected regions were at less than 2%. The overall figure for previously-infected individuals in Spain was assessed to be approximately 5%. The test looked for both IgG and IgM antibodies. In the context of the above information, it should be noted that the test used was reported to show 100% specificity and 79% sensitivity for IgG. This means it would miss 21% of those previously infected and also may miss some people in the early stages of infection. It was wise of them to choose a 100% specificity test, and in the context of a serological survey to assess prevalence within a large population, the results can be adjusted to account for the low sensitivity of the test.

Data from other countries is in line with that from the Spanish study. For example, study results announced on April 23rd from another of the world’s major hotspots, New York State, USA, found 21% of people to be antibody positive in New York City. High figures (10-20%) were also seen in other areas while outside of the most-affected areas in the state the average prevalence was 3.6%.

 

Conclusion

The prevalence of SARS-CoV-2 in some locales is 2-20%. At the upper end of this range the % of positive tests that would be false when using a 97% specificity test would be significant and unacceptably high. At the low end of this range there would be more false positives returned than true positives, rendering such a test completely useless. Beware SARS-CoV-2 antibody tests with <100% specificity.

 

Available from Logical Biological

  • SARS-CoV-2 PCR positive serum/plasma/swabs
  • SARS-CoV-2 IgG positive serum/plasma – 1ml samples and bulk units
  • SARS-CoV-2 IgM positive serum/plasma – 1ml samples and bulk units
  • Pre-Covid19 serum/plasma from normal healthy donors – any quantity

Due to the dynamic nature of the SARS-CoV-2 pandemic this blog post will be out of date shortly after it is written.