While there is no specific treatment or vaccine for COVID-19, a recent study has suggested the potential effectiveness of an existing vaccine, the Bacille Calmette-Guérin (BCG) vaccine, against COVID-19.

 The BCG vaccine is used in some countries for the prevention of tuberculosis (TB), a bacterial infectious disease caused by Mycobacterium tuberculosis (Mtb), which accounted for 1.5 million deaths globally in 2018. National BCG vaccination policies vary across the world, with universal BCG vaccination at birth still occurring in high TB-burden countries such as India and Ethiopia, while countries with lower overall TB incidence have largely discontinued universal BCG vaccination in favour of targeted vaccination of specific TB risk groups (such as children from families immigrating from high TB burden countries). The BCG World Atlas, created by the McGill International TB Centre, provides data on BCG policies and practices around the world.

The study

For the study by Miller et al, BCG vaccination policies across countries were collected from the BCG World Atlas. Data of COVID-19 cases and death per country were obtained on March 21, 2020, from https://google.org/crisisresponse/covid19-map. The ecological analysis found an association between COVID-19 death rates and universal BCG vaccination policies among middle and high income countries. The authors also observed an association between COVID-19 cases per capita and BCG vaccination policies, with countries with a universal BCG vaccination policy having lower cases per capita and death rates. Based on the analysis, the authors state “The correlation between the beginning of universal BCG vaccination and the protection against COVID-19 suggests that BCG might confer long-lasting protection against the current strain of coronavirus.”

Critique

Study Design

The main issue with this study lies in its design. Ecological studies often conflate population level exposures, such as country-wide policies, with exposures at the individual level, such as the effect of the BCG vaccine on a human body. What may hold at the aggregate level, will not necessarily be true when the heterogeneity of individuals is considered. In this study, low quality evidence observed at the population level is used to make sweeping inferences about BCG’s effectiveness on an individual level. Ecological analysis techniques simply cannot directly inform etiology of exposure/disease relationships; rather, they serve us well as strictly hypothesis-generating queries and should not be stretched beyond this purpose.

Timing matters

Since the analysis (COVID-19 data were downloaded on March 21, 2020), the pandemic has escalated in many countries, including low and middle income countries (LMICs) where BCG is given at birth. For example, COVID-19 cases in India have increased from 195 on March 21  to 1071 on March 31. In South Africa, cases have increased from 205 on March 21 to 1326 on March 31. Therefore, the paper’s findings could look very different as countries progress in their epidemic timelines (and expand diagnostic capacity). For example, once we see the full effects of community transmission in large, densely-populated countries such as India, where large numbers of day-labourers are now leaving cities after being put out of work amid the country-wide lockdown. This underlines that drawing premature conclusions from rapidly changing data in a pandemic is problematic.

Lack of testing

Accurate analysis is dependent on accurate data and we know that the current number of coronavirus cases is dramatically underestimated around the world due to shortages of diagnostic tests. The number of tests performed per capita varies wildly by country. Underestimation of the number of cases in lower income countries could entirely explain the authors’ observed results. India, for example, has one of the lowest COVID-19 testing rates in the world. Additionally, countries have each applied their own algorithm to determine who gets tested. Thus, the number and composition of known cases is heavily influenced by testing strategy. The authors of this paper acknowledge the concerns surrounding current estimates of COVID-19 cases and also use COVID-19 deaths as a measure of the severity of each countries’ outbreak. Unfortunately, even current COVID-19 death data are unreliable in some parts of the world. In wealthy countries, it is true that most deaths due to COVID-19 will occur in a hospital after having received a diagnosis of COVID-19. However, we know that in LMICs, even before this current pandemic, most deaths occurred at home without medical assistance. People dying at home in LMICs are unlikely to receive a confirmed diagnosis of COVID-19 and may not have their death accounted for in government records. Coronavirus deaths may also be missed in high-income countries that have been overwhelmed by the virus, like Italy. It is very difficult to draw conclusions about any potential impact of BCG vaccination when the data are so flawed.

Correlation is not causation

A critical flaw in the methodology of this study is apparent in the fact that a perceived relationship between an exposure and an outcome does not mean one causes the other - a concept often expressed through the well-known adage “correlation does not imply causation.” This can occur in situations where other underlying factors influence both the exposure and the outcome and thereby create a spurious association between the two (in epidemiology, this situation is referred to as confounding). Specifically, in the case of the relationship between universal BCG vaccination and the apparent lower COVID-19 case numbers and COVID-19-associated deaths, a multitude of underlying factors are at play. The crucial take-away here is that if these factors could be appropriately accounted for, we would very likely not see the association between BCG vaccination and protection against COVID-19 suggested by this paper. The concerns about data quality now become a potential source of confounding in the relationship between BCG vaccination and lower COVID-19 case numbers. It is likely that under-funded health systems in (LMICs) have low COVID-19 testing capacity, and at the same time, are more likely to have universal BCG vaccination policies still in place, given that TB burden is highest in LMICs. Therefore, the study actually does not demonstrate that populations in countries with universal BCG vaccination policies are somehow more protected against COVID-19, because lower case numbers and deaths may be more attributable to low testing capacity than to any supposed protectiveness of BCG. Therefore, statements in the paper that imply causality, such as the statement that “BCG vaccination also reduced the number of reported COVID-19 cases in a country” are dangerously misleading, particularly in a context where reported case numbers are in large part a function of testing capacity and testing strategy, rather than actual relative burden. 

A further important consideration is that it is difficult to draw causal conclusions across a broad range of countries with varying underlying demographic characteristics, such as differences in the age distribution of the populations of interest. This is particularly true of diseases for which severity differs by age. As shown in a recent study from Wuhan, China, older individuals are at higher risk for severe COVID-19, so differences in the age distribution of populations may significantly influence death rates observed across countries (for example, the WHO regions of South-East Asia and Africa have a median age of 27.0 and 18.7 years, respectively, whilst the population in the European region is much older (median 38.6 years), which may in part contribute to the higher death rates observed in Europe). In light of these considerations, given that no age-standardization method was used in the Miller et al. study, the strong limitations of the inferences made need to be emphasised.

Biological plausibility

Lastly, some of the “common sense” appeal of this paper’s claim is based on a bit of a false equivalence. The authors twice mention that BCG protects broadly against respiratory diseases. The idea that a vaccine for an infectious disease of the lungs could also protect against other infectious diseases of the lungs seems somewhat consistent and thus palatable. This hits a snag though, when we look at the type of TB against which BCG most effectively protects. Due to an unfortunate natural experiment wherein there was a global shortage of the BCG vaccine from 2013 to 2015, retrospective data show there was a surge in cases of TB meningitis, that is, TB of the central nervous system. The tidy idea of a lung disease vaccine protecting against another lung disease, i.e. COVID-19, is not quite so straight-forward, since BCG seems to offer most significant protection against TB meningitis. Additionally, it is well-established in the TB field that the BCG vaccine only confers protection against TB in young children, and that protection almost completely wanes by age 12. Recent studies have even examined whether applying a booster vaccination to older adolescents or adults would help extend this anti-TB protection. The important take-away is that BCG’s protective effect fades over time. So it is not clear how a vaccine that offers protection early in an individual’s life would protect individuals against COVID-19 once they are of an advanced age.

Conclusions

From the latest piece in Forbes to Bloomberg, the study’s findings are gaining more and more traction. There is danger in citing that there is evidence that a century-old vaccine may boost immunity in individuals, providing non-specific protection to other illnesses, and by extension protecting against COVID-19 or reducing severity of its presentation based on this analysis alone. Accepting these findings at face value has the potential for complacency in response to the pandemic, particularly in LMICs. One needs only to look at how this has been portrayed in news outlets of several LMICs already; the dangers of such portrayals misinforming the public should not be underestimated, for example, in  countries such as India, the wide BCG coverage offered by their universal vaccination policy may create  a false sense of security and lead to inaction. If claims such as these, based on insufficient evidence, interfere with pandemic response,  the most vulnerable populations will be those hardest hit by the consequences. 

With further research, it may emerge that the BCG vaccine does confer protection against COVID-19; however, with the current state of knowledge we cannot state this with any degree of certainty, and an ecological study does not provide sufficient evidence. More basic scientific research and appropriately designed trials are needed to parse out this relationship. An Australian trial is set to start imminently, aiming to investigate whether BCG vaccination protects against COVID-19 or reduces severity of COVID-19 in Australian healthcare workers. If, in a well-performed trial, we do in fact see  a benefit in re-purposing this vaccine, the evidence would be much more convincing.