Relationship Between Coronavirus Disease 2019 Vaccination Rates and Rare But Potentially Fatal Adverse Events: A Regression Discontinuity Analysis of Western Countries

Seung Hoon Chae; Hyung Jun Park; Munkhzul Radnaabaatar; Hojun Park; Jaehun Jung

doi:10.3346/jkms.2023.38.e94

Journal List > J Korean Med Sci > v.38(11) > 1516081840

Go to TopGo to Top Go to BottomGo to Bottom

TOOLS

Chae, Park, Radnaabaatar, Park, and Jung: Relationship Between Coronavirus Disease 2019 Vaccination Rates and Rare But Potentially Fatal Adverse Events: A Regression Discontinuity Analysis of Western Countries

Original Article

Global Health

Journal of Korean Medical Science 2023; 38(11): e94.

Published online: 9 March 2023

DOI: https://doi.org/10.3346/jkms.2023.38.e94

Relationship Between Coronavirus Disease 2019 Vaccination Rates and Rare But Potentially Fatal Adverse Events: A Regression Discontinuity Analysis of Western Countries

Seung Hoon Chae^1,^*

, Hyung Jun Park^2,^*

, Munkhzul Radnaabaatar³

, Hojun Park^3,⁴

, Jaehun Jung^3,⁴

¹Department of Politics and IR, University of Oxford, Oxford, United Kingdom.

²Department of Pulmonary and Critical Care Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea.

³Artificial Intelligence and Big-Data Convergence Center, Gil Medical Center, Gachon University College of Medicine, Incheon, Korea.

⁴Department of Preventive Medicine, Gachon University College of Medicine, Incheon, Korea.

Address for Correspondence: Jaehun Jung, MD, PhD. Department of Preventive Medicine, Gachon University College of Medicine, 38-13 Dokjeom-ro 3-beon-gil, Namdong-gu, Incheon, 21565, Republic of Korea. eastside1st@gmail.com

Equal contributor:

^*Seung Hoon Chae and Hyung Jun Park contributed equally to this work as a first author.

Received 17 August 2022 Accepted 22 December 2022

The Korean Academy of Medical Sciences

https://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Owing to limited experience with the new vaccine platforms, discussion of vaccine safety is inevitable. However, media coverage of adverse events of special interest could influence the vaccination rate; thus, evaluating the outcomes of adverse events of special interest influencing vaccine administration is crucial.

Methods

We conducted regression discontinuity in time analysis to calculate the local average treatment effect (LATE) using datasets from Our World in Data and Johns Hopkins University Center for Systems Science and Engineering. For the United States, the United Kingdom, and Europe, the cutoff points were April 23rd and June 23rd, April 7th, and the 14th week of 2021, respectively.

Results

The LATE of the Advisory Committee on Immunization Practices (ACIP) meeting held on April 23rd was −0.249 for all vaccines, −0.133 (−0.189 to −0.076) for Pfizer, −0.064 (−0.115 to −0.012) for Moderna, and −0.038 (−0.047 to −0.030) for Johnson & Johnson. Discontinuities were observed for all three types of vaccines in the United States. The June 23rd meeting of the ACIP (mRNA vaccines and myocarditis) did not convene any discontinuities. Furthermore, there was no significant drop in the weekly average vaccination rates in Europe following the European Medicines Agency (EMA) statement on April 7th. Conversely, there was a significant drop in the first-dose vaccination rates in the United Kingdom related to the EMA report. The first-dose vaccination rate for all vaccines changed by −0.104 (−0.176 to −0.032).

Conclusion

Although monitoring and reporting of adverse events of special interest are important, a careful approach towards public announcements is warranted.

Graphical Abstract

Keywords: SARS-CoV-2, COVID-19, Regression Discontinuity Analysis

INTRODUCTION

Vaccine administration is an essential component of the strategy for controlling the coronavirus disease 2019 (COVID-19) pandemic. Following the surge of delta variant cases,1 2 medical authorities, such as the World Health Organization (WHO), have encouraged the public to undergo vaccination as soon as possible.3 As of October 5, 2021, seven vaccines were approved by the WHO, and 22 vaccines were approved by at least one country.4 Among them, Pfizer/BioNTech [BNT162b2] was approved in 100 countries, and Moderna [mRNA-1273] in 76 countries, after demonstrating safety and efficacy through large phase 3 trials. Both trials showed that serious adverse effects of these vaccines are rare.5 6

However, there have been reports of rare or serious adverse effects following large-scale vaccine administration. A review of the risk of rare blood clots associated with the Oxford/AstraZeneca [AZD1222] vaccine was conducted by the European Medicines Agency (EMA), involving 30 cases of thromboembolic events, which had been reported among 5 million vaccinations.7 8 Furthermore, by April 7, 2021, the EMA acknowledged that the Oxford/AstraZeneca vaccine was associated with unusual blood clots (6.53 cases per 1,000,000 as of April 4, 2021).9 Several countries have limited Oxford/AstraZeneca vaccine administration to younger populations, even though the evidence of vaccine benefits outweighs the risks.10 By the end of March, Canada and Germany suspended the use of the Oxford/AstraZeneca vaccine for those under the age of 55 and 60 years, respectively.11 Italy and Spain restricted the vaccine to those > 60 years following the EMA statement. As for the United Kingdom, the Joint Committee on Vaccination and Immunization recommended alternative vaccines for people aged < 30 years.10 The South Korean Oxford/AstraZeneca vaccine age restriction changed to > 30 years on April 12, 2021.12 Safety of the Johnson & Johnson [Ad26.COV2.S] vaccine, which is the same type of vaccine as Oxford/AstraZeneca, was also investigated. As of April 21, 2021, the reported rates of thrombosis with thrombocytopenia syndrome (TTS) in the United States were 7 cases per 1,000,000 for women aged 18–49 years and 0.9 cases per 1,000,000 for women aged > 50 years. However, the Advisory Committee on Immunization Practices (ACIP) reaffirmed that the Johnson & Johnson vaccine is highly effective, and vaccination continues to be recommended for people aged > 18 years based on the risk-benefit analysis on April 23, 2021.13 Myocarditis has been recognized as a complication of mRNA [mRNA-1273, BNT162b2] vaccinations by the ACIP on June 23, 2021. Myocarditis reporting rates following the second doses of mRNA vaccines were 40.6 cases per 1,000,000 for males aged 12–29 years and 2.4 per 1,000,000 for males aged > 30 years; rates among females in these age groups were 4.2% and 1.0 per 1,000,000, respectively.14 However, vaccine benefits clearly surpass the risk of myocarditis; therefore, mRNA vaccines are recommended for people aged > 12 years in the US.15 16

Because of limited experience with new vaccine platforms, vaccine safety discussion is inevitable.2 However, the reasons for vaccine acceptance and hesitancy could be connected to public announcements regarding the rare and fatal side effects of vaccines.17 Therefore, to overcome the pandemic, it is essential to evaluate whether the outcomes of vaccine adverse events influence the worldwide vaccination rate. In this study, regression discontinuity in time (RDiT) analysis was conducted to investigate the impact of adverse events of special interest (AESI) on the vaccination rate.

METHODS

Data

We integrated two datasets: one dataset from our world, which is produced and maintained by the Global Change Data Lab and contains the vaccination counts of all countries, and the other dataset from the Johns Hopkins University Center for Systems Science and Engineering, which contains the COVID-19 confirmed and death cases for each date. By integration, the dataset covered the number of vaccinations (completed), newly confirmed cases, and cases of death in 47 countries, including the US and European countries. Vaccination rates were calculated as the ratio of the number of completed vaccinations to the total population by country and date (Supplementary Table 1).

Statistical analysis

The study employed a series of RDiT analyses to evaluate whether AESI could have decreased the national vaccination rates. RDiT designs have been used in situations where the treatment is assigned based on a specific date in time.18 19 Following methodological developments in the regression discontinuity literature,20 this study utilized nonparametric polynomial estimators to estimate average treatment effects. In the original regression discontinuity design, when X_i ∈ {0,1} is assigned to a sample of i ∈ {1,2,3,…n} such that X_i = 1 for T_i > c and X_i = 0 for T_i ≤ c (where T_i is a continuous time variable), the local average treatment effect (LATE)

τ = E [Y_{i} (1) - Y_{i} (0)| T_{i} = c] = \lim_{x ↓ c} E [Y_{i} | T_{i} = x] - \lim_{x ↑ c} E [Y_{i} | T_{i} = x]

is estimated by linearly regressing the outcome variable Y on T − c, the treatment variable X, and the interaction (T − c) · X at local values close to the cutoff c. However, because we adopted a non-parametric estimation strategy, τ is approximated through weighted polynomials, whose weights are calculated through a uniform kernel function for each observation distance from the cutoff c.

The time variable for the US and the UK was the calendar day, whereas in Europe, the time variable was the calendar week. Since vaccination rates in Europe were not always reported daily, we used the weekly averages of daily vaccinations. For the US, RDiT analysis was conducted using two cutoff points, April 23rd, 2021(the day the US Center for Disease Control’s ACIP convened to vote on the continued administration of the Johnson & Johnson vaccine) and June 23rd, 2021 (the day an ACIP meeting was held regarding mRNA vaccines and myocarditis). For the UK, the cut-off was April 7th, 2021(the day the EMA held a press briefing regarding Oxford/AstraZeneca’s connection to thromboembolic events). For Europe, RDiT analysis was conducted using the fourteenth week of 2021 (the week of April 7, 2021) as the cutoff, excluding countries where the cumulative vaccination rate was still below 5% four weeks later (Table 1, Supplementary Fig. 1). The main models used 60-day bandwidths for the US and UK and 10-week bandwidths for Europe. All models were computed by applying uniform kernel functions. All estimations were made using the rdrobust command in the Stata software (version 15.0; StataCorp, College Station, TX, USA) (Supplementary Methods 1).

Table 1

Daily vaccination rates per vaccine type

Vaccine type	United States		Europe		United Kingdom
Vaccine type	Mean	Standard deviation	Mean	Standard deviation	Mean	Standard deviation
All vaccines	0.475	0.287	0.400	0.280	0.329	0.299
Pfizer	0.272	0.143	0.317	0.235	-	-
Moderna	0.186	0.132	-	-	-	-
Johnson & Johnson	0.024	0.027	-	-	-	-
Astra Zeneca	-	-	0.073	0.088	-	-

Vaccination rate unit: daily vaccination dose/total population. European countries with less than 5% cumulative vaccination by 5th May (4 weeks after the cutoff) were excluded.

In the sensitivity analysis, we introduced additional features to the main models to address the potential sources of bias. First, we used a variety of other bandwidths, including mean squared error (MSE) optimal bandwidths. Because our models did not have many observations close to the cutoff point, shorter bandwidths served as tougher tests for the significance of the local average treatment effects. Second, because bandwidth selectors could generate substantial bias in the calculated estimator, we derived estimates of average treatment effect that are robust to MSE optimal bandwidths.20 Third, because the vaccination rates may have been influenced by the cumulative vaccination rates, we conducted a set of analyses including the cumulative vaccination rate as a control. Finally, we conducted placebo tests to assess the validity of the treatment cut-off. To address the possibility that the April 23rd cutoff for the US, being a Friday, simply reflects lower vaccination rates on the weekend, the first placebo test had 16th of April as its cutoff. The remaining placebo tests switched the cutoff dates for the US and Europe/UK.

Ethics statement

We used an open data source that is publicly available. Thus, approval from the Institutional Review Board and participant consent were not needed. All the study procedures were performed in accordance with the Declaration of Helsinki.

RESULTS

Our study analyzed vaccination rates in Europe and the US between January 1, 2021, and September 1, 2021. The European nations included in the study are listed in the supplement. The cumulative vaccination rates in the US were daily, and those in Europe were weekly averages (Supplementary Fig. 1).

United States

On April 23, 2021, the day the US Center for Disease Control’s ACIP convened to vote on the continued administration of the Johnson & Johnson vaccine, the daily vaccination rate seemed to have experienced discontinuities in all three types of vaccines. The local average treatment effect of the ACIP meeting was −0.249 (95% confidence interval [CI], −0.360 to −0.138) for all vaccines, −0.133 (95% CI, −0.189 to −0.076) for Pfizer, −0.064 (95% CI, −0.115 to −0.012) for Moderna, and −0.038 (95% CI, −0.047 to −0.030) for Johnson & Johnson (Fig. 1A, Table 2). These are sizeable effects, as they are larger than half of each vaccine’s mean daily vaccination rates.

Fig. 1

Regression discontinuities in the United States with (A) April 23rd, 2021, as the cutoff (B) June 23rd as the cutoff.

Table 2

Main analysis

Region (Threshold)	Vaccine	ATE	P value	95% confidence interval
United States (23rd April, 2021)	All vaccines	−0.249	0.000	(−0.360 to −0.138)
	Pfizer	−0.133	0.000	(−0.189 to −0.076)
	Moderna	−0.064	0.015	(−0.115 to −0.012)
	Johnson & Johnson	−0.038	0.000	(−0.047 to −0.030)
United States (23rd June, 2021)	All vaccines	−0.019	0.346	(−0.059 to 0.021)
	Pfizer	−0.019	0.346	(−0.060 to 0.021)
	Moderna	0.013	0.300	(−0.011 to 0.037)
	Johnson & Johnson	−0.008	0.000	(−0.012 to −0.004)
Europe (7th April: 14th Week of 2021)	All vaccines	0.129	0.000	(0.093 to 0.166)
	Pfizer	0.133	0.000	(0.102 to 0.163)
	Astra Zeneca	0.000	0.986	(−0.026 to 0.0.26)
United Kingdom (7th April, 2021)	All vaccines	−0.104	0.005	(−0.176 to −0.032)

ATE = average treatment effect, CI = confidence interval.

In contrast, on June 23, 2021, on the day an ACIP meeting was held regarding mRNA vaccines and myocarditis, the daily vaccination rate did not exhibit any substantial discontinuities. The change in vaccination rate was −0.019 (95% CI, −0.059 to 0.021) for all vaccines, −0.019 (95% CI, −0.060 to 0.021) for Pfizer, 0.013 (95% CI, −0.011 to 0.037) for Moderna, and −0.008 (95% CI, −0.012 to −0.004) for Johnson & Johnson (Fig. 1B, Table 2). Despite a P value < 0.001 for the Johnson & Johnson vaccine, the treatment effect was small.

Europe

On April 7th, 2021, which was the day the EMA held a press briefing regarding Oxford/AstraZeneca’s connection to thromboembolic events, no significant drop in the weekly averages of vaccination rates was observed. The change in vaccination rate was 0.129 (95% CI, 0.093 to 0.166) for all vaccines, 0.133 (95% CI, 0.102 to 0.163) for Pfizer, and −0.000 (95% CI, −0.026 to 0.026) for Oxford/AstraZeneca (Fig. 2, Table 2).

Fig. 2

Regression discontinuities in the United Kingdom and Europe with April 7th, 2021, as the cutoff.

When the RDiT analysis was conducted only among European countries that reported daily vaccination rates, there was no observable drop in vaccination rates. Oxford/AstraZeneca showed similar rates to pretreatment levels, whereas Pfizer’s vaccination rate increased by 0.130 (95% CI, 0.060 to 0.210) (Supplementary Fig. 2).

United Kingdom

In the UK, a significant drop was observed in the first dose vaccination rates following the reports on the EMA’s press briefing on Oxford/AstraZeneca’s connection to thromboembolic events on April 7th. The first-dose vaccination rate for all vaccines changed by −0.104 (95% CI, −0.176 to −0.032) (Fig. 2, Table 2), which amounts to approximately a third of the mean daily vaccination rate in the UK.

Sensitivity analysis

To evaluate the robustness of the discontinuity, sensitivity tests were performed on the US data using 40-day and MSE-optimal bandwidths and by conducting placebo tests with different cutoff thresholds. With 40-day bandwidths in the US, Pfizer and Johnson & Johnson vaccines exhibited discontinuities on April 23, 2021, with average treatment effects of −0.096 (95% CI, −0.168 to −0.024) and −0.047 (95% CI, −0.056 to −0.038), respectively. However, the results were not robust to MSE optimal bandwidths. Because the unit of analysis was country-day, the MSE-optimal bandwidths severely restricted the sample. Fig. 3 illustrates the results of sensitivity analysis when the cumulative vaccination rate was included as a covariate.

Fig. 3

Sensitivity analysis with cumulative vaccination rate as a covariate on 2021 (US and UK).

In the UK, vaccination rates fell significantly after April 7, 2021, in a sensitivity test that used a 40-day bandwidth. The first-dose vaccination rates changed by −0.150 (95% CI, −0.234 to −0.067) after the cutoff. However, as with the US, these results are not robust to MSE optimal bandwidths (Table 3).

Table 3

Sensitivity tests

Region (Threshold)	Vaccine	ATE (95% CI)	P value	ATE type	Bandwidth
United States (23rd April, 2021)	Pfizer	−0.096 (−0.168 to −0.024)	0.009	Conventional	40 days
	Moderna	−0.040 (−0.105 to 0.025)	0.229	Conventional	40 days
	Johnson & Johnson	−0.047 (−0.056 to −0.038)	0.000	Conventional	40 days
	Pfizer	−0.013 (−0.153 to 0.075)	0.836	Robust	18 days (MSE optimal)
	Moderna	0.017 (−0.073 to 0.106)	0.714	Robust	29 days (MSE optimal)
	Johnson & Johnson	−0.016 (−0.036 to 0.004)	0.133	Robust	3 days (MSE optimal)
Europe (7th April: 14th Week of 2021)	Pfizer	0.048 (0.008 to 0.087	0.018	Conventional	5 weeks
	Astra Zeneca	0.003 (−0.029 to 0.035)	0.862	Conventional	5 weeks
	Pfizer	0.018 (−0.060 to 0.096)	0.652	Robust	3 weeks (MSE optimal)
	Astra Zeneca	0.136 (0.111 to 0.161)	0.000	Robust	2 weeks (MSE optimal)
United Kingdom	All	−0.150 (−0.234 to −0.067)	0.000	Conventional	40 days
United Kingdom	All	0.263 (0.082 to 0.444)	0.000	Robust	10 days (MSE optimal)

ATE = average treatment effect, CI = confidence interval, MSE = mean squared error.

Finally, the placebo tests yielded the expected results, with the exception of the Johnson and Johnson vaccines in the US (Supplementary Table 2). The Friday the week before the US’s 23rd April, 2021 threshold did not result in a negative regression discontinuity. Using Europe’s cut-off date in the US and vice versa also yielded no significant drop in vaccination rates (Supplementary Table 2).

DISCUSSION

The EMA statement related to TTS caused vaccine discontinuity in the UK and US. However, reports of myocarditis were not related to vaccine discontinuities. The following factors could have affected the differences in discontinuities between the two vaccine platforms.

(1) Fatality rate of severe adverse events

Both adverse events occurred mostly in the younger population, depending on the case. TTS had a fatality rate of 20–30%.21 Conversely, very rare fatal cases of myocarditis have been reported.22

(2) Differences in policy responses to vaccines

Regulators placed age limits based on the incidence of adverse events and fatality rates.10 Oxford/AstraZeneca is restricted in most countries. Conversely, despite safety concerns regarding mRNA vaccines among children, no age restriction was imposed on the older population (> 12 years).

(3) Confidence gaps between the vaccine platforms

The survey conducted by YouGov, a British polling firm, shows that confidence in the Oxford/AstraZeneca vaccine decreased sharply in European countries (February and March 2021).23

(4) Date of publication of adverse events

The adverse reactions of the mRNA vaccine were acknowledged two months later.22 Furthermore, the time to confirm rare adverse events did not significantly affect the entire population.

This study is the first to evaluate how government announcements on severe vaccine-related side effects had an impact on the vaccination rate based on data from Western countries that administered multiple types of vaccines. Additionally, changes over time were observed in RDiT analyses.

This study had several limitations. First, as a time-series analysis, we could not include all the factors that changed over time in our study design. For instance, regulatory decisions by countries regarding the rapid spread of the delta variant and increasing public awareness regarding vaccines. Second, despite the gap of 2 months, the trend of the epidemic could have acted as a confounding factor. The public is expected to return to pre-pandemic conditions in April owing to the high vaccination rate and social distancing schemes. However, with the surge in the delta variant in June, countries have emphasized the importance of vaccination.24 Third, decreased vaccination rates could be influenced by a decline in the target groups owing to the EMA statement.25 However, it should be noted that the US did not modify the age recommendation. In addition, some European countries have restricted the use of Oxford/AstraZeneca to the younger population before the EMA statement. Fourth, vaccination could be independently affected by disparities in vaccine supply. Even though the target countries in our study achieved a smooth supply of vaccines, most countries faced limited supplies in the first half of 2021.26 Moreover, several European countries have offered alternative vaccines to replace Oxford/AstraZeneca.27 Therefore, it is possible that vaccination rates are less affected by the adverse effects of adenoviral vector vaccines. Lastly, vaccination rates may have been influenced by cumulative vaccination rates. Despite high vaccine acceptance in some countries, the vaccination rate could not reach > 80% of the population; an inclined S pattern can be seen in the vaccination curve.28 However, even if vaccine hesitancy is influenced by the cumulative vaccination rate, mRNA vaccines should be most affected by the intensive media coverage in June. However, there were no significant shifts in the mRNA vaccination rates.

In conclusion, vaccine conviction is a key component of high vaccination coverage to overcome the pandemic. The relationship between vaccine acceptance, hesitancy, and documentation of AESI should be highlighted when implementing the vaccine coverage plan. Although it is important to monitor and report adverse effects of vaccines, a careful approach towards public announcements is warranted.

Notes

Funding: This study was supported by grants from the Gachon University Gil Medical Center (2019-11) and National Research Foundation of Korea grant funded by the Korea government (NRF-2021R1A5A2030333). Also supported in part by the MSIT (Ministry of Science and ICT), supervised by the NIPA (National IT Industry Promotion Agency), and supported by a grant (22183MFDS431) from Ministry of Food and Drug Safety in 2022. The sponsor of the study was not involved in the study design, analysis, and interpretation of data; writing of the report; or the decision to submit the study results for publication.

Disclosure: The authors have no potential conflicts of interest to disclose.

Data Availability Statement: The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Author Contributions:

Conceptualization: Chae SH, Park HJ, Jung J.
Formal analysis: Chae SH, Park HJ.
Funding acquisition: Jung J.
Investigation: Park H.
Methodology: Park H.
Project administration: Jung J.
Resources: Radnaabaatar M.
Writing - original draft: Chae SH, Park HJ.
Writing - review & editing: Radnaabaatar M, Park H, Jung J.

References

1. Lopez Bernal J, Andrews N, Gower C, Gallagher E, Simmons R, Thelwall S, et al. Effectiveness of Covid-19 vaccines against the B. 1.617. 2 (Delta) variant. N Engl J Med. 2021; 385(7):585–594. PMID: 34289274.

2. Klein NP, Lewis N, Goddard K, Fireman B, Zerbo O, Hanson KE, et al. Surveillance for adverse events after COVID-19 mRNA vaccination. JAMA. 2021; 326(14):1390–1399. PMID: 34477808.

3. World Health Organization (WHO). COVID-19 advice for the public: getting vaccinated. Updated July 14, 2021. Accessed October 5, 2021. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/covid-19-vaccines/advice .

4. McGill COVID19 Vaccine Tracker Team. Vaccines candidates by trial phase. Updated October 1, 2021. Accessed October 5, 2021. https://covid19.trackvaccines.org/vaccines/?fbclid=IwAR0rJdUx1T1j9TalYCJzwvnqm6MWluN2wwWtnuDP0GW6YvKLh3nW-Ex0ZKU .

5. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020; 383(27):2603–2615. PMID: 33301246.

6. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021; 384(5):403–416. PMID: 33378609.

7. Østergaard SD, Schmidt M, Horváth-Puhó E, Thomsen RW, Sørensen HT. Thromboembolism and the Oxford-AstraZeneca COVID-19 vaccine: side-effect or coincidence? Lancet. 2021; 397(10283):1441–1443. PMID: 33798498.

8. Huh K, Na Y, Kim YE, Radnaabaatar M, Peck KR, Jung J. Predicted and observed incidence of thromboembolic events among Koreans vaccinated with ChAdOx1 nCoV-19 vaccine. J Korean Med Sci. 2021; 36(27):e197. PMID: 34254476.

9. European Medicines Agency (EMA). AstraZeneca’s COVID-19 vaccine: EMA finds possible link to very rare cases of unusual blood clots with low blood platelets. Updated 2021. Accessed October 5, 2021. https://www.ema.europa.eu/en/news/astrazenecas-covid-19-vaccine-ema-finds-possible-link-very-rare-cases-unusual-blood-clots-low-blood .

10. Wise J. Covid-19: how AstraZeneca lost the vaccine PR war. BMJ. 2021; 373(921):n921. PMID: 33853827.

11. Dyer O. Covid-19: EMA defends AstraZeneca vaccine as Germany and Canada halt rollouts. BMJ. 2021; 373(883):n883. PMID: 33795216.

12. Lee E, Lee YK, Kim TE, Hwang I, Jung YH, Lee HR, et al. Reports of anaphylaxis after coronavirus disease 2019 vaccination, South Korea, 26 February to 30 April 2021. Euro Surveill. 2021; 26(33):2100694. PMID: 34414880.

13. MacNeil JR, Su JR, Broder KR, Guh AY, Gargano JW, Wallace M, et al. Updated recommendations from the advisory committee on immunization practices for use of the Janssen (Johnson & Johnson) COVID-19 vaccine after reports of thrombosis with thrombocytopenia syndrome among vaccine recipients - United States, April 2021. MMWR Morb Mortal Wkly Rep. 2021; 70(17):651–656. PMID: 33914723.

14. Gargano JW, Wallace M, Hadler SC, Langley G, Su JR, Oster ME, et al. Use of mRNA COVID-19 vaccine after reports of myocarditis among vaccine recipients: update from the advisory committee on immunization practices - United States, June 2021. MMWR Morb Mortal Wkly Rep. 2021; 70(27):977–982. PMID: 34237049.

15. Bozkurt B, Kamat I, Hotez PJ. Myocarditis with COVID-19 mRNA vaccines. Circulation. 2021; 144(6):471–484. PMID: 34281357.

16. World Health Organization (WHO). COVID-19 subcommittee of the WHO Global Advisory Committee on Vaccine Safety (GACVS): updated guidance regarding myocarditis and pericarditis reported with COVID-19 mRNA vaccines. Updated July 9, 2021. Accessed October 05, 2021. https://www.who.int/news/item/09-07-2021-gacvs-guidance-myocarditis-pericarditis-covid-19-mrna-vaccines .

17. Pullan S, Dey M. Vaccine hesitancy and anti-vaccination in the time of COVID-19: a Google Trends analysis. Vaccine. 2021; 39(14):1877–1881. PMID: 33715904.

18. Anderson ML. Subways, strikes, and slowdowns: the impacts of public transit on traffic congestion. Am Econ Rev. 2014; 104(9):2763–2796.

19. Bakolis I, Stewart R, Baldwin D, Beenstock J, Bibby P, Broadbent M, et al. Changes in daily mental health service use and mortality at the commencement and lifting of COVID-19 ‘lockdown’ policy in 10 UK sites: a regression discontinuity in time design. BMJ Open. 2021; 11(5):e049721.

20. Calonico S, Cattaneo MD, Titiunik R. Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica. 2014; 82(6):2295–2326.

21. Park YS. Thrombosis and severe acute respiratory syndrome coronavirus 2 vaccines: vaccine-induced immune thrombotic thrombocytopenia. Clin Exp Pediatr. 2021; 64(8):400–405. PMID: 34237213.

22. Shay DK, Shimabukuro TT, DeStefano F. Myocarditis occurring after immunization with mRNA-based COVID-19 vaccines. JAMA Cardiol. 2021; 6(10):1115–1117. PMID: 34185047.

23. Smith M. Perceived safety drops substantially in France, Germany, Spain and Italy. Updated March 22, 2021. Accessed October 7, 2021. https://yougov.co.uk/topics/international/articles-reports/2021/03/22/europeans-now-see-astrazeneca-vaccine-unsafe-follo .

24. Bian L, Gao Q, Gao F, Wang Q, He Q, Wu X, et al. Impact of the Delta variant on vaccine efficacy and response strategies. Expert Rev Vaccines. 2021; 20(10):1201–1209. PMID: 34488546.

25. Karron RA, Key NS, Sharfstein JM. Assessing a rare and serious adverse event following administration of the Ad26.COV2.S vaccine. JAMA. 2021; 325(24):2445–2447. PMID: 33929484.

26. Torjesen I. Covid-19 vaccine shortages: what is the cause and what are the implications? BMJ. 2021; 372(781):n781. PMID: 33741547.

27. Beaubien J. It’s the vaccine that’s lost a lot of trust. But AstraZeneca still has its fans. Updated June 1, 2021. Accessed October 5, 2021. https://www.npr.org/sections/goatsandsoda/2021/06/01/1002067808/astrozenecas-rocky-rollout-the-woes-of-the-vaccine-of-the-world .

28. Mathieu E, Ritchie H, Rodés-Guirao L, Appel C, Giattino C, Hasell J, et al. Coronavirus Pandemic (COVID-19). Updated 2020. Accessed October 5, 2021. https://ourworldindata.org/coronavirus .