Evaluating Claims of Excess Cancer Deaths in the USA During the Pandemic.

An anonymous, apparently very experienced and prolific analytical systems modeler who is heavily followed on twitter, @EthicalSkeptic, has been modeling the publicly CDC provisional deaths by cause data for the past 6 months, and concludes from his models that a specific factor (which must not be named) in early Spring 2021 introduced an inherent change point into the data leading to increasingly high numbers of excess deaths in the USA.

I do not know the identity of this person, whether male or female, but will (arbitrarily) refer to them using masculine pronouns throughout.

He claims the biggest effect of this factor is evident in cancer deaths, with his model suggesting an "11-sigma" event occurring since then and leading to frequent plots like the following:

These posts have been very influential, with many touting them as evidence of serious harm, and others dismissing them outright. The lack of detailed documentation of the analytical steps make it difficult for others to evaluate his model and conclusions.

In his blog, he has recently released a post describing his results and outlining the steps of his analysis. From this it is clear that his analytical steps include:

1. Estimate Weekly Baseline Cancer Deaths: From the 2014-2019 death date he estimates a seasonal-annual baseline with growth, estimating a seasonal trend that is a week-specific average of 2014-2019 weekly death counts with an estimated linear trend over time. This is well described.

2. Moving-Average Smoothing of Actual and Baseline Weekly Cancer Deaths: uses a 6-week moving average smoother of the past 3 weeks, the current week, and next 2 weeks to smooth out some of the noise of both the actual counts and the baseline estimates.

3. Lag-Factor Profile to Adjust for Provisional Nature of Data: From a database of historic shortages in reported death records in the CDC provisional data from the current week back 21 weeks, he estimates a lag taper function that is used to try to impute the missing data from underreporting for the most recent 21 weeks. He mentions he has many models for this, but the specific one used is not given.

4. Pull Forward Effects to Adjust for Excess Deaths Earlier in the Pandemic: An uncertain number of individuals who were destined to die of cancer between 2020 and 2022 in a counterfactual sans-pandemic world instead died of COVID-19 or other pandemic-related factors earlier in the pandemic. This suggests the baseline weekly cancer death rate should be decreased by this factor, which is called a "pull forward effect." He estimates and removes this PFE effect from the baseline based on historical shortages from the MMWR Weeks going back to the beginning of the pandemic. He mentions the average PFE over all death causes is 4.9% and decreasing linearly to 0 within 130 weeks, leading to an estimated pull forward count total of ~11% of all COVID-19 deaths, that he considers conservative. It is not clear exactly what number he uses for cancer deaths.

Unfortunately, he is not able to share his scripts for running these analyses to evaluate all of the unknown details or to run sensitivity analyses to different assumptions/estimates for the background, lag, and PFE modeling, citing proprietary reasons since similar models are used in his professional work.

His modeling is done on two publicly released CDC data sets:

1. Weekly_Counts_of_Deaths_by_State_and_Select_Causes_2014-2019.csv

2. Weekly_Provisional_Counts_of_Deaths_by_State_and_Select_Causes__2020-2022

This hyperlink goes to the current release of this data set (8/24/2022) right now that includes weekly counts through the week ending 8/13/2022. This link will change to them more updated version later, so here is the actual data set I used:

Weekly_Provisional_Counts_of_Deaths_by_State_and_Select_Causes__2020-2022

.csv

Download CSV • 1.26MB

And here is the actual 2014-2019 data set:

Weekly_Counts_of_Deaths_by_State_and_Select_Causes__2014-2019

.csv

Download CSV • 2.49MB

In his analyses, he uses the 6/2/2022 release of the provisional 2020-2022 death data. He does this after noting that subsequent releases reassigned/removed certain deaths that were previously reported, including ~9k cancer deaths that he purports were assigned to other causes, claiming it was done purposefully by the CDC for obfuscation.

Unfortunately, the 6/2/2022 release is not available from the CDC website (as subsequent releases are placed in the same spot) and there does not appear to be any archive of this data set on web.archive.org, so I have been unable to evaluate and comment on these changes here. I have asked him for his copy of this old release so I could compare to the newer data, but he has not shared with me at this point. So I only have access to the current 8/24/2022 release.

In this blog post, I evaluate these data to assess his claims, and to determine if his results are driven by particular assumptions in his analytical work flow.

A few caveats for what I will present:

• Given the difficulty of accurate lag adjustment for the most recent provisional deaths and lack of documentation of the specific model he uses, instead of applying a lag model I will limit my summaries to data through March 12, 2022, which omits the most recent 22 weeks (5 months) of data that contain the most incomplete data for which lag modeling is most important. I will wait to comment on the March-August period until later releases are available with more complete data for this time frame. Note that his lag modeling goes back 21 weeks so this should be acceptable.

• I have my own seasonal-annual linear baseline trend that I estimate that may differ slightly from his. I plot the fit of the model with the data to demonstrate that the baseline trend fits I use fits well.

• I plot the raw data but also include 6wk moving average smooths for comparability.

Given lack of details about the PFE model and inability to access the 6/2/2022 release before the purported cancer death removals/reassignments, I asked him for documentation or information about these effects, and he insisted that the PFE tapers to zero so has little effect on results, and that his conclusions of high cancer excess deaths is still significant even without adding in the extra deaths he mentions were removed/reassigned subsequent to the 6/2/2022 release (BTW the total number of "removed" deaths was 23k, and he claims about 9k were cancer deaths).

Thus, here I will:

• Fit the current release data 8/24/2022, and consider later the implications of 9k additional cancer deaths would have on my conclusions.

• Given the lack of firm data to estimate a PFE and lack of documentation of the specific model he uses, I will present results without any PFE. He mentions this should have little effect at this point, so may not make a difference, and at the end I will mention how any PFE may affect my conclusions.

I am not trying to do my own detailed all-encompassing analysis, but wanting to assess whether the data support his assertions, and secondarily to identify which assumptions if any might be primary drivers of his conclusions.

I think reproducibility and transparency are paramount, especially in divisive times as during this pandemic, so as usual I provide the full R script to download the data and reproduce my results and figures:

CDC Excess Deaths Cancer scripts 8-24-22

.R

Download R • 16KB

I encourage anyone to check my work, or adapt my scripts to do their own analyses, e.g. exploring various assumptions on baseline death rate, PFE, or lag adjustments.

Also, while this blog post focuses on the cancer deaths outcome, the data set includes all causes and the scripts are easily adapted to generate analogous results and figures for any of the other causes.

Hopefully this will encourage a more open exchange of ideas and models exploring these data.

Results

Here are the raw weekly cancer deaths data from 2014-2022 from concatenating the 2014-2019 and 2020-2022 data sets.

Here is the seasonal/annual linear trend effect I estimated from the 2014-2019 data and used as the baseline model. It appears to fit the data very well.

Applying this baseline to the full 2014-2022 time period we get the following. Again, I stopped at March 12, 2022 to avoid the most recent 22 weeks most affected by the undercounting inherent in the provisional counts.

To make it easier to see potential excess (or deficit) cancer deaths during the pandemic period, here I plot from January 1, 2020 through March 12, 2022.

Note that the local maxima and minima do not quite match up with the historical seasonal patterns.

Subtracting the baseline (blue line) from weekly cancer deaths (black dots), we get a plot of weekly excess cancer deaths throughout the pandemic relative to this estimated baseline.

Note the periodic nature of the cancer excess deaths over time, going through excess and deficit periods. In particular note the major deficit period in mid-2020 followed by a shorter excess period, and the major deficit period in early 2021 followed by a high excess period in the middle of 2021 decreasing to a lower magnitude excess later in the year with an apparent decrease below baseline in early 2022. This decline continues in the provisional data between March and August, but as mentioned above we do not consider this data since clearly lag adjustments would be necessary as those data are far from complete.

We can compute a cumulative sum on these excess deaths to see a running cumulative total of excess deaths since the beginning of the pandemic.

We see how the the mid-2020 and early 2021 major deficits resulted in ~4k deficit cancer deaths relative to baseline since the beginning of the pandemic by spring 2021, which recovers all the way to baseline by late 2021 and reaches a maximum of ~2k excess cancer deaths by early 2022, after which it starts to decline again.

Based on these data and the proposed seasonal/annual linear baseline, there were major deficits of cumulative cancer deaths during the pandemic relative to baseline through Spring 2021, which then recovered and moved into excess territory in early 2022, with about 2k excess (which is <2 days of cancer deaths at the baseline 11.5k weekly rate).

Thus, I don't see how without PFE or the 9k extra cancer deaths one can confidently conclude that what we are seeing is an excess caused by a newly introduced factor, or recovery of previous deficits.

Even if we add in all of the 9k "reassigned" cancer deaths after the June 2, 2022 (assuming they were all true and occurred before March 12, 2022), this would result in a cumulative total of ~11k excess deaths during the pandemic, which amounts to ~1 week of baseline cancer deaths, or 2% of the annual cancer deaths number, not a large magnitude difference.

It seems like the PFE might be a stronger contributor to his results than he claimed. It is possible that numerous baseline cancer deaths that were expected to occur in 2021 or 2022 were individuals who died of COVID-19 or other pandemic-related causes in 2020 or 2021, and so the baseline should be decreased by this number. I welcome anyone to edit my scripts and add in any PFE effects they think appropriate and transparently share their analysis and scripts.

However, it appears the bigger factor in his conclusions are his lag assumptions and the later 2022 data. Note in his plots how the 2022 data take off during the lag period to much higher levels appearing any other time during the pandemic. I am not sure why his lag model produces such results, but this seems strongly influential on his "11 sigma" dramatic claims. If we look at the provisional data all the way out to August 17, 2022, we see the numbers have continued to come down -- it is not clear how his lag adjustment could cause this sharp decline to become the sharpest incline of the pandemic, but again this is why I wish he would release his scripts so we could reproduce and evaluate his results.

The key to assessing his claims is of course to see the 2022 data as they become more mature and we have more complete data through the summer. If the increase in cancer deaths from the nadir in Spring 2021 continues throughout 2022, then this would provide more support for his hypothesis, but if the decrease that appears to have started in early 2022 continues, then this would weaken the claim of inherently higher excess cancer deaths initiated by some factor introduced in early 2021.

The reassignment and/or removal of deaths subsequent to the June 2, 2022 release requires explanation, and I wish that @EthicalSkeptic would share the June 2, 2022 release so that those of us without access could investigate and characterize the changes. This also needs to be sorted out, but for now we can just consider the potential effect of these 9k reassigned cancer deaths if as he claims they were inappropriately reassigned. Many oncologists have stated that the delaying of cancer screenings/preventative care during the pandemic could lead to an eventual increase in cancer deaths, with cancers diagnosed later, but at a later and less treatable stage. Indeed this may occur and may be starting to occur, but is not fully apparent yet in these data. Of course, those claiming high excess cancer deaths including @EthicalSkeptic are not considering that factor, but are claiming it is due to another "must not be named factor" starting in Spring 2021. Based on my understanding of carcinogenesis from >25 years of experience in cancer research, even after a major carcinogenic insult, I would not expect it to lead to a high number of excess cancer deaths for many years given the time it typically takes to for cancer to initiate, develop, become advanced, and lead to death even in cases of extreme established carcinogens. In order to advance such a claim for this unnamed factor, one would need to at least have a plausible hypothesis and some evidence for how this factor could produce advanced cancers and deaths so quickly, beyond near-baseless speculation.

Undoubtedly, more cancer incidence and death data will accrue and be assembled in 2022 from the USA and other parts of the world, and we will eventually get answers to the degree of excess cancer deaths we have seen during the pandemic and if substantial, identify the causes.