United States COVID-19 model passes Codecheck

The software behind a major Imperial study warning of a potential US coronavirus resurgence has received a Codecheck endorsement.

The key findings in the 'Report 23' from Imperial College were reproducible. Dr Stephen Eglen University of Cambridge

The independent review of the Imperial COVID-19 Response Team's code for Report 23 was led by Dr Stephen Eglen, Reader in Computational Neuroscience in the Department of Applied Mathematics and Theoretical Physics at the University of Cambridge.

The analysis confirms the code's reproducibility and quality.

The report's code can be viewed on GitHub, alongside the Codecheck certification. 

The review grants the code a Codecheck.org.uk “certificate of reproducible computation”: the gold standard in independent execution of computations underlying research articles. 

In late May, when the US had 100,000 confirmed COVID-19 deaths, Report 23 found that at least 24 US states had a reproduction number above 1. It also warned of a potential doubling of COVID-19 deaths by August: an estimate that is sadly looking likely to come to fruition as confirmed deaths exceed 120,000 on 22 June.

In his Codecheck review, Dr Stephen Eglen said: "The key findings in the 'Report 23' from Imperial College were reproducible. I was able to re-run their code and generate qualitatively similar results to those shown in their manuscript. Differences in absolute values in results are due to the stochastic nature of the analysis. All code to reproduce the data worked as expected, and all key datasets were provided."

The Imperial team's recent Nature paper, which found that lockdowns in 11 European countries may have prevented 3.1 million deaths, also passed a Codecheck of its underlying software. 

The news follows a previous Cambridge-led Codecheck confirming the reproducibility of Imperial Report 9. 

Despite high-profile criticism from some commentators - including multiple inaccurate pieces of reportage and op-eds in blogs and newspapers - the Imperial team's code continues to hold-up when subject to expert scrutiny.