Professor Neil Ferguson OBE speaks at Westminster event on modelling

Modelling transparency and contingency planning will be crucial in modelling any future crisis according to Professor Ferguson OBE.

The event, jointly hosted by the Forum and the Institute for Government, was the final in a series exploring Net Zero and the role of science and technology in policymaking. Professor Ferguson OBE, Director of the MRC Centre for Global Infectious Disease Analysis and the Jameel Institute at Imperial College, spoke alongside Ben Chu, Economics Editor for Newsnight, Richard Hughes, Chair of the Office for Budget Responsibility (OBR), and Ruth Kelly, Chief Analyst at the National Audit Office (NAO).

Modelling the pandemic

Professor Ferguson stated that the UK had a well organised science advice system in government during the pandemic.

Twelve separate groups produced individual models before agreeing a consensus that was provided to government. He summarised the importance of three types of modelling: retrospective to better understand and explain what happened; models for short-term projections to inform situational awareness, and prospective counterfactual models to examine different future policy options to inform decision-making. However, he said that the government’s lack of a clear strategic objective in tackling the pandemic in 2020 made it difficult for such modelling to be used as effectively as it might have been - and later was in 2021.

“Science and modelling have improved but decision-making is crucial” Professor Neil Ferguson OBE Director of the MRC Centre for Global Infectious Disease Analysis and the Jameel Institute

The pandemic presented a unique challenge to financial modelling being both a supply and demand crisis, which meant that conventional models were insufficient. Richard Hughes said the OBR built new sectorally-stratified models to fit the new reality as the pandemic had a distinctly different impact on hospitality than on financial services.

Ruth Kelly reflected that the government relies on thousands of models for the day-to-day running of government and evidence-based policymaking. People’s lives are affected enormously by the decisions that modelling underpins. However, auditors regularly find errors in government models. She cited a lack of work on contingencies as a concern.

Government Modelling Capability

Professor Ferguson highlighted modelling as a UK strength, but that government will require surge capacity during future crises that would be inefficient to maintain permanently on standby within government agencies. Surge capacity will therefore continue to be obtained from commercial and academic partners, as during the pandemic.

For Richard Hughes, using multiple models to drive consensus and help characterise uncertainties (the SAGE approach) was critical. Models can be produced quickly but the civil service can lack expertise and required data to take quick decisions from them. The OBR struggled to access real time data to inform decision-making during the pandemic.

Engaging policymakers with downside scenarios is a challenge agreed with by the other panellists. Ruth Kelly stated that within the civil service, there is often an undue focus on best estimates.

Professor Ferguson cautioned that models will never be precisely accurate in forecasting something as complex as a pandemic three-months ahead, but that insights longer-term scenario modelling provide can still provide an evidence-base for decisions if politicians are willing to engage and understand model limitations and assumptions. He noted that the first mathematical model on global warming was published in 1938 but that serious political engagement with that modelling took several decades to develop.

Public Understanding

Ben Chu outlined three rules for presenting models to the public to build trust and understanding:

• Language matters: make clear that models are not definitive forecasts
• Caveats matters: emphasise the assumptions and scenarios included and those not included
• Accountability matters: the media should retrospectively assess models to demonstrate to the public which assumptions were right/wrong and why.

Transparency is expected and should be the norm said Ruth Kelly but isn’t always the case. Richard Hughes cited the OBR publicly releasing their working in April 2020 as an act of transparency during a novel crisis. People’s financial decision-making was informed by the data.

Professor Ferguson gave the caveat that transparency could be destabilising in situations of great uncertainty. Models produced at the beginning of the pandemic had alarming results and suffered from volatile scenarios. A two to six week delay on release of SAGE documents was agreed to ensure that models could respond to the fast-moving situation, include more data, and respond to behaviour.

Modelling Risks

Professor Ferguson cited the inclusion of live, up-to-date data as essential for producing valuable models. Meanwhile Ruth Kelly called for uncertainty analysis to be built into early models.

Richard Hughes noted that the 21^st century was likely to be more unpredictable than the late 20^th century. Modelling is therefore needed to account for emerging risks and not dispense with ‘small-issue’ risks that could grow. A failure to do this would harm public trust in modelling.

Cognitive bias and group think are risks in modelling and difficult to disregard entirely, but Professor Ferguson confirmed that modellers are conscious of them. Citing experience in the United States, he concluded that establishing parallel teams is a useful approach to provide oversight and challenge decision making.

You can watch the event back on Youtube, following this link.