Decision-making under uncertainty

Working with probabilities

One reason the approach we looked at in the last section is elegant is that it does not require us to know the probability of unforeseen events occurring. However, Professor Nilay Shah, head of Imperial’s Department of Chemical Engineering, points out that in many contexts – in particular, those where it is not possible to build in a recovery phase – it is useful to incorporate probabilities into decision-making.

“Some of the biggest things that come up are demands for products, so you’re trying to plan some manufacturing, or you’re deciding how big a facility to build, and you just don’t know what the demand will be”, he says. “The other thing is prices: you’re producing something that could be as simple as electricity or hydrogen and you don’t know what price you’ll get. You’re trying to make a decision about what to do given these uncertainties.”

The solution can be illustrated using a classical example from inventory management. A newspaper vendor must decide how many copies to purchase each day in the face of uncertain demand, knowing that any unsold newspapers at the day’s end will be worthless. A naïve solution would be to take the average number of copies sold each day and purchase that many. However, if the cost price is £1 and the sales prices £1.50, this would probably be suboptimal as the cost of buying too many newspapers is greater than the opportunity cost of buying too few.

A better approach is to use historical data and informed guesswork to formulate a probability distribution that can be represented as a curve on a graph with newspapers demanded on one axis and a probability between 0 and 1 on the other. Using this information, the vendor can then draw a curve for each potential inventory volume (say, 100, 150 and 200 newspapers) with probability on one axis and the other now showing overall profit or loss. By comparing these curves, the vendor can identify the inventory volume that will maximise their expected value (the average profit they would make if the day were replayed an infinite number of times). It also gives them the option of choosing a purchase volume that minimises their possible loss. This is a simple case of an approach to decision-making under uncertainty called stochastic optimisation.

Photo: A news kiosk in Lisbon, Portugal. In reality, newspapers are often sold to vendors on a sale or return basis because it is in the publisher's interest to err toward offering too many copies for sale rather than too few. Credit: zulufriend via Getty Images

Seed production

Professor Shah and colleagues in the Department of Chemical Engineering apply stochastic optimisation to a variety of real-world business and engineering contexts.

In one project, Professor Shah’s master’s student, Yanbin Zhu, applied the technique to seed production using data from seed manufacturer Syngenta. With the global population increasing, there is a greater need for more efficient agriculture. Most of the world’s crops comes from seeds, which are often supplied to farmers by commercial seed manufacturers. Mr Zhu and colleagues developed a model that could allow a seed manufacturer to plan seed production at regional level.

The company needs to decide which seeds to plant in which regions, accounting for several variables. These include expected yield (which varies by variety, region and season), demand (which varies by variety and season) and the costs of growing, processing, and transport. Using a deterministic model that did not account for uncertainty, the researchers produced an optimal plan that did better at minimising cost and land-use than a real plan provided by Syngenta and that took much less time to devise.

When they introduced a way of accounting for uncertain yields, factoring in overproduction and underproduction as costs (either in additional production cost or the opportunity cost of failing to meet demand) they found the model considerably reduced the overall cost in situations where the yield was low, with a relatively small increase in cost in the event of an average or high yield.

Photo: Bags of Syngenta AG Golden Harvest brand hybrid seed corn sit stacked in a barn in Princeton, Illinois. Credit: Daniel Acker/Bloomberg via Getty Images

Macroeconomics

We are often uncertain not only about future events but also laws (for example economic laws) that determine whether one event will follow another. Plans can go wrong not only because we fail to foresee future events, but also because we hold incorrect assumptions about the effects our decisions will have.

This is one kind of uncertainty addressed by Emeritus Professor Berc Rustem, leader of Imperial’s Computational Optimisation Group, which he founded over 30 years ago, helping establish a research programme that is still pursued at Imperial and other universities.

In the 1970s, Professor Rustem began work on a long-running research programme to optimise macroeconomic policy with control theorist Professor John Westcott, macroeconomist Professor Maurice Peston (later Lord Peston) and Labour MP Dr Jeremy Bray with funding from research councils (first the ESRC and later the EPSRC). The team advised policymakers such as parliament and HM Treasury and provided written evidence to House of Commons Treasury Committee.

At first, the group produced their own macroeconomic models – sets of assumptions about how the economy works that could be used to help drive optimisation algorithms. However, there was at this time increasing debate among economists about what sort of model to adopt. When Margaret Thatcher became prime minister, she favoured monetarism, which states that the best way to promote economic growth is by using the interest rate to regulate the money supply, over the longer-established demand-side economics, which calls on the government to use spending to regulate demand.

“There was huge controversy, and it became political,” says Professor Rustem. “We therefore abandoned modelling ourselves and focused on using existing models, and let that economic fight take place among the modellers.”

The team began to use models produced by the London Business School, the National Institute of Economic and Social Research, and HM Treasury. “Which model to choose became a real problem,” says Professor Rustem. “When you apply your policy, assuming a monetarist model is a fair representation of the economy gives you one result. If you choose another model and apply your policy, you get another result. This is when we started using robust optimisation. What we did was to combine all these models together in one framework and aimed to decide on a policy that would take into account all possible models and minimise the downside of choosing the wrong model.”

Finance

After the project ended in 1991, Professor Rustem began applying related techniques in chemical engineering and finance. In the latter case, he has focused on portfolio optimisation, the task of deciding which assets to invest in. This, he explains, uses historical data about how assets have risen or declined in value to optimise future returns and manage risk. The latter involves taking account of the uncertainty (e.g. standard deviation, downside risk) gleaned from historical data.

An issue, here, is that we need to consider the kind of worst-case scenario that happens from time to time, such as the 2008 financial crisis. If you base your decisions on the historical change in the value of an asset over the last few years, this may not help protect you against the consequences of a sudden crash.

“Sometimes the worst case happens, and you need to be prepared for it. You can’t always assume that the worst case will happen and stop investing entirely. At the same time, you need to also think deep in the back of your mind that the worst case can and does happen from time to time,” says Professor Rustem. “All this leads to robust optimisation technology that takes into consideration worst-case events, or worst-case probabilities, in the future.”

To protect against the worst case, his portfolio optimisation algorithms make use of competing scenarios, analogous to the competing macroeconomic models built-in to his earlier work. Instead of computing the optimal outcome under a single scenario, they compute an optimal outcome assuming a possible range of scenarios that may come to pass, aiming to minimise the loss occurred in view of the worst case.

Professor Rustem emphasises that this is not a counsel of despair against the possibility of understanding how future events will unfold. Using a refined version of this approach, he computes the probability distribution for a range of scenarios, allowing, for example, a 70% probability that one model or scenario will prove correct and a 30% probability that another will. This makes it possible to reconcile trust in models with a healthy degree of robustness.

Photo: The HM Treasury building in Westminster, London.

Longer term decision-making

We have seen that it is an advantage to have a plan that can be easily adapted when unforeseen events occur, and where this is not possible, to use probability distributions to appropriately balance reward with downside risk. For longer-term decision-making, another valuable tool – which can be combined with both these approaches – is to carry out a plan in phases, adapting it as more information becomes available. The key here is to build in flexibility, the ability to adapt to a changing environment, and recognise the right time to adapt.

Dr Michel-Alexandre Cardin, a senior lecturer in the Dyson School of Design Engineering, develops decision-making techniques for major engineering projects such as the building of new aerospace, building, energy and transportation systems. These require financial investments in the millions or billions and significant physical resources. They are also highly vulnerable to a changing natural environment, economy and technological landscape, and events like the COVID-19 pandemic.

Decision-makers sometimes tie themselves into major projects without building in flexibility, because they are attracted by the economies of scale achieved by committing all at once to significant investments, or they are confident about future projections of major uncertainty drivers like price or demand. However, recent studies have shown that building in flexibility can improve expected value to investors or society by 10–30% and sometimes more.

Dr Cardin offers the example of the Iridium satellite phone network, which launched 66 satellites in a single year in the 90s, anticipating major demand just as land-based mobile phone networks entered the mainstream. The unexpectedly low demand, combined with early engineering decisions that meant the satellites could not adjust their orbits, meant revenue failed to cover the huge $4 billion investment – and ultimately led to the company’s bankruptcy, despite its award-winning technology.

This situation could have been mitigated, Dr Cardin says, if Iridium had adopted a flexible capacity expansion strategy by launching the satellites in phases and designing the satellites to change orbital configuration in space in line with new information on demand and coverage requirements. Many other engineering projects shared a similar story, for example the IUT Global waste-to-energy system in Singapore and Ghost Cities in China.

Real options analysis, a technique that became popular in the 80s and 90s, offers decision-makers a way to value a flexible project. A real option is the right, but not obligation, to take an investment decision such as expanding or reducing capacity or delaying or abandoning investment. To analyse real options, we break a project down into decision-making phases and, given the current state of the project (e.g. capacity) and expected future realisation, determine the value of exercising each option.

This analysis can sometimes show that the value of expanding a system’s capacity in phases, for example the Iridium network, is greater than the value of expanding all at once. The phased approach can be more valuable because it makes it possible to expand capacity only when needed and to expand more than originally planned if conditions are favourable. Moreover, the analysis may value the project as a whole more highly than a valuation that does not break the project down into phases, an approach that might even give the project a negative expected value by failing to incorporate the value of being flexible and leaving options open.

A limitation with the classical real options approach is that it assumes that if a parameter such as the price of the product we are selling increases by £10 and then in the next time period decreases by £10, the value of the asset or project is equivalent to it decreasing and then increasing by the same amount. This is often a bad assumption in industrial contexts, because it does not account for path dependencies, such as the fact that you are likely to expand capacity when the price is high, and that this expansion cannot always be reversed without a significant cost in case of downturns.

Dr Cardin has developed a version of real options analysis better suited for industry practice, which uses decision-rules. These are rules that specify the best time to change the system to capitalise on an upside opportunity or reduce exposure to downside risks, so as to improve the expected value. By using decision-rules, we can account for path dependency by modelling the decisions that will be made in each phase and reflecting these in the values of the choice sets available in subsequent phases. This is computationally complex, and he is currently exploring the role that deep reinforcement learning could play to build and analyse effective models more efficiently than humans can.

In addition to financial decision-making, Dr Cardin examines the engineering decisions – including the mechanical nuts and bolts – that need to be considered at the outset of a project to build flexibility into the design of a system: for Iridium, for instance, each satellite would have needed to be designed to change orbit. His approach is in line with the Dyson School of Design Engineering’s focus on systems thinking, which considers product design and engineering not only from the point of view of user needs, but decision maker’s needs along with other components of the system a product is embedded in. He is currently interacting with private equity firms in infrastructure, who need better data-driven decision support tools to enable them to make best use of their resources and increase value.

“The mission is to create a new mindset,” Dr Cardin says. “We need to design systems in a way that enables flexible decision-making to face future uncertainty, threats, and opportunities. Think of future buildings – these will need to embed more flexibility so we can adapt to future health crises like the COVID-19 pandemic, or climate change. A lot of these decisions will not be possible unless systems are appropriately designed from the outset to build in flexibility.”

Photo: The Delta-II Rocket, carrying the Iridium network's first five satellites, lifts off on 4 May 1997 in California. The satellite phone network was launched just as land-based mobile phone networks were entering the mainstream, leading to unexpectedly low demand. Credit: STR/AFP via Getty Images.