Formula milk trials are not reliable, warn researchers

Researchers find trials lack scientific rigour and say change needed to protect trial participants and protect consumers from misleading information.

Formula milk trials have a high risk of bias, almost always report favourable conclusions, lack transparency, and selectively report findings, says a review of evidence by Imperial College London researchers published by The BMJ today.

The study reviewed 125 recently published trials. 

Study author Dr Robert Boyle from Imperial’s National Heart & Lung Institute, said: “These findings support the need for a substantial change in the conduct and reporting of formula trials to adequately protect participants from harm and protect consumers from misleading information.”

Evaluating formula milk trials

Formula milk is consumed by most European and North American infants, and new formula products need to be tested in clinical trials. But concerns have been raised that formula trials are biased and could undermine breastfeeding.

“These findings support the need for a substantial change in the conduct and reporting of formula trials." Dr Robert Boyle Imperial's National Heart & Lung Institute

To explore this further, the international team of researchers set out to evaluate the conduct and reporting of formula milk trials. They were specifically interested in understanding the risk of bias in published trials and if trial procedures could cause harm by undermining breastfeeding of participants.

They carried out a detailed analysis of 125 trials published since 2015 that compared at least two formula milk products in a total of 23,757 children less than three years of age. 

The trials were mainly carried out in Europe, Asia, or North America, and the most common outcome measures were weight gain, intestinal health, nutrient absorption, behaviour, and allergy. 

The researchers found that only 17 (14%) of these trials were conducted independently of formula companies, 26 (21%) were prospectively registered with a clear aim and main outcome, and 11 (9%) had a publicly available protocol.

High risk of bias 

Most (80%) of trials had a high risk of bias for their main outcome, usually because of inappropriate exclusions of participants from the analysis, and selective reporting (when results are not fully or accurately reported, to suppress negative or undesirable findings). 

Favourable conclusions were reported for more than 90% of trials.

While the study is a comprehensive evaluation, the researchers acknowledge that many formula trials are not in the public domain, and so their findings cannot fully represent all trials conducted.

Nevertheless, they say their study demonstrates that formula trials are not reliable and do not adequately protect participants in the trials. 

First author Bartosz Helfer, from Imperial’s National Heart and Lung Institute, said: “The formula industry is closely involved in formula trials, findings are almost always reported as favourable, and little transparency exists about the aims of the trial or reporting of results.”

Co-author, Dr Zhaoli Dai-Keller from The University of Sydney, said: “The widespread use of formula products, their associated health risks, and the high individual levels of intake during early development highlight the importance of the evidence generated from formula milk trials.”

This article is adapted from materials created by The BMJ.