A new machine learning tool from Imperial could help researchers track animal behaviour and pave the way for more AI use in the biological sciences.

Biologists often study large numbers of animals to collect data on collective and individual behaviour. New machine learning tools promise to help scientists process the huge amount of data this work generates more quickly while lessening workload.

Now, a new tool called replicAnt simplifies and streamlines the way the training images for these machine learning tools are created, making it quicker and easier to record observations about lots of animals at once, starting with insects.

Animal database

Existing AI-enabled tools for this purpose require users to painstakingly hand annotate hundreds of frames to provide a database for the computer to learn from. To combat this, replicAnt automatically creates thousands of annotated images with the click of a mouse, seamlessly incorporating variations in species and environments. Ultimately, these AI-generated data may increase the speed and robustness of using AI tools in animal research.

The work is published in Nature Communications.

Lead author Fabian Plum, PhD researcher at Imperial College London’s Department of Bioengineering, said: “It takes a lot of time to set up studies on large numbers of animals, and to learn how to use new tools. replicAnt lowers the entry barrier for biologists to use machine learning to optimise their work.”

“Understanding animal behaviour, particularly as our climate changes, is crucial. We hope our tool can help to make the time-intensive process of collecting crucial data easier and faster.” Fabian Plum First author and PhD researcher in Imperial's Department of Bioengineering

The tool builds on the research team’s previous tool, scAnt – a 3D scanner which photographs small animals in meticulous detail to produce high-resolution 3D models of critters. The 3D models generated by scAnt were used within replicant, which uses the 3D software Unreal Engine, to produce training images for detecting and tracking animals in the lab and in nature, freeing up researchers’ time and streamlining their work.

To demonstrate the utility of replicAnt, the researchers trained neural networks – sets of algorithms that recognise underlying relationships in data – on these images. This allowed the neural networks to recognise individuals and track their movements across different environments out-of-the-box. For others, the required hand-labelling of real images was reduced by an order of magnitude.

Fabian added: “Understanding animal behaviour, particularly as our climate changes, is crucial. We hope our tool can help to make the time-intensive process of collecting crucial data easier and faster.”

Further applications might include using real-time movement data to inform character movement in film and video games.

This project was funded by an Imperial’s President’s PhD Scholarship to Fabian and the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program to Dr David Labonte at the Department of Bioengineering.

“replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine” by Plum et al., published 8 November 2023 in Nature Communications.

All image credits: Fabian Plum