(Article by Engela Duvenhage for the digital research magazine of the Division for Research Development (DRD))
Dr Emmanuel Dufourq has never seen one of the last 30 remaining Hainan gibbons on Earth, but he knows very well what their calls sound like, or at least what a spectrogram image of the highs and lows of their chatter looks like.
“Their calls are very singsong. They sing individually and then, at some point, they sing together. It’s a very pleasant sound,” says Dufourq, the African Institute for Mathematics Sciences (AIMS) Junior Chair in Data Science for Climate Resilience and a lecturer in Stellenbosch University’s (SU’s) School for Data Science and Computational Thinking and its Department of Industrial Engineering.
Hainan gibbons (Nomascus hainanus) are considered the rarest apes on Earth, and among the world’s most threatened mammals. Most of these gibbons live in two square kilometres of forest on Hainan Island in the South China Sea. A gibbon weighs about as much as a human baby of a few months. Males are almost completely black in colour, while females sport a combination of golden to buff fur and black patches on their heads. The nearest Dufourq has come to visiting the home of these rare apes was in 2013 when he visited Vietnam to present a paper on his MSc work. The latter focused on using genetic programming for data classification, a technique in the field of artificial intelligence.
“My journey has evolved from wanting to be a computer scientist to wanting to do something impactful. And then I found ecology,” explains Dufourq, an avid nature photographer.
These days, he is something of a ‘computer conservationist’. As the leader of the Data Science for Eco-Systems (DS4ES) research group at SU, Dufourq uses his computational skills to make it easier for conservation ecologists to analyse the large volumes of animal sound, video and photo footage that they capture for their research. His group’s mission is to do this one keyboard stroke, one algorithm at a time.
“We are trying to use all the tools we have in the fields of mathematics and machine learning to support conservation ecology projects, and to assist field rangers,” he explains.
“We are one of the few research groups in Africa using machine learning for ecological outcomes. It’s really rewarding to know that, in the process, we are developing students to improve the combination of these two fields,” says Dufourq, who works with collaborators at, for instance, AIMS, the University of Cape Town, the University of KwaZulu-Natal, the University of Lisbon in Portugal, the University of St Andrews in the United Kingdom, and various non-profit organisations around the world.
Dufourq’s interest in bioacoustics and the manner in which it can be used to conserve the planet and its inhabitants was first piqued in 2019. He was finishing his PhD in applied mathematics at the University of Cape Town when a statistical ecologist told him about an interesting conundrum faced by a group of British ecologists.
“The Zoological Society of London, along with their collaborators, had collected a very large dataset of about 1,5 terabytes of gibbon call recordings – hours’ worth – but did not have anyone to process it.”
To easily transform these recordings into something truly useful to researchers became Dufourq’s mission during his postdoctoral years. His work has since shown that methods such as data augmentation can be used to automatically and quite accurately convert sound recordings into spectrograms, which are in turn automatically classified as images.
“Our best model converted acoustic recordings into spectrogram images on the mel frequency scale, using these to train a convolutional neural network,” Dufourq and his fellow researchers wrote in a 2021 article on this topic, published in Remote Sensing in Ecology and Conservation.
“Our model is a time-saver of note”, he says. “A post-processing step that identifies intervals of repeated calling managed to reduce an eight-hour recording, for the purposes of manual processing, to an average of only 22 minutes. Moreover, it did not miss a single calling bout over the 72 hours of test recordings.”
Dufourq believes the model can be adapted for use in the analysis of other animal sounds too. He and his students have since started working on algorithms to automatically monitor animal behaviour captured in large datasets of video footage and images. The specific focus of the students’ work ranges from the body language of endangered African penguins to the calls of Indian humpback dolphins in False Bay, understudied beaked whales, spotted hyenas, endangered birds in Malawi, and the critically endangered black-and-white ruffed lemurs in Madagascar. Some are developing an algorithm to identify and count the birds caught on webcam footage in South Africa’s national parks.
Much effort is being put into compressing the sound files that researchers collect so that it is easier to analyse in real time, as opposed to only after the recordings have been made. The conservation of the planet is top of mind in all these efforts. “Model development and implementation must always be informed and guided by ecological objectives,” Dufourq emphasises. “We cannot solve climate change, but we can at least solve the problem of ecologists wasting 6 months of work on processing 10 terabytes of data, when we can do it in 10 minutes.”
Photo: (insert) Dr Emmanuel Dufourq
????: Jessica Bryant (Zoological Society of London)
This article has originally been published in Stellenbosch University’s new publication, Research for Impact.
