Cross-regional bird species recognition method integrating audio and ecological niche information

doi:10.17520/biods.2024259

Abstract

Abstract:

Aim: Passive acoustic monitoring plays a pivotal role in studying avian populations, community dynamics, and behaviors. For extensive passive acoustic monitoring, employing deep learning techniques to automatically identify bird species from their vocalizations is essential. However, closely related species often produce highly similar calls, leading to confusion and false positives, which can compromise the effectiveness of deep learning models. This paper presents a novel method that integrates audio data with ecological niche information to enhance species recognition accuracy. Here, ‘ecological niche’ encompasses a species’ role in its environment, including its habitat, diet, and behavior.

Methods: The approach begins with the development of an audio recognition model using the ResNet18, a prominent deep learning framework known for its capability to extract high-level features from audio signals. Subsequently, a maximum entropy model is employed to estimate the distribution of bird species and derive ecological suitability indices for various locations. These indices provide the necessary ecological niche information. An integrated model, NicheNet, is then constructed to combine audio features with ecological niche data for improved species recognition.

Results & Conclusion: The integration of audio and ecological niche information through NicheNet demonstrates substantial improvements in recognition accuracy. Specifically, NicheNet enhances Top-1 recognition accuracy by 12.9% and Top-5 recognition accuracy by 10.6% compared to the ResNet18 model. Additionally, NicheNet reduces the near species error rate by 3.1%, the near genus error rate by 1.8%, and the near family error rate by 8.0%. Analysis of recognition outcomes for congeneric species with similar vocalizations reveals that NicheNet significantly refines classification by leveraging ecological niche information, thereby improving the discrimination of vocally similar but ecologically distinct species. This method effectively addresses the challenge of misidentification among closely related and vocally similar bird species that differ in their ecological niches, thereby advancing the accuracy of cross-regional bird species recognition based on vocalizations.

Key words: passive acoustic monitoring, bird vocalization recognition, residual network, deep learning, ecological niche

Jiangjian Xie, Chen Shen, Feiyu Zhang, Zhishu Xiao. Cross-regional bird species recognition method integrating audio and ecological niche information[J]. Biodiv Sci, 2024, 32(10): 24259.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.biodiversity-science.net/EN/10.17520/biods.2024259

https://www.biodiversity-science.net/EN/Y2024/V32/I10/24259

Figures/Tables 11

Fig. 1 The map of bird audio files used in this study. The recording locations of all bird audio used are shown as orange dots.

Table 1 Structure and parameters of audio recognition model

层名 Layer name	输出尺寸 Output size	每层参数 Layer parameters
输入 Input	3 × 80 × 157	-
conv1_x	64 × 40 × 79
conv2_x	64 × 20 × 40
conv2_x	64 × 20 × 40	双重堆叠卷积模块 Double stacked convolutional module $3 × 3, 64 3 × 3, 64 × 2$
conv3_x	128 × 10 × 20
conv4_x	256 × 5 × 10
conv5_x	512 × 3 × 5
平均池化 Average pool	512 × 1 × 1	自适应平均池化 Adaptive average pool
输出 Output	1 × 156	全连接层 Fully connected layer: Softmax

Table 1 Structure and parameters of audio recognition model

层名 Layer name	输出尺寸 Output size	每层参数 Layer parameters
输入 Input	3 × 80 × 157	-
conv1_x	64 × 40 × 79
conv2_x	64 × 20 × 40
conv2_x	64 × 20 × 40	双重堆叠卷积模块 Double stacked convolutional module $3 × 3, 64 3 × 3, 64 × 2$
conv3_x	128 × 10 × 20
conv4_x	256 × 5 × 10
conv5_x	512 × 3 × 5
平均池化 Average pool	512 × 1 × 1	自适应平均池化 Adaptive average pool
输出 Output	1 × 156	全连接层 Fully connected layer: Softmax

Fig. 2 Examples of predicted results of species distribution. Blue to red indicates the predicted probability of the species’ presence from 0 to 1.

Fig. 3 Flow chart of obtaining niche information

Fig. 4 The architecture of NicheNet. MFCC, Mel-scale frequency cepstral coefficients.

Table 2 Comparison of different model performance

模型名称 Model name	Top-1准确率 Top-1 accuracy	Top-5准确率 Top-5 accuracy	近种错误率 Near species error rate	近属错误率 Near genus error rate	近科错误率 Near family error rate
ResNet18	0.6140	0.8007	0.0531	0.0810	0.2518
NicheNet	0.7432	0.9062	0.0220	0.0630	0.1719

Fig. 5 Comparison of the number of recognition errors of different models. ResNet18, Residual Neural Network 18. NicheNet, A recognition model integrating audio and niche information.

Fig. 6 Spectrograms and distribution prediction results of Cossypha caffra and Cercotrichas leucophrys. In figure c and d, blue to red indicates the predicted probability of the species’ presence from 0 to 1.

Table 3 Misidentification of Cossypha caffra

模型名称 Model name	误识别样本数 Number of misidentified samples (%)	近属误识别样本数 Number of near genus misidentified samples (%)	误识别为白眉薮鸲样本数 Number of samples misidentified as Cercotrichas leucophrys (%)
ResNet18	283 (41.6%)	124 (18.2%)	68 (10.0%)
NicheNet	188 (27.6%)	46 (6.8%)	14 (2.1%)

Fig. 7 Spectrograms and distribution prediction results of Delichon urbicum and Hirundo rustica. In figure c and d, blue to red indicates the predicted probability of the species’ presence from 0 to 1.

Table 4 Misidentification of Delichon urbicum

模型名称 Model name	误识别样本数 Number of misidentified samples (%)	近属误识别样本数 Number of near genus misidentified samples (%)	误识别为家燕样本数 Number of samples misidentified as Hirundo rustica (%)
ResNet18	328 (35.2%)	104 (11.2%)	56 (6.0%)
NicheNet	314 (33.7%)	98 (10.5%)	48 (5.1%)

References 23

[1]	Bao YY, Li YK, Lin WY, Zhou ZQ, Xiao XB, Xie XY (2023) The current situation of horseshoe crabs in the offshore waters of northern South China Sea with analysis of the potential habitat distribution of juvenile Tachypleus tridentatus in Beibu Gulf. Biodiversity Science, 31, 22407. (in Chinese with English abstract)
	[鲍虞园, 李银康, 林吴颖, 周志琴, 肖晓波, 颉晓勇 (2023) 中国南海北部近海鲎资源调查及北部湾潮间带中华鲎幼鲎潜在栖息地评估. 生物多样性, 31, 22407.] DOI
[2]	Bold N, Zhang C, Akashi T (2019) Cross-domain deep feature combination for bird species classification with audio-visual data. IEICE Transactions on Information and Systems, E102, 2033-2042.
[3]	Bota G, Manzano-Rubio R, Catalán L, Gómez-Catasús J, Pérez-Granados C (2023) Hearing to the unseen: AudioMoth and BirdNET as a cheap and easy method for monitoring cryptic bird species. Sensors, 23, 7176.
[4]	Cai JM, He PY, Yang ZP, Li LY, Zhao QJ, Pan F (2023) A deep feature fusion-based method for bird sound recognition and its interpretability analysis. Biodiversity Science, 31, 23087. (in Chinese with English abstract) DOI
	[蔡建民, 何培宇, 杨智鹏, 李露莹, 赵启军, 潘帆 (2023) 基于深度特征融合的鸟鸣识别方法及其可解释性分析. 生物多样性, 31, 23087.] DOI
[5]	Chu G, Potetz B, Wang WJ, Howard A, Song Y, Brucher F, Leung T, Adam H (2019) Geo-aware networks for fine-grained recognition. In:2019 IEEE/CVF International Conference on Computer Vision Workshop, pp. 247-254. Seoul, Korea (South).
[6]	Feng XJ, Mi XC, Xiao ZS, Cao L, Wu H, Ma KP (2019) Overview of Chinese biodiversity observation network (Sino BON). Bulletin of Chinese Academy of Sciences, 34, 1389-1398. (in Chinese with English abstract)
	[冯晓娟, 米湘成, 肖治术, 曹垒, 吴慧, 马克平 (2019) 中国生物多样性监测与研究网络建设及进展. 中国科学院院刊, 34, 1389-1398.]
[7]	Florentin J, Dutoit T, Verlinden O (2020) Detection and identification of European woodpeckers with deep convolutional neural networks. Ecological Informatics, 55, 101023.
[8]	Jeantet L, Dufourq E (2023) Improving deep learning acoustic classifiers with contextual information for wildlife monitoring. Ecological Informatics, 77, 102256.
[9]	Kahl S, Wood M, Eibl M, Klinck H (2021) BirdNET: A deep learning solution for avian diversity monitoring. Ecological Informatics, 61, 101236.
[10]	Lin CT, Huang XW, Wang JN, Xi TY, Ji LQ (2021) Learning niche features to improve image-based species identification. Ecological Informatics, 61, 101217.
[11]	Liu J, Zhang Y, Lv D, Lu J, Xie SS, Zi JL, Yin Y, Xu HF (2022) Birdsong classification based on ensemble multi scale convolutional neural network. Scientific Reports, 12, 8636.
[12]	Mei JJ (2022) Study on the Patterns of Vocal Activity and Interspecific Relationship of the Cuculidae Species in Dabie Mountains. PhD dissertation, University of Science and Technology of China, Hefei. (in Chinese with English abstract)
	[梅金娟 (2022) 大别山区杜鹃科鸟类鸣叫活动模式与种间关系研究. 博士学位论文, 中国科学技术大学, 合肥.]
[13]	Morales G, Vargas V, Espejo D, Poblete V, Tomasevic JA, Otondo F, Navedo JG (2022) Method for passive acoustic monitoring of bird communities using UMAP and a deep neural network. Ecological Informatics, 72, 101909.
[14]	Peterson AT, Soberón J, Sánchez-Cordero V (1999) Conservatism of ecological niches in evolutionary time. Science, 285, 1265-1267. DOI PMID
[15]	Phillips SJ, Anderson RP, Schapire RE (2006) Maximum entropy modeling of species geographic distributions. Ecological Modelling, 190, 231-259.
[16]	Thant ZM, Leimgruber P, Williams AC, Oo ZM, Røskaft E, May R (2023) Factors influencing the habitat suitability of wild Asian elephants and their implications for human- elephant conflict in Myanmar. Global Ecology and Conservation, 43, e02468.
[17]	Wang HL, Xu YF, Yu Y, Lin YC, Ran JH (2022) An efficient model for a vast number of bird species identification based on acoustic features. Animals, 12, 2434.
[18]	Wu KY, Ruan WD, Zhou DF, Chen QC, Zhang CY, Pan XY, Yu S, Liu Y, Xiao RB (2023) Syllable clustering analysis-based passive acoustic monitoring technology and its application in bird monitoring. Biodiversity Science, 31, 22370. (in Chinese with English abstract) DOI
	[吴科毅, 阮文达, 周棣锋, 陈庆春, 张承云, 潘新园, 余上, 刘阳, 肖荣波 (2023) 基于音节聚类分析的被动声学监测技术及其在鸟类监测中的应用. 生物多样性, 31, 22370.] DOI
[19]	Xie JJ, Yang J, Xing ZL, Zhang Z, Chen X (2020) Bird species recognition method based on multi-feature fusion. Journal of Applied Acoustics, 39, 199-206. (in Chinese with English abstract)
	[谢将剑, 杨俊, 邢照亮, 张卓, 陈新 (2020) 多特征融合的鸟类物种识别方法. 应用声学, 39, 199-206.]
[20]	Xie JJ, Zhong YJ, Zhang JG, Liu S, Ding CQ, Triantafyllopoulos A (2023) A review of automatic recognition technology for bird vocalizations in the deep learning era. Ecological Informatics, 73, 101927.
[21]	Xie ZF, Li DZ, Sun HX, Zhang AM (2023) Deep learning techniques for bird chirp recognition task. Biodiversity Science, 31, 22308. (in Chinese with English abstract) DOI
	[谢卓钒, 李鼎昭, 孙海信, 张安民 (2023) 面向鸟鸣声识别任务的深度学习技术. 生物多样性, 31, 22308.] DOI
[22]	Xu J, Zhang XQ, Zhao CM, Geng ZL, Feng YR, Miao K, Li YJ (2024) Improving fine-grained image classification with multimodal information. IEEE Transactions on Multimedia, 26, 2082-2095.
[23]	Yan N, Chen AB, Zhou GX, Zhang ZQ, Liu XY, Wang JW, Liu ZH, Chen WJ (2021) Birdsong classification based on multi-feature fusion. Multimedia Tools and Applications, 80, 36529-36547.