Identification of common native grassland plants in northern China using deep learning

doi:10.17520/biods.2023435

Abstract

Abstract:

Aims: The classification and identification of grassland plants is an essential part of grassland resource surveillance and biodiversity monitoring. Rapid advancements in computer vision and deep learning have created opportunities for automating this process, however, there is currently a shortage of datasets and models specifically tailored for the identification of grassland plants.

Methods: This study established a dataset comprising images of 831 species of native grassland plants in northern China. Employing state-of-the-art image classification architectures based on convolutional neural networks (CNN) and vision transformers (ViT), we trained models for the recognition of grassland plant images. Four models (Eva-02, ResNet_RS, MobileNetV3, and MobileViTv2) were evaluated for accuracy, recognition speed, and size.

Results: Regarding model recognition accuracy, the Top1 accuracy of the Eva-02, MobileViTv2, ResNet_RS, and MobileNetV3 models on the test set were 96.78%, 94.29%, 95.57%, and 91.53%, respectively. The Top5 accuracy on the test set were 99.17%, 98.93%, 98.79%, and 97.56%, respectively. In terms of model size and recognition speed, the MobileNetV3 model exhibited the smallest parameter size and fastest recognition speed, followed by MobileViTv2, making these models suitable for deployment on mobile devices. Conversely, the Eva-02 model had the largest parameter size and the slowest detection speed. Comparing with Pl@ntNet, HuaBanLv, and Baidu-Shitu, all four models developed in this study outperform these three recognition systems.

Conclusion: The plant recognition models trained in this study can recognize the largest number of natural grassland plant species with the highest accuracy compared to other popular recognition systems. The four models strike a balance between model recognition accuracy and performance that is suitable for deployment on both desktop and mobile platforms. They also fulfill the requirements for indoor and outdoor application scenarios.

Key words: grassland, species recognition, deep learning, convolutional neural network, vision transformer

Yongcai Wang, Huawei Wan, Jixi Gao, Zhuowei Hu, Chenxi Sun, Na Lü, Zhiru Zhang. Identification of common native grassland plants in northern China using deep learning[J]. Biodiv Sci, 2024, 32(4): 23435.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.biodiversity-science.net/EN/10.17520/biods.2023435

https://www.biodiversity-science.net/EN/Y2024/V32/I4/23435

Figures/Tables 8

Fig. 1 The example of image augmentation

Fig. 2 Loss variation on the training and validation datasets for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

Fig. 3 The Top1 and Top5 accuracy variation on the validation dataset for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

Table 1 The recognition accuracy for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

模型 Model	测试集 Test		验证集 Valid
模型 Model	Top1 (%)	Top5 (%)	Top1 (%)	Top5 (%)
Eva-02	96.78	99.17	96.25	99.11
MobileViTv2	94.29	98.93	93.97	98.83
ResNet-RS	95.57	98.79	95.34	98.78
MobileNetV3	91.53	97.56	91.30	97.85

Table 2 The parameter size and inference performance for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

模型 Model	参数量 Params (M)	裁剪大小 Crop size	每秒浮点运算次数 Flops	每秒样本数 Samples/s
Eva-02	303.78	448,448	310.15 G	29.62
MobileViTv2	18.40	384,384	16.09 G	197.14
ResNet-RS	93.21	256,256	20.26 G	84.62
MobileNetV3	4.17	256,256	280.44 M	200.12

Fig. 4 The number of species with recognition accuracy exceeding 90% in both Top1 and Top5 for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

Fig. 5 The recognition accuracy of plant species from different families for all four models of Eva-02, MobileViTv2, ResNet-RS, and MobileNetV3

Table 3 The recognition results of three plant identification systems on the test dataset

模型 Model	识别植物种类 Recognized plant species	Top1 (%)	Top5 (%)
Pl@ntNet	207	15.14	26.51
百度识图 Baidu-Shitu	509	56.12	73.59
花伴侣 HuaBanLv	569	41.08	62.87

References 44

[1]	Alzubaidi L, Zhang JL, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8, 1-74.
[2]	Belhumeur PN, Chen DZ, Feiner S, Jacobs DW, Kress WJ, Ling HB, Lopez I, Ramamoorthi R, Sheorey S, White S, Zhang L (2008) Searching the world’s herbaria:A system for visual identification of plant species. In: European Conference on Computer Vision (eds Forsyth D, Torr P, Zisserman A), pp. 116-129. Springer, Berlin.
[3]	Bello I, Fedus W, Du XZ, Cubuk ED, Srinivas A, Lin TY, Shlens J, Zoph B (2021) Revisiting ResNets: Improved training and scaling strategies. Advances in Neural Information Processing Systems, 34, 22614-22627.
[4]	Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA (2020) Albumentations: Fast and flexible image augmentations. Information, 11, 125.
[5]	Chytrý M, Schaminée JHJ, Schwabe A (2011) Vegetation survey: A new focus for Applied Vegetation Science. Applied Vegetation Science, 14, 435-439.
[6]	Donahue J, Jia YQ, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) DeCAF: A deep convolutional activation feature for generic visual recognition. Proceedings of the 31st International Conference on International Conference on Machine Learning, 32, 647-655.
[7]	Elhariri E, El-Bendary N, Hassanien AE (2014) Plant classification system based on leaf features. In: 2014 9th International Conference on Computer Engineering & Systems (ICCES), pp. 271-276. IEEE, Cairo, Egypt.
[8]	Fang YX, Sun Q, Wang XG, Huang TJ, Wang XL, Cao Y (2023) Eva-02: A visual representation for neon genesis. ArXiv, doi: 2303.11331.
[9]	Garcin C, Joly A, Bonnet P, Lombardo JC, Affouard A, Chouet M, Salmon J (2021) Pl@ntNet-300K:A plant image dataset with high label ambiguity and a long-tailed distribution. In: NeurIPS 2021—35th Conference on Neural Information Processing Systems. Online.
[10]	Gao HY, Gao XH, Feng QS, Li WL, Lu Z, Liang TG (2020) Approach to plant species identification in natural grasslands based on deep learning. Pratacultural Science, 37, 1931-1939. (in Chinese with English abstract)
	[高宏元, 高新华, 冯琦胜, 李文龙, 鲁征, 梁天刚 (2020) 基于深度学习的天然草地植物物种识别方法. 草业科学, 37, 1931-1939.]
[11]	Goëau H, Bonnet P, Joly A (2022) Overview of PlantCLEF 2022:Image-based plant identification at global scale. In: Conference and Labs of the Evaluation Forum (CLEF) (Working Notes), pp. 1916-1928. Bologna, Italy.
[12]	Goëau H, Joly A, Bonnet P, Selmi S, Molino JF, Barthélémy D, Boujemaa N (2014) LifeCLEF plant identification task 2014. In: Conference and Labs of the Evaluation Forum (CLEF), pp. 598-615. Sheffield, UK.
[13]	Gyires-Tóth BP, Osváth M, Papp D, Szűcs G (2019) Deep learning for plant classification and content-based image retrieval. Cybernetics and Information Technologies, 19, 88-100.
[14]	Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, Tang Y, Xiao A, Xu C, Yang Z, Zhang Y, Tao D (2022) A survey on vision transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 87-110.
[15]	Hart AG, Bosley H, Hooper C, Perry J, Sellors-Moore J, Moore O, Goodenough AE (2023) Assessing the accuracy of free automated plant identification applications. People and Nature, 5, 929-937.
[16]	Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M, Wang W, Zhu Y, Pang R, Vasudevan V, Le Q, Adam H (2019) Searching for MobileNetv3. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 1314-1324.
[17]	Joly A, Bonnet P, Goëau H, Barbe J, Selmi S, Champ J, Dufour-Kowalski S, Affouard A, Carré J, Molino JF, Boujemaa N, Barthélémy D (2016) A look inside the Pl@ntNet experience: The good, the bias and the hope. Multimedia Systems, 22, 751-766.
[18]	Joly A, Goëau H, Glotin H, Spampinato C, Bonnet P, Vellinga WP, Planque R, Rauber A, Fisher B, Müller H (2014) LifeCLEF:Multimedia life species identification. In: Environmental Multimedia Retrieval, pp. 7-13. Sheffield, UK.
[19]	Joly A, Goëau H, Kahl S, Picek L, Lorieul T, Cole E, Šulc M (2022) LifeCLEF 2022 teaser: An evaluation of machine- learning based species identification and species distribution predictio. In: European Conference on Information Retrieval, pp. 390-399. Springer International Publishing, Cham.
[20]	Kong JX, Zhang ZC, Zhang J (2019) Classification and identification of plant species based on multi-source remote sensing data: Research progress and prospect. Biodiversity Science, 27, 796-812. (in Chinese with English abstract) DOI
	[孔嘉鑫, 张昭臣, 张健 (2019) 基于多源遥感数据的植物物种分类与识别: 研究进展与展望. 生物多样性, 27, 796-812.] DOI
[21]	Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez IC, Soares JV (2012) Leafsnap: A computer vision system for automatic plant species identification, pp. 502-516. Springer, Berlin.
[22]	Lewkowycz A (2021) How to decay your learning rate. arXiv, doi: 2103.12682.
[23]	Li M (2018) “flower partners”: A new medium of knowledge service in the era of artificial intelligence. Publishing Reference, (8), 23-24. (in Chinese)
	[李敏 (2018) “花伴侣”:人工智能时代知识服务的新媒介. 出版参考, (8), 23-24.]
[24]	Mehta S, Rastegari M (2022) Separable self-attention for mobile vision transformers. arXiv, doi: 2206.02680.
[25]	Munisami T, Ramsurn M, Kishnah S, Pudaruth S (2015) Plant leaf recognition using shape features and colour histogram with K-nearest neighbour classifiers. Procedia Computer Science, 58, 740-747.
[26]	Parr CS, Wilson MN, Leary MP, Schulz KS, Lans MK, Walley ML, Corrigan Jr MRJ (2014) The Encyclopedia of Life v2: Providing global access to knowledge about life on earth. Biodiversity Data Journal, 2, e1079.
[27]	Picek L, Šulc M, Matas J, Heilmann-Clausen J, Jeppesen TS, Lind E (2022a) Automatic fungi recognition: Deep learning meets mycology. Sensors, 22, 633.
[28]	Picek L, Šulc M, Patel Y, Matas J (2022b) Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings. Frontiers in Plant Science, 13, 787527.
[29]	Prasad S, Kudiri KM, Tripathi RC (2011) Relative sub-image based features for leaf recognition using support vector machine. In:Proceedings of the 2011 International Conference on Communication, Computing & Security, 343-346.
[30]	Rahman A, Khanom A (2013) Taxonomic and ethno-medicinal study of species from Moraceae (mulberry) family in Bangladesh Flora. Research in Plant Sciences, 1(3), 53-57.
[31]	Robertson T, Döring M, Guralnick R, Bloom D, Wieczorek J, Braak K, Desmet P (2014) The GBIF integrated publishing toolkit: Facilitating the efficient publishing of biodiversity data on the internet. PLoS ONE, 9, e102623.
[32]	Rzanny M, Wittich HC, Mäder P, Deggelmann A, Boho D, Wäldchen J (2022) Image-based automated recognition of 31 Poaceae species: The most relevant perspectives. Frontiers in Plant Science, 12, 804140.
[33]	Shalev-Shwartz S, Ben-David S (2014) Understanding Machine Learning:From Theory to Algorithms. Cambridge University Press, Cambridge.
[34]	Sharif Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: An astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 806-813.
[35]	Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv, doi: 1409.1556.
[36]	Song G, Wang Q (2023) Species classification from hyperspectral leaf information using machine learning approaches. Ecological Informatics, 76, 102141.
[37]	Sun Y, Liu Y, Wang G, Zhang H (2017) Deep learning for plant identification in natural environment. Computational Intelligence and Neuroscience, 2017, 2-4.
[38]	Šulc M, Matas J (2017) Fine-grained recognition of plants from images. Plant Methods, 13, 115. DOI PMID
[39]	Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Rabinovich A (2014) Going deeper with convolutions. arXiv, doi: 1409.4842v1.
[40]	Van Horn G, Mac Aodha O, Song Y, Cui Y, Sun C, Shepard A, Belongie S (2018) The iNaturalist Species Classification and Detection Dataset-Supplementary Material. Reptilia, 32, 1-3.
[41]	Wang JX (2019) Research on the communication effect of “Huajian” article on shape-color App based on “5W” communication mode. Audio-Visual, 142(2), 131-132. (in Chinese)
	[王加希 (2019) 基于“5W”传播模式对形色App “花间”文章传播效果的探究. 视听, 142(2), 131-132.]
[42]	Wäldchen J, Mäder P (2018) Machine learning for image based species identification. Methods in Ecology and Evolution, 9, 2216-2225.
[43]	Xu ZH, Liu SY, Zhao Y, Tu WQ, Chang ZF, Zhang ET, Guo J, Zheng D, Geng J, Gu GY, Guo CP, Guo LL, Wang J, Xu CY, Peng C, Yang T, Cui MQ, Sun WC, Zhang JT, Liu HT, Ba CQ, Wang HQ, Jia JC, Wu JZ, Xiao C, Ma KP (2020) Evaluation of the identification ability of eight commonly used plant identification application softwares in China. Biodiversity Science, 28, 524-533. (in Chinese with English abstract) DOI
	[许展慧, 刘诗尧, 赵莹, 涂文琴, 常诏峰, 张恩涛, 郭靖, 郑迪, 耿鋆, 顾高营, 郭淳鹏, 郭璐璐, 王静, 徐春阳, 彭钏, 杨腾, 崔梦琪, 孙伟成, 张剑坛, 刘皓天, 巴超群, 王鹤琪, 贾竞超, 武金洲, 肖翠, 马克平 (2020) 国内8款常用植物识别软件的识别能力评价. 生物多样性, 28, 524-533.] DOI
[44]	Zhang S, Huang W, Huang YA, Zhang C (2020) Plant species recognition methods using leaf image: Overview. Neurocomputing, 408, 246-272.