Using deep learning techniques to detect and classify weeds in Fagopyrum esculentum crops

In recent years, the technology of weed mapping using unmanned aerial vehicles has been actively developing. Determining the location of weeds makes it possible to create task maps for herbicide treatments. This helps to reduce the amount of herbicides used and reduce the environmental load. However, to select the optimal control strategy, it is necessary to identify the species composition, quantitative abundance and growth phases of weeds in a particular field. The development of deep learning methods based on convolutional neural networks helps in solving this problem. In particular, the trained algorithm is able to automatically extract information from images, detect and classify weeds. The article discusses the construction of image classifiers using the ResNet-18, ResNet-34 and ResNet-50 architecture. The results of phytosanitary monitoring of buckwheat (Fagopyrum esculentum Moench) crops and photographs of survey sites are used as initial information. The crops were dominated by annual monocotyledonous weeds – wild oat (Avena fatua L.), barnyard grass (Echinochloa crus-galli (L.) Beauv.) and field bindweed (Convolvulus arvensis L.). According to the quantitative assessment of the weeds in the study sites, each species was assigned an identifier of 0 – for those exceeding the economic threshold of harmfulness and 1 – for those not exceeding it. The task of the classifier is to recognize these weeds in the photograph and determine one of two gradations of weediness of the site (low or high). The efficiency of the proposed approach is confirmed by a sufficiently high quality of the classifier predictions (the number of correct classifications for the original set of 24 images varies from 83 to 88%) and the construction of a Confusion matrix. 

1. Jones J.W., Antle J., Basso B., Boote K.J., Conant R.T., Foster I., Herrero M., Janssen S., Porter C.H., Wheeler T.R. Brief history of agricultural systems modeling. Agricultural Systems, 2017, vol. 155 (3), pp. 240–254.

2. Walter A., Finger R., Huber R., Buchmann N. Smart farming is key to developing sustainable agriculture. Proceedings of the National Academy of Sciences, 2017, vol. 114, no. 24, pp. 6148– 6150.

3. Caldera U., Breyer C. Assessing the potential for renewable energy powered desalination for the global irrigation sector. Science of the total environment, 2019, vol. 694, p. 133598.

4. Dubenok N.N., Zayats O.A., Strizhakova E.A. Formation of buckwheat (Fagopyrum esculentum L.) productive capacity depending on the level of mineral fertilization and sowing method. Izvestiya Timiryazevskoy sel’skokhozyaistvennoy akademii = Izvestiya of Timiryazev Agricultural Academy, 2017, no. 6, pp. 29–41. (In Russian). DOI: 10.26897/0021-342X-2017-6-29-41.

5. Ahmad A., Saraswat D., Aggarwal V., Etienne A., Hancock B. Performance of deep learning models for classifying and detecting common weeds in corn and soybean production systems. Computers and electronics in agriculture, 2021, vol. 184, p. 106081. DOI: 10.1016/j.compag.2021.106081.

6. Hasan A.M., Sohel F., Diepeveen D., Laga H., Jones M.G. A survey of deep learning techniques for weed detection from images. Computers and electronics in agriculture, 2021, vol. 184, p. 106067. DOI: 10.1016/j.compag.2021.106067.

7. Lottes P., Behley J., Chebrolu N., Milioto A., Stachniss C. Robust joint stem detection and crop‐weed classification using image sequences for plant‐specific treatment in precision farming. Journal of field robotics, 2020, vol. 37, no. 1, pp. 20–34. DOI: 10.1002/rob.21901.

8. Dian Bah M., Hafiane A., Canals R. Deep Learning with unsupervised data labeling for weeds detection on UAV images. arXiv, 2018, no. 1805, p. 12395.

9. Priya S.J., Sundar G.N., Narmadha D., Ebenezer S. Identification of weeds using HSV color spaces and labelling with machine learning algorithms. International journal of recent technology and engineering, 2019, vol. 8, no. 1, pp. 1781–1786.

10. Riksen V., Shpak V. Convolutional neural network for identification and classification of weeds in buckwheat crops. Machine intelligence for smart applications: opportunities and risks. Cham: Springer Nature Switzerland, 2023, pp. 61–72. DOI: 10.1007/978-3-031-37454-8_3.

11. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, vol. 60, no. 6, pp. 84–90. DOI: 10.1145/3065386.

12. Howard D., Gugger S. Deep learning with fastai and PyTorch: minimum formulas, minimum code, maximum efficiency. S.-Petersburg: Piter, 2022, 624 p. (In Russian).