Attention-Driven and Hierarchical Feature Fusion Network for Crop and Weed Segmentation with Fractal Dimension Estimation

Abstract

In precision agriculture, semantic segmentation enhances the crop yield by enabling precise disease monitoring, targeted herbicide application, and accurate crop-weed differentiation. This enhances yield; reduces the overuse of herbicides, water, and fertilizers; lowers labor costs; and promotes sustainable farming. Deep-learning-based methods are particularly effective for crop and weed segmentation, and achieve potential results. Typically, segmentation is performed using homogeneous data (the same dataset is used for training and testing). However, previous studies, such as crop and weed segmentation in a heterogeneous data environment, using heterogeneous data (i.e., different datasets for training and testing) remain inaccurate. The proposed framework uses patch-based augmented limited training data within a heterogeneous environment to resolve the problems of degraded accuracy and the use of extensive data for training. We propose an attention-driven and hierarchical feature fusion network (AHFF-Net) comprising a flow-constrained convolutional block, hierarchical multi-stage fusion block, and attention-driven feature enhancement block. These blocks independently extract diverse fine-grained features and enhance the learning capabilities of the network. AHFF-Net is also combined with an open-source large language model (LLM)-based pesticide recommendation system made by large language model Meta AI (LLaMA). Additionally, a fractal dimension estimation method is incorporated into the system that provides valuable insights into the spatial distribution characteristics of crops and weeds. We conducted experiments using three publicly available datasets: BoniRob, Crop/Weed Field Image Dataset (CWFID), and Sunflower. For each experiment, we trained on one dataset and tested on another by reversing the process of the second experiment. The highest mean intersection of union (mIOU) of 65.3% and F1 score of 78.7% were achieved when training on the BoniRob dataset and testing on CWFID. This demonstrated that our method outperforms other state-of-the-art approaches.