Attention-Fused ConvNeXt and MobileViT Framework with FastSAM Lesion Extraction for Multi-Class Crop Disease Detection

The problem of crop disease recognition from leaf images is a difficult multi-class visual classification problem due to disease evidence appearing at various spatial scales and the possibility of image content other than disease symptom including healthy tissue, background information, and visually similar symptoms. This paper presents an attention-fused dual-branch framework that combines Fast Segment Anything (FastSAM) based lesion extraction, ConvNeXt based full-leaf representation learning, and MobileViT based lesion-focused representation learning. The experimental study adopted a controlled subset of 4000 images from the 20k+ Multi-Class Crop Disease Images collection and applied class normalization and removed one small class, leaving 41 classes. There were 2777 training images, 588 validation images, and 635 testing images in the final split. The candidate masks were obtained by FastSAM, and then filtered and ranked with lesion-oriented criteria, then the selected region was cropped and refined by performing Lab-space contrast enhancement, robust chromatic deviation, Otsu thresholding and morphological operations. The original leaf was processed by ConvNeXt and the refined ROI processed by MobileViT. They were projected into a common 256-dimensional feature space and aggregated using a learnable two-branch gate that includes entropy regularization. The Fusion Model got 93.07% accuracy and 85.27% macro F1 on the common test set, while ConvNeXt got 89.76% accuracy and 81.55% macro F1, and MobileViT got 72.76% accuracy and 68.07% macro F1 on the common test set. The Fusion Model also achieved a one-versus-rest macro-average AUC of 0.9978. Grouped confusion matrices indicate that the fusion approach reduces off-diagonal errors at the crop-group level. A prototype of a Streamlit deployment was also created to showcase the following features: upload of images, lesion preprocessing, prediction, reporting of confidence, inspection of branches, and image visualization with Grad-CAM. The results support the use of a combination of global and lesion-specific representations rather than a single visual pathway.

Crop disease detection, FastSAM, lesion extraction, ConvNeXt, MobileViT, attention fusion, deep learning

1. Zhu, D., Tan, J., Wu, C., Yung, K., & Ip, A. W. H. (2023). Crop disease identification by fusing multiscale convolution and vision transformer. Sensors, 23(13), 6015. [Google Scholar] [Crossref]

2. Tonmoy, M. R., Hossain, M. M., Dey, N., & Mridha, M. F. (2025). MobilePlantViT: A mobile-friendly hybrid ViT for generalized plant disease image classification. arXiv preprint arXiv:2503.16628. [Google Scholar] [Crossref]

3. Rodrigo, K. U. K., Marcial, J. H. H. S., Brillo, S. C., Mata, K. E., & Morano, J. C. (2025). An enhancement of CNN algorithm for rice leaf disease image classification in mobile applications. Journal of Information Systems Engineering and Management, 10(6s). [Google Scholar] [Crossref]

4. Guo, J. (2023). Classification of crop disease images based on convolutional neural network. Advances in Computer and Communications, 4(3), 205–209. [Google Scholar] [Crossref]

5. Ergün, E. (2025). Attention-enhanced hybrid deep learning model for robust mango leaf disease classification via ConvNeXt and vision transformer fusion. Frontiers in Plant Science, 16, 1638520. [Google Scholar] [Crossref]

6. Hughes, D. P., & Salathé, M. (2015). An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv preprint arXiv:1511.08060. [Google Scholar] [Crossref]

7. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. [Google Scholar] [Crossref]

8. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, M., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16×16 words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations (ICLR). [Google Scholar] [Crossref]

9. Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022). A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11976–11986). [Google Scholar] [Crossref]

10. Mehta, S., & Rastegari, M. (2022). MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer. In Proceedings of the International Conference on Learning Representations (ICLR). [Google Scholar] [Crossref]

11. Zhao, X., Ding, W., An, Y., Du, Y., Yu, T., Li, M., Tang, M., & Wang, J. (2023). Fast segment anything. arXiv preprint arXiv:2306.12156. [Google Scholar] [Crossref]

12. Selvaraju, R. R., Cogswell, A., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020). Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128(2), 336–359. [Google Scholar] [Crossref]

13. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770–778). [Google Scholar] [Crossref]

14. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, R., Le, Q. V., & Adam, H. (2019). Searching for MobileNetV3. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 1314–1324). [Google Scholar] [Crossref]

15. Borhani, Y., Khoramdel, J., & Najafi, E. (2022). A deep learning based approach for automated plant disease classification using vision transformer. Scientific Reports, 12, 11554. [Google Scholar] [Crossref]

16. Aboelenin, S., Elbasheer, F. A., Eltoukhy, M. M., El-Hady, W. M., & Hosny, K. M. (2025). A hybrid framework for plant leaf disease detection and classification using convolutional neural networks and vision transformer. Complex & Intelligent Systems, 11, 142. [Google Scholar] [Crossref]

17. Gebremedhin, T. G., Asegede, H. S., Tesheme, B. W., Gebremichael, T. B., & Redae, K. G. (2025). Automated plant disease and pest detection system using hybrid lightweight CNN–MobileViT models for diagnosis of indigenous crops. arXiv preprint arXiv:2512.11871. [Google Scholar] [Crossref]

18. Chen, J., Hu, H., & Yang, J. (2024). Plant leaf disease recognition based on improved SinGAN and improved ResNet34. Frontiers in Artificial Intelligence, 7, 1414274. [Google Scholar] [Crossref]

19. Huang, X., Xu, D., Chen, Y., Zhang, Q., Feng, P., Ma, Y., Dong, Q., & Yu, F. (2025). EConv-ViT: A strongly generalized apple leaf disease classification model based on the fusion of ConvNeXt and transformer. Information Processing in Agriculture, 12, 466–477. [Google Scholar] [Crossref]

20. Ashurov, M., Ochilov, B., Park, S., & Kulekov, E. (2025). Enhancing plant disease detection through deep learning: A depthwise CNN with squeeze and excitation integration and residual skip connections. Frontiers in Plant Science, 15, 1505857. [Google Scholar] [Crossref]

21. Pantelaios, D., Vafeiadis, A., Iosifidis, A., & Tzovaras, D. (2025). AI-driven smart agriculture using hybrid transformer-CNN for real-time disease detection in sustainable farming. Scientific Reports, 15, 25817. [Google Scholar] [Crossref]

22. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press. [Google Scholar] [Crossref]

23. Howard, A., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. [Google Scholar] [Crossref]

24. Salman, Z., Saeed, A., Asim, M. N., Ibrahim, M. A., Dengel, W., & Ahmed, S. (2025). Plant disease classification in the wild using vision transformers and mixture of experts. Frontiers in Plant Science, 16. [Google Scholar] [Crossref]

25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30). [Google Scholar] [Crossref]

26. Ali, J. (2024). 20k+ multi-class crop disease images [Dataset]. Kaggle. [Google Scholar] [Crossref]

27. https://www.kaggle.com/datasets/jawadali1045/20k-multi-class-crop-disease-images [Google Scholar] [Crossref]

• Public Governance Quality and Tax Compliance of SMEs in North-East Nigeria
• The Influence of Conflict Resolution on Job Commitment of Non- Teaching Staff in Public Universities in Southwest, Nigeria
• Effect of Curcumin on the Viability and Proliferation of Triple-Negative Breast Cancer (MDA-MB-231) Cells
• Predicting the Effect of Job Crafting on Employee Innovative Performance: Employee Adaptability as a Mediating Variable
• Institutional Conditions Linking Transparency to Anti-Corruption Accountability in the Philippine Public Sector: A Systematic Literature Review