Original papers
An in-field automatic wheat disease diagnosis system

https://doi.org/10.1016/j.compag.2017.09.012Get rights and content

Highlights

  • An in-field automatic wheat disease diagnosis system (DMIL-WDDS) is firstly proposed.

  • DMIL-WDDS achieves identification and localization for wheat diseases.

  • DMIL-WDDS outperforms conventional CNN-based architectures on recognition accuracy.

  • A new in-field wheat disease dataset WDD2017 is collected.

  • DMIL-WDDS has been designed into a real-time mobile application.

Abstract

Crop diseases are responsible for the major production reduction and economic losses in agricultural industry worldwide. Monitoring for health status of crops is critical to control the spread of diseases and implement effective management. This paper presents an in-field automatic wheat disease diagnosis system based on a weakly supervised deep learning framework, i.e. deep multiple instance learning, which achieves an integration of identification for wheat diseases and localization for disease areas with only image-level annotation for training images in wild conditions. Furthermore, a new in-field image dataset for wheat disease, Wheat Disease Database 2017 (WDD2017), is collected to verify the effectiveness of our system. Under two different architectures, i.e. VGG-FCN-VD16 and VGG-FCN-S, our system achieves the mean recognition accuracies of 97.95% and 95.12% respectively over 5-fold cross-validation on WDD2017, exceeding the results of 93.27% and 73.00% by two conventional CNN frameworks, i.e. VGG-CNN-VD16 and VGG-CNN-S. Experimental results demonstrate that the proposed system outperforms conventional CNN architectures on recognition accuracy under the same amount of parameters, meanwhile maintaining accurate localization for corresponding disease areas. Moreover, the proposed system has been packed into a real-time mobile app to provide support for agricultural disease diagnosis.

Introduction

Crop disease diagnosis is of great significance to prevent the spread of diseases and maintain the sustainable development of agricultural economy. In general, the crop disease diagnosis is performed manually by visual observation or microscope techniques, which are proven to be time-consuming and have the risk of error due to subjective perception. In this context, various spectroscopic and imaging techniques have been studied for identifying crop disease symptoms (Bravo et al., 2004, Moshou et al., 2005, Chaerle et al., 2007, Belasque et al., 2008, Qin et al., 2009). Although these techniques can make a relatively rapid diagnosis for crop diseases, they can’t be separated from the support of expensive and bulky sensors.

With the development of computer vision, concerns are growing about the image-based detection technologies for crop disease (Camargo and Smith, 2009, Arivazhagan et al., 2013, Barbedo, 2014, Rastogi et al., 2015), which gets rid of the shackles of time cost and molecular analysis equipments (Martinelli et al., 2015). Instead, people just need common cameras and consumer-level electronic storage devices to perform crop disease identification. However, these methods are almost task-specific and need expert knowledge to design handcrafted feature extractors, only working for the crop images under ideal experimental environment. To make an automatic crop disease diagnosis system that can be applied to in-field images, one has to face some intractable challenges summarized in Barbedo (2016): (1) complex image backgrounds, e.g. leaves, soils, stones and even the people’s hands, (2) uncontrollable capture conditions, e.g. illumination, camera angle and image quality, (3) co-occurrence of multiple leaves or multiple disease areas in one image, (4) various characterization for different stages of disease development, (5) similarities in appearance between different disease categories. Some challenging samples, which are selected from our collected dataset of wheat diseases, are demonstrated in Fig. 1. To our best knowledge, few researches have been done to mitigate or eliminate the above challenges for crop disease diagnosis.

The overall objective of this work is to develop an automatic wheat disease diagnosis system to identify disease categories and locate corresponding disease areas simultaneously for in-field wheat images. In order to avoid the expensive and laborious manual annotation, the twofold task of identification and localization for wheat diseases is modeled as a weakly supervised learning task. As far as we know, we firstly propose to jointly handle the twofold task for wheat diseases in an in-field scenario.

In this paper, an novel in-field automatic wheat disease diagnosis system is proposed based on deep learning and multiple instance learning (MIL) (Dietterich et al., 1997), which can be deployed on mobile handsets to perform real-time diagnosis. As showed in Fig. 2, our framework is composed of mobile client and computing sever in device level. On the one hand, a fully convolutional network (FCN) is exploited to perform local feature extraction and local disease estimation for a resized image captured from mobile camera in wild conditions. Consequently, the FCN produces some spatial score maps over different disease species where each score point corresponds to a particular respective local window of raw image. Then these estimations of different local windows are fed into MIL framework to aggregate the overall evaluation for the whole image. On the other hand, the spatial score maps go through an up-sampling operation to roughly delineate the disease location, then a bounding boxes approximation (BBA) step is performed to accurately lock disease positions. The proposed model is named multiple instance learning based wheat disease diagnosis system (DMIL-WDDS). In particular, the DMIL-WDDS can achieve end-to-end training with enough prepared training data. To verify the practicability and effectiveness of our DMIL-WDDS, an in-field disease dataset Wheat Disease Database 2017 (WDD2017) is collected, which consists of 9,230 images with 7 different classes (6 for common wheat diseases, 1 for healthy wheat). The results of experiments on WDD2017 show that the proposed DMIL-WDDS outperforms conventional CNN architectures on recognition accuracy for disease categories, but also keeping precise localization for corresponding disease areas.

Our main contributions are summarized as:

  • A weakly supervised learning framework based on DMIL is firstly exploited for wheat disease diagnosis, which has the ability to cope with intractable in-field wheat images.

  • An integration of identification and localization for wheat diseases is achieved by proposed system, which outperforms conventional CNN-based recognition architecture under the same amount of parameters in deep model.

  • A new in-field wheat disease dataset WDD2017 is collected to demonstrate the effectiveness of proposed system as well as building a benchmark for subsequent works.

Section snippets

Related work

Many image-based methods have been developed to handle crop disease identification. Based on the fusion of hyper-spectral and multi-spectral fluorescence measurements, Moshou et al. (2005) presented a self-organizing map (SOM) based disease classifier. Camargo and Smith (2009) proposed to identify the visual symptoms of crop diseases via color transformation and image color segmentation. In Phadikar et al. (2013), Fermi energy based segmentation method was proposed to isolate infected region

Wheat Disease Database 2017 (WDD2017)

To facilitate the researches of crop disease diagnosis, some relevant datasets have been released openly and freely, e.g. PlantVillage, which includes over 50,000 expertly annotated images on healthy and infected leaves of crops. However, it is a pity that all images in PlantVillage have been processed into an ideal state which is hard to be seen in wild conditions, namely one crop leaf in a pure background. To our best knowledge, no suitable dataset for in-field crop disease diagnosis is

Experiments

We verify the effectiveness of proposed DMIL-WDDS upon WDD2017 by performing some experiments in which several models are built to estimate the test accuracy and show the localization results. Below we give details of our experiments.

Wheat disease identification

Table 3 presents the final class-wise accuracy and total accuracy over 5-fold cross-validation for different deep model configurations. To compare our DMIL-WDDS framework with conventional CNN architectures, we firstly observe the results of VGG-CNN-S and VGG-FCN-S. Note that the VGG-FCN-S with three aggregated functions (Max-agg, Avg-agg and Soft-agg) significantly and consistently outperforms VGG-CNN-S on all classes, improving the total accuracy by 17.34%, 19.86% and 22.12% respectively.

Conclusion

In this paper we propose an novel wheat disease diagnosis framework based on deep multiple instance learning, namely DMIL-WDDS, which aims to deal with in-field wheat images without any technical preprocessing. Only with the supply of image-level labelled training data, our system realizes an integration of identification for wheat diseases and localization for disease areas. We exploit four different deep models to perform wheat disease recognition on the newly collected in-field dataset

Acknowledgments

The authors would like to thank the reviewers for their valuable suggestions on improving this paper. This work is funded by NSFC (Grant No. 61473167 and No. 61621136008) and the German Research Foundation (DFG) in Project Crossmodal Learning, DFC TRR-169.

References (33)

  • M. Zhang et al.

    Automatic citrus canker detection from leaf images captured in field

    Pattern Recogn. Lett.

    (2011)
  • Al-Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A.,...
  • G. Alenyà Ribas et al.

    Robotized plant probing: leaf segmentation utilizing time-of-flight data

    IEEE Robot. Automat. Mag.

    (2013)
  • S. Arivazhagan et al.

    Detection of unhealthy region of plant leaves and classification of plant leaf diseases using texture features

    Agric. Eng. Int.: CIGR J.

    (2013)
  • J.G.A. Barbedo

    An automatic method to detect and measure leaf disease symptoms using digital image processing

    Plant Dis.

    (2014)
  • J. Belasque et al.

    Detection of mechanical and disease stresses in citrus plants by fluorescence spectroscopy

    Appl. Opt.

    (2008)
  • Cited by (0)

    View full text