Next Article in Journal
Multi-Objective Optimisation in Multi-QoS Routing Strategy for Software-Defined Satellite Network
Next Article in Special Issue
Digital Integration and Automated Assessment of Eye-Tracking and Emotional Response Data Using the BioSensory App to Maximize Packaging Label Analysis
Previous Article in Journal
Semantics Aware Dynamic SLAM Based on 3D MODT
Previous Article in Special Issue
Non-Invasive Monitoring of Ethanol and Methanol Levels in Grape-Derived Pisco Distillate by Vibrational Spectroscopy
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Computer Vision and Machine Learning Analysis of Commercial Rice Grains: A Potential Digital Approach for Consumer Perception Studies

1
Digital Agriculture, Food and Wine Group, School of Agriculture and Food, Faculty of Veterinary and Agricultural Sciences, University of Melbourne, Parkville, VIC 3010, Australia
2
Faculty of Chemical Engineering Technology, University Malaysia Perlis, Arau 02600, Perlis, Malaysia
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(19), 6354; https://doi.org/10.3390/s21196354
Submission received: 22 August 2021 / Revised: 16 September 2021 / Accepted: 22 September 2021 / Published: 23 September 2021
(This article belongs to the Special Issue Novel Contactless Sensors for Food, Beverage and Packaging Evaluation)

Abstract

:
Rice quality assessment is essential for meeting high-quality standards and consumer demands. However, challenges remain in developing cost-effective and rapid techniques to assess commercial rice grain quality traits. This paper presents the application of computer vision (CV) and machine learning (ML) to classify commercial rice samples based on dimensionless morphometric parameters and color parameters extracted using CV algorithms from digital images obtained from a smartphone camera. The artificial neural network (ANN) model was developed using nine morpho-colorimetric parameters to classify rice samples into 15 commercial rice types. Furthermore, the ANN models were deployed and evaluated on a different imaging system to simulate their practical applications under different conditions. Results showed that the best classification accuracy was obtained using the Bayesian Regularization (BR) algorithm of the ANN with ten hidden neurons at 91.6% (MSE = <0.01) and 88.5% (MSE = 0.01) for the training and testing stages, respectively, with an overall accuracy of 90.7% (Model 2). Deployment also showed high accuracy (93.9%) in the classification of the rice samples. The adoption by the industry of rapid, reliable, and accurate methods, such as those presented here, may allow the incorporation of different morpho-colorimetric traits in rice with consumer perception studies.

1. Introduction

Commercial rice (Oryza sativa) is available in various grades to meet consumer needs according to price and consumer preferences. The diverse rice germplasm consumed worldwide has high variability in its quality traits and has been linked with the physicochemical properties of the rice grains [1,2,3,4,5]. These traits are related to consumer acceptance of size and shape, color, odor/aroma, purity, homogeneity, and texture [6]. Raw rice quality is commonly associated with consumer perception, mainly before purchasing the product. It is evaluated visually based on the appearance of the rice grains, which is considered an important factor affecting buying decisions [7,8]. For example, the length, uniformity of size and shape, color, chalkiness, and percentage of broken rice were among the traits used to evaluate consumer perception of rice quality [6,7,8,9]. Meanwhile, the appearance, taste, aroma, and texture were the main quality parameters affecting consumer perception of the cooked rice [6,10,11]. A study conducted by Jeesan and Seo [12] showed that the color cues of cooked rice elicited consumer perceptions of the aroma, affected acceptance, and evoked a range of emotional responses.
Rice-quality assessment is an essential routine in rice production to maintain high-quality rice in the market and ensure high consumer acceptability. Standards for rice milling quality have been established to promote rice trading and marketing. According to the United States Standards for Milled Rice [13], rice is graded into categories considering (i) the maximum limits of the paddy kernel; (ii) the red rice; (iii) the chalky kernel; (iv) the broken kernel and other types of rice; (v) the color requirements of milled rice; and (vi) the minimum milling requirement. Besides, the Ministry of Agriculture in China established the standard for milling quality (NY/T593-2013) to improve rice grain quality production, emphasizing the standard requirements for different rice types such as indica and japonica rice [14].
Rice quality is commonly determined through visual inspections and manual measurements [1]. However, this approach is time-consuming, subjective, and prone to human error. Currently, there are analytical instruments based on imaging techniques to automate rice quality assessment, such as the Image-Rice Grain Scanner (Selgron, Blumenau, Brazil) and the SeedCount SC5000 Image Analyzer (Next Instruments, Condell Park, City of Canterbury-Bankstown, NSW, Australia). The Image-Rice Grain Scanner (Selgron, Blumenau, Brazil) provides 39 outputs of rice grain traits, including grain size, chalkiness, grain defect, and milling quality based on three-dimensional measurement of the rice grain images obtained from a camera [15]. Hence, it enables the rice breeder to select the desired rice quality traits in a shorter time with high accuracy. Furthermore, the SeedCount SC5000 Image Analyzer was developed using a flatbed scanning system in reflectance mode to obtain the individual rice sample images to measure the grain dimensions, chalkiness degree, and elongation [16]. For instance, it was used to determine the length, width, and length-to-width ratio of Australian wild rice [17] and chalkiness percentage in rice grains [18,19]. However, the lab-based instruments are costly and may hinder their application among small enterprises, especially in developing countries. Therefore, it is important to develop an alternative method using a rapid, reliable, cost-effective, and less complicated approach.
Emerging technologies such as computer vision (CV) and machine learning (ML) techniques have been applied to classify images of rice varieties [20], whole and broken rice grains [21], chalky rice [22,23], and discolored rice [24]. This technique requires the image acquisition of rice samples and computer vision algorithms to pre-process, analyze, and extract valuable information from the images to develop the classification models. Software, such as Matlab (Mathworks, Inc. Natick, MA, USA) [25,26,27] and LabVIEW (National Instruments, Austin, TX, USA) [28,29,30] and open-source libraries, such as OpenCV (Intel, Santa Clara, CA, USA) [31,32,33] and Python (Python Software Foundation, Wilmington, DE, USA) [34] are some of the most popular used among researchers. The artificial neural networks (ANNs) for supervised ML are well-known for solving multiclass classifications due to their ability to deal with non-linear data for pattern recognition to obtain high accuracy. For example, the ANN models were used in previous studies to classify mulberry fruit according to the ripeness levels [27], detect beer faults using the electronic nose [35], and classify aphid infestation levels using the electronic nose and near-infrared spectroscopy [36].
Rice from different cultivars differs in its physicochemical properties [4,37,38]. The morpho-colorimetric parameters, such as the grain’s major and minor axis length; aspect ratio; perimeter; eccentricity; roundness; red, green, and blue (RGB); and CIELab color spaces are key parameters that can be extracted using computer vision techniques. These have been used in previous studies, for example, to classify rice grains according to low-, medium-, and high-quality [39] and sound, broken, discolored, un-husked paddy, deformed, and withered grains [24]. Besides, the fractal dimension (FD) obtained by implementing the box-counting method has been used in previous research to classify grapevine leaves of different cultivars [40], characterize pork loin, and salmon sliced tissue [41], and analyze the microstructure of baked food products [42]. Hence, the FD could also be used as a critical input parameter to classify rice grains morphometrically. Nevertheless, the work on classifying rice using dimensionless parameters extracted automatically from digital images has not been much reported, especially for an extensive range of commercial rice samples. Moreover, little work has been conducted to evaluate the deployment accuracy of different conditions based on the developed model, resulting in a robust classification model.
Recent advancements in new-generation smartphones with high-resolution cameras, built-in sensors, powerful processors, and high-rate data transfer have enabled this technology adoption to be a cost-effective measurement and sensing tool. Hence, smartphone-based applications with computer vision algorithms for agriculture and food sciences have gained attention among researchers. These kinds of applications include the VitiCanopy (The University of Adelaide, Adelaide, Australia), which allows winegrowers to estimate the canopy vigor and porosity of the grapevines [43]; Plantix (PEAT GmbH, Berlin, Germany) used to diagnose pests, diseases, and nutritional deficiencies in 30 types of crops [44]; and FruitSize (Central Queensland University, Queensland, Australia), used to measure fruit size obtained from images captured from a smartphone camera [45]. Moreover, the smartphone has been used in previous studies to, for example, capture rice grain images for moisture content estimation for in-field application at harvest [46], detect milk adulteration [32], assess dietary information based on food and drink images [47], and estimate leaf area index (LAI) and plant height for canopy structures [48], which could be further developed into a smartphone app. Therefore, smartphone technology advancement has great potential to enable on-site measurement and rapid analysis at a lower cost, especially for the agricultural and food industries.
This study presents a smartphone-based imaging system as a tool to acquire images, semi-automated CV algorithms, and ML for rapid assessment techniques to classify commercial rice grains. This study used 15 commercial rice images to extract the morpho-colorimetric parameters using the customized CV algorithms written in Matlab® R2021a. The ML model based on pattern recognition of artificial neural networks (ANNs) was developed to classify the commercial rice samples using morpho-colorimetric parameters as inputs. The proposed method and the classification model were then retrained to deploy the model in a different imaging condition to simulate real-time application. The proposed method would form the foundation for a smartphone-based app as a viable alternative to the conventional approach, a mobile, cost-effective, and user-friendly tool for the rapid assessment of the rice quality traits associated with consumer perception.

2. Materials and Methods

2.1. Rice Samples

In this study, 15 commercial rice grain types were obtained from local retailers in Australia (Table 1). The samples consisted of two main categories of rice on the market: the white rice produced from whole-grain rice by a milling process to remove the outer bran layer of the grain and whole-grain (unpolished) rice. For each sample, 2 g of whole kernels were selected from each packet in triplicates, corresponding to a different total number of rice grains per type.

2.2. Image Acquisition

The images of rice samples were acquired using the rear camera (12-megapixel; focal length, f = 26 mm; aperture = f/1.8) of an iPhone 11 (Apple Inc., Cupertino, CA, USA). The Lightbox 1, a foldable lightbox tent with (Unbranded, Unhobest, China; dimensions: 40 cm × 40 cm × 40 cm), consisting of two daylight LED strips with 70 LEDs each, was used to acquire the images from the top opening at approximately 15 cm (Figure 1). The images were obtained via an Adobe Lightroom application (v6.1.0 Adobe Inc., San Jose, CA, USA) to allow a custom white balance using a white reference paper. A black background was used to capture the white and brown rice, and white background was used for the black and wild rice to provide good contrast between the foreground and background images. The rice samples were arranged in non-touching and random positions to minimize noise by touching and overlapping rice grains into the ANN modelling. However, the code includes watershed segmentation procedures for deployment to facilitate extraction of individual rice features even when touching each other or overlapping.
Figure 2 shows images obtained from each set of rice samples using the experimental setup. The images were acquired at a 3024 × 4032 pixels resolution and saved in Joint Photographic Group (JPG) format. Images captured using the smartphone were uploaded to the Matlab Drive® through Matlab Mobile for further processing using Matlab® R2021a on a personal computer (PC).

2.3. Image Pre-Processing, Segmentation and Extraction of Morpho-Colorimetric Features

The individual rice kernels were automatically analyzed to extract morpho-colorimetric features from the images using customized CV algorithms modified from previous work on leaf classification [25,40] in Matlab® R2021a. The overview of the process is presented in Figure 3; it consists of the following steps: (i) image capturing and reading; (ii) image pre-processing; (iii) image analysis; and (iv) feature extraction to retrieve the morpho-colorimetric features of the rice grains.

2.4. Morpho-Colorimetric Parameters of Commercial Rice Grains

Table 2 shows the five morphometric and four colorimetric parameters extracted from the individual rice grain image. Unitless morphometric parameters were included in the study to simplify the image-capturing step and allow the user to independently capture the rice images in the lightbox at any object distance. The identification of single rice seeds was based on blob analysis from binarized images, which identifies contours of blobs to automatically label each grain to extract statistical shape measurements using the regionprops function in Matlab® R2021a. From that analysis, automatic statistics are calculated, such as centroid, the aspect ratio and the area-parameter ratio index, computed using the major and minor axis length, area (A), and perimeter (P) of the rice images. The fractal dimension (FD) of the rice grains was determined using the box-counting method based on previously published work to recognize and analyze grapevine leaves [40] and medicinal plant leaves to extract similar features [25]. The binarized image of rice grains was also used as a mask on the original image to extract automatically colorimetric parameters from each rice grain of CIELab and RGB color scales. The yellowness index was computed from color scale parameters to determine the yellowness degree of the rice grains based on the previous work conducted by Rhim et al. [49]. The extracted features from the individual rice grains were automatically saved in Microsoft Excel Binary File Format (.xls) and were used as inputs for ML model development to classify the 15 commercial rice grains.

2.5. Statistical Analysis

A one-way analysis of variance (ANOVA; p < 0.05) and Tukey’s Honestly Significant Difference (HSD) post hoc test (α = 0.05) were conducted using Minitab 19.1 (Minitab Inc., State College, PA, USA). It was performed to assess whether there were significant differences between the means of the morpho-colorimetric parameters of the commercial rice samples obtained using Lightbox 1.
Multivariate data analysis based on the principal component analysis (PCA) and cluster analysis was conducted using a customized code written in Matlab® R2021a. The main use of the PCA in this study was to find relationships between variables and samples as they are constructed using covariance methods as a parameter engineering justification for the ANN modelling presented [51,52,53,54,55]. Besides, cluster analysis helps to visualize the relative grouping of commercial rice samples according to these parameters. This type of analysis to support parameter engineering has been used in several ANN works for food and beverage applications [56,57,58] and helps non-experts in AI or machine-learning understand better the relationships between different parameters from the physicochemical point of view. This type of multivariate data analysis also helps to clarify the “black-box” properties of supervised machine learning such as ANN and to visualize that ANN correctly estimates the targets and that they are not artifacts from non-related inputs.

2.6. Machine Learning Modeling

The classification ML models were developed using the ANN algorithm for pattern recognition using a customized code written in Matlab® R2021a. The model was developed by testing 17 training algorithms (data not shown), which consisted of three types of main functions: (i) backpropagation with Jacobian derivatives; (ii) backpropagation with gradient derivatives; and (iii) supervised weight and bias training functions [59]. The optimum classification algorithm was then selected by assessing the accuracy and performance of the model, indicated based on mean squared error (MSE); to assess any signs of under- or over-fitting, the MSE value for the training stage must be lower than the value for the testing stage. Furthermore, the number of inputs must be <70% of the number of samples to avoid over-fitting, which this model meets with only nine inputs [60,61,62]. Furthermore, the receiver operating characteristic (ROC) curves were used to analyze the model’s sensitivity (true positive rate) and specificity (true negative rate) to classify each type of rice [63]. A neuron trimming exercise was conducted using ten, seven, five, and three hidden neurons to find the optimal neuron number, followed by retraining the model several times to assess the consistency of the results and find the best model based on accuracy and performance. The number of neurons must also be considered to assess under- or over-fitting; a larger number of neurons, usually >10, is more likely to lead to over-fitting, while a very low number, usually below three, may lead to under-fitting [56].
In this study, Model 1 was developed using a data set extracted from the rice images obtained from Lightbox 1. After the screening, the Bayesian Regularization (BR) with seven neurons was selected because the model presented high accuracy, best performance, and no under- or over-fitting signs. The model thus consisted of a two-layer feedforward neural network with a sigmoid function using nine morpho-colorimetric parameters as inputs to classify the grains according to 15 types of commercial rice (Figure 4). The data set from the population of the rice grain samples for training and testing was randomly divided into 70% (n = 2687) and 30% (n = 1152), respectively.

2.7. Retraining and Deployment of the Machine Learning Model

A test was conducted to evaluate the deployment of the proposed method to simulate the practical application in a different condition. Therefore, a new data set corresponding to the 15 rice samples was acquired using Lightbox 2 (2D PhotoBench 120, Ortery Technologies Inc., Irvine, CA, USA; Dimension = 61.0 cm × 61.0 cm × 71.1 cm) with 5700 K daylight LED lighting, using similar approaches to those described in Section 2.2 with slight modification. The same smartphone was attached on a mini tripod at 15 cm from the samples. The images were acquired using Adobe Lightroom to set a custom white balance using white reference paper. Data from eight replicate images containing around 50 rice grains in an image were acquired and extracted from the customized CV codes described in Section 2.3.
The new data from Lightbox 2 were fed to retrain the original Model 1 using the steps mentioned earlier in Section 2.6; the retrained Model 1 was then named Model 2. It was developed based on the nine morpho-colorimetric parameters as inputs to classify 15 commercial rice grains as targets, similar to the original Model 1 using a random data division of 70% (n = 6887) for training and 30% (n = 2952) for testing data sets (Figure 5). After testing the model on the 17 ANN algorithms, followed by the neuron trimming test, the best model was obtained using the BR algorithms with a sigmoid activation function and ten hidden neurons.
Deployment accuracy was tested using a new image captured for each type of rice obtained from Lightbox 2 using the same procedure as those used to develop Model 2. The image acquired using the smartphone was sent to Matlab Drive®, followed by the following steps to process the retrieved image in a laptop computer to detect individual rice grains, extract morpho-colorimetric features, and classify them according to their corresponding class ID using the developed ML model embedded in the code. Finally, a decision image was displayed with the labeled predicted class ID for each rice grain in the image. Figure 6 shows the flow diagram of the rice classification process.

3. Results

3.1. Morpho-Colorimetric Parameters of Commercial Rice Grains

Table 3 shows the ANOVA results of the morpho-colorimetric parameters from the dataset obtained using the foldable lightbox tent for each type of rice. Significant differences (p < 0.05) were observed between the samples in all parameters. The mean values for the FD obtained ranged from 1.57 (LGW) to 1.83 (BMB). The FD values were higher among the short-grain samples (the KHO, the SRS, the BMB, and the CLP), while medium-grain samples mainly had intermediate FD values.
The WRO had the lowest mean for Cir (0.46), while the short-grain rice samples, such as the CLP (0.91) and the BMB (0.91), were among the samples with high mean values. The AR is the ratio between the major and minor axis length and showed low mean values among the short-grain rice samples (1.63–1.97) compared to the medium- and long-grain rice samples (2.06–5.19). The short-grain rice samples such as the CLP (0.74), the BMB (0.73), the KHO (0.72), and the SRS (0.72) were the rice samples with high mean values for Ext, and the WRO (0.45) was the rice sample with the lowest Ext. The APIdx calculated using the pixel area, and the perimeter ratio showed that the BAS (0.27) had the lowest mean value among rice samples. Among all the samples, the WRO had a high AR value and low values for Cir, Ext, and APIdx, showing that its size and shape were different from the other types of rice, which are very long and narrow, reflecting its characteristics as a long and slender-shaped grain.
The L mean value was higher for the white rice samples, such as the JAS (59.46), the LGW (59.38), and the BAS (59.16), and lower L values were obtained for highly pigmented rice, such as the BKR and the WRO (27.54 and 29.60, respectively). As opposed to the L, the YI for the WRO and the BKR was higher than the white rice samples. The inverse trends for the L and YI parameters described the rice grain’s lightness and yellowness, respectively. A low positive value was observed in mean values of a for all white rice samples, and higher mean values were obtained for both highly pigmented rice samples. The unpolished rice samples had a higher b value compared to the white rice samples.

3.2. Multivariate Data Analysis

Figure 7a shows the PCA biplot for the nine morpho-colorimetric parameters and the 15 commercial rice samples from the data set obtained from the foldable light box tent. The PCA explained the 82.7% total data variability (PC1: 59.8%; PC2: 22.90%). Based on factor loadings (FL), principal component one (PC1) was characterized by AR (FL = 0.40) and YI (FL = 0.34) on the positive side of the axis, whereas Ext (FL = −0.39), Cir (FL = −0.39), and L (FL =−0.34) represented PC1 on the negative side. The principal component two (PC2) was mainly represented by b (FL = 0.43) and Y1 (FL = 0.41) on the positive side, and L (FL = −0.35) on the negative side.
The colorimetric parameters, such as YI, b, and a, were positively related and associated with the BKR and, to a lesser extent, the brown rice samples, such as the MGB, the DGR, and the BDM, which are located in the center of the PCA. In contrast, the L was negatively related to the latter parameters and associated with the MOB (brown rice). The morphometric parameters such as Ext, Cir, FD, and APIdx had a positive relationship and were associated with most rice samples belonging to the short and medium grains. Parameter AR was negatively related to the latter parameters and associated with the long-grain rice (JAS, LGW, and BAS).
Figure 7b shows the cluster analysis using the Euclidean linkage of PCA based on the nine morpho-colorimetric parameters to group the rice. The unpolished rice, the WRO, and the BKR were found in the same group, while the rest of the rice samples were clustered in a group. Likewise, Figure 7a showed groups of rice samples identified for black rice (BKR) and wild rice (WRO), brown rice (DGR, MGB, and BDM), and long-grain (JAS, LGW, and BAS) rice. However, there was no clear distinction between short-grain (BMB, CLP, KHO, and SRS) and medium-grain (ARB and CLS) rice.

3.3. Machine Learning Modelling

Table 4 shows the statistical results of classifying the commercial rice grains based on nine morpho-colorimetric parameters using the Bayesian Regularization algorithm. Both classification models had high overall accuracies (>90%), with a lower MSE value for training (MSE < 0.01) than testing (MSE = 0.01) stages. Moreover, comparable accuracy was obtained for the training and testing stage for both models. This showed that the models had no signs of under- or over-fitting.
Figure 8 shows the receiver operating characteristic (ROC) curve of the true-positive (sensitivity) versus the false-positive rates for both ANN models for classifying commercial rice grains. Based on the plot, the curves are closer to the true-positive rate at the y-axis, showing that the classification model had high true-positive rates (sensitivity) for classifying the rice samples correctly. It also showed that the models had a high predictive power to classify into each type of rice.

3.4. Evaluation of Commercial Rice Grains Classification Using ML

The deployment accuracies of the developed models for classifying new data sets of rice samples are shown in Table 5. The model’s performance in classifying the rice samples (n = 50) showed a high mean accuracy of 93.9%. The results also showed that the model successfully classified all the rice samples according to their respective rice class with ≥82%, which is acceptable for the application.

4. Discussion

4.1. Morpho-Colorimetric Features

The developed codes to extract nine morpho-colorimetric parameters were modified from previous works on leaf classifications for grapevine cultivars [40] and Chinese medicinal plants [25]. The novelty in this work was the incorporation of dimensionless morphological parameters, so the distance of the camera from the rice grains can vary with different setups. For fractal analysis parameters, the FD was formerly used by Jinorose et al. [64] to examine the effect of the parboiling process and cooking time on the physical changes of cooked rice grains based on image analysis. The morphometric parameters could discriminate short-grain rice groups indicated by high mean values for FD, Cir, Ext, APIdx, and low AR. Moreover, Cir and AR showed their potential as key parameters for ML modeling in classifying the ARB and the CLS under a similar grouping. This is in accordance with Calingacion et al. [37], who suggested that these types of rice were categorized as medium-length with bold-shape grains.
The colorimetric parameters extracted from the rice showed that white rice and unpolished rice vary because of the pigmentation on the bran layer of unpolished grain. Besides, the ARB, the CLP, and the BMB rice samples used for risotto and paella have chalky kernels that are more opaque than the translucent rice cultivars such as the LGW, JOM, and JSR [65]. High pigmentation on the BKR and the WRO rice samples discriminated by high a (redness) value were in accordance with previous research, which suggested that the pigmentation for those types of rice was related to the reddish color on the rice grains [17,66]. The high pigmentation corresponds to the carotenoid and anthocyanin content, which is linked with significant health benefits [67]. It is known that different colors of rice grains may depend on several factors, such as the varieties, the milling degree, the aging, and the parboiling process. A previous study showed that the relationship between the chalkiness and the physicochemical properties in rice was reflected by high total-starch accumulation, low total protein, and amino acids in the chalky part [68]. Besides, the milling process to polish the brown rice by removing the bran layer for white rice production could also affect the color variation in rice grains because different types of rice may be polished at different milling degrees during the process [21,69]. Therefore, the significant differences in grain pigmentation were considered relevant parameters to develop the classification model when using supervised ML in the parameter engineering process of supervised ML modeling.

4.2. Multivariate Data Analysis

Based on PCA and cluster analysis, it was found that most short- and medium-grain rice samples were grouped, as shown by the high similarities among the rice samples, which could be explained by their positive associations with FD, Cir, and Ext and negative associations with AR. Commonly, the AR is used to categorize the rice into three shape classes: bold (<2), medium (2.1–3), and slender (>3) [70]. This could be explained by previous research conducted by Calingacion et al. [37], in which both short- and medium-grain rice may have bold- and medium-shaped rice. As the PCA and cluster analysis showed an unclear separation among the rice samples, it was important to explore the potential of machine learning modeling to classify the rice.

4.3. Machine Learning Modeling

For ML models developed in this study, comparable MSE values were obtained for the training and testing stage, implying that the developed model showed no signs of over-fitting [71]. A similar finding from previous work was reported based on a comparative empirical study between BR and LM algorithms to develop the ANN model for social data prediction. The BR showed better performance than the LM algorithm for data prediction and supported its performance in dealing with high complexity data [62].
A previous study reported the classification of three types of commercial Basmati rice images using a k-Nearest Neighbor (k-NN) classifier based on morphometric parameters, such as the area, major axis, minor axis, eccentricity, and perimeter [72]. However, the overall classification accuracy was 79%, which could lead to poor estimation during deployment. Moreover, using specific dimensions from rice may jeopardize the accuracy of models using different settings, especially camera distance from the objects of interest. In contrast, Anami et al. [73] compared classification models developed using the Multilayer Back Propagation Neural Network (BPNN), the Support Vector Machine (SVM), and k-NN to classify five levels of adulteration in bulk paddy grain. The latter model was developed using the combination of color and texture parameters extracted from the images. The BPNN model was identified as the best model to classify the rice adulteration level at an average of 93.31%. However, it required a high number of input data (40 principal component coefficients) compared to the present study, which used only nine easily derived morpho-colorimetric inputs to obtain a comparable accuracy. Therefore, this study demonstrates the importance of selecting the appropriate inputs, classifiers, training algorithms, and hyperparameters to optimize the classification accuracy, which may help avoid under- and over-fitting and mitigate the high computational requirements.
The default data division of 70% for training and 30% for testing was used in the study as they represented a sufficient number of samples in each category. This method uses independent sets of samples for each stage and evaluates the overall accuracy by including all samples. A similar data division to develop ANN models was used in previous studies [35,59,74]. Besides, several retraining attempts were conducted to assess the consistency of the results, obtaining similar results in every attempt. Furthermore, the deployment using new data further validates the accuracy and performance of the model.

4.4. Deployment of the Classification Machine Learning Model

One of the challenges of supporting practical application in a different controlled environment is the high sensitivity to lighting conditions and camera settings, contributing to high misclassification when testing on a new data set obtained under different conditions. Therefore, Model 2 was developed using the new data sets obtained from Lightbox 2 to evaluate the deployment accuracy of the initial Model 1. The high deployment accuracy showed that the developed model based on adimensional morphological parameters is robust and reliable. Unlike the existing classification models, the ANN model developed in this study included a comprehensive and complex range of commercial rice samples as targets, including white, brown, black, and wild rice. Therefore, the model appears to exhibit robustness in classifying the different commercial rice grains available globally in the market.
The advantage of using dimensionless parameters is that the user is allowed to capture images without strict settings. The demonstrated method is independent of the type of camera and the camera’s distance from the object. Moreover, the classification model was developed using two lightbox systems, increasing model generalizability and adaptability to new data.
The work in this study is highly significant to the rice industry as the extracted morpho-colorimetric parameters were associated with consumer perceptions of raw [6,7,8,9] and cooked rice [6,10,11] quality. Therefore, these parameters could predict consumer perceptions of rice associated with the appearance quality traits for rice types. Furthermore, the automatic extraction of features and ANN modelling will help the industry to certify rice types and prevent adulteration [75]; furthermore, the ANN modelling based on feature extraction as inputs could also be used to target consumer perception and quality parameters as it has been performed for other food and beverage products [57,59,76].
This grain-by-grain scale approach proposed for quality assurance may avoid manual analysis and destructive assessment and save time compared to traditional descriptive sensory analysis with trained panelists. The effort of utilizing a smartphone camera to capture the images paired with a semi-automated CV algorithm could accelerate this development, making it cost-effective, user-friendly, rapid, and convenient. Moreover, since the smartphone is portable, it can also support on-site assessment instead of using specialized equipment in the laboratory.

5. Conclusions

This study showed the development of a cost-effective and rapid method to classify commercial rice samples obtained from a smartphone camera. It was achieved by integrating CV algorithms to extract morpho-colorimetric parameters and ML to classify 15 types of commercial rice grains. High classification accuracies were obtained based on ML models developed using the dimensionless parameters as inputs captured from different lightboxes, which increases model generalization. Further studies are required to link these easily obtained parameters with other quality traits and compositional parameters of rice grains that are important for the industry. Moreover, the methodologies proposed in this study can be applied by the industry to develop a smartphone application integrated with cloud-based computing to automatically assess consumer perception associated with rice quality traits and in real-time. The latter is achieved by acquiring consumer sensory perceptions through images of rice and cooked rice. This can benefit the industry in monitoring rice quality conveniently along the rice supply chains as well as at the consumer end.

Author Contributions

Conceptualization, A.A., C.G.V. and S.F.; methodology, A.A., A.P., C.G.V. and S.F.; software, A.A., C.G.V. and S.F.; validation, C.G.V., A.P. and S.F.; formal analysis, A.A.; investigation, A.A., C.G.V. and S.F.; resources, C.G.V., A.P. and S.F.; data curation, A.A., C.G.V. and S.F.; writing—original draft preparation, A.A.; writing—review and editing, C.G.V., A.P. and S.F.; visualization, A.A., C.G.V., A.P. and S.F.; supervision, S.F. and A.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data and intellectual property belong to The University of Melbourne; any sharing needs to be evaluated and approved by the University.

Acknowledgments

The authors would like to acknowledge the support from the Digital Agriculture, Food, and Wine Group from the Faculty of Veterinary and Agricultural Sciences (FVAS), The University of Melbourne.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Butardo, V.M.; Sreenivasulu, N. Improving head rice yield and milling quality: State-of-the-art and future prospects. In Rice Grain Quality: Methods and Protocols; Sreenivasulu, N., Ed.; Springer: New York, NY, USA, 2019. [Google Scholar] [CrossRef]
  2. Li, H.; Fitzgerald, M.A.; Prakash, S.; Nicholson, T.M.; Gilbert, R.G. The Molecular Structural Features Controlling Stickiness in Cooked Rice, a Major Palatability Determinant. Sci. Rep. 2017, 7, 43713. [Google Scholar] [CrossRef] [Green Version]
  3. Yanjie, X.; Yining, Y.; Shuhong, O.; Xiaoliang, D.; Hui, S.; Shukun, J.; Shichen, S.; Jinsong, B. Factors Affecting Sensory Quality of Cooked Japonica Rice. Rice Sci. 2018, 25, 330–339. [Google Scholar] [CrossRef]
  4. Graham-Acquaah, S.; Mauromoustakos, A.; Cuevas, R.P.; Manful, J.T. Differences in Physicochemical Properties of Commercial Rice from Urban Markets in West Africa. J. Food Sci. Technol. 2020, 57, 1505–1516. [Google Scholar] [CrossRef] [PubMed]
  5. Zhu, L.; Bi, S.; Wu, G.; Zhang, H.; Wang, L.; Qian, H.; Qi, X.; Jiang, H. Comparative Analysis of the Texture and Physicochemical Properties of Cooked Rice Based on Adjustable Rice Cooker. LWT 2020, 130, 109650. [Google Scholar] [CrossRef]
  6. Custodio, M.C.; Cuevas, R.P.; Ynion, J.; Laborte, A.G.; Velasco, M.L.; Demont, M. Rice Quality: How Is It Defined by Consumers, Industry, Food Scientists, and Geneticists? Trends Food Sci. Technol. 2019, 92, 122–137. [Google Scholar] [CrossRef] [PubMed]
  7. Keith, T.; John, M.; Joseph, G.; Bernice, K.; Tamakloe, I. Study of Sensory Evaluation, Consumer Acceptability, Affordability and Market Price of Rice. J. Sci. Food Agric. 2007, 87, 1564–1575. [Google Scholar] [CrossRef]
  8. Cuevas, R.P.; Pede, V.O.; McKinley, J.; Velarde, O.; Demont, M. Rice Grain Quality and Consumer Preferences: A Case Study of Two Rural Towns in the Philippines. PLoS ONE 2016, 11, e0150345. [Google Scholar] [CrossRef] [Green Version]
  9. Mottaleb, K.A.; Rahut, D.B.; Mishra, A.K. Modeling Rice Grain-Type Preferences in Bangladesh. Br. Food J. 2017, 119, 2049–2061. [Google Scholar] [CrossRef]
  10. Champagne, E.T.; Bett-Garber, K.L.; Fitzgerald, M.A.; Grimm, C.C.; Lea, J.; Ohtsubo, K.; Jongdee, S.; Xie, L.; Bassinello, P.Z.; Resurreccion, A.; et al. Important Sensory Properties Differentiating Premium Rice Varieties. Rice 2010, 3, 270–281. [Google Scholar] [CrossRef] [Green Version]
  11. Meullenet, J.F.; Griffin, V.K.; Carson, K.; Davis, G.; Davis, S.; Gross, J.; Hankins, J.A.; Sailer, E.; Sitakalin, C.; Suwansri, S.; et al. Rice External Preference Mapping for Asian Consumers Living in The United States. J. Sens. Stud. 2001, 16, 73–94. [Google Scholar] [CrossRef]
  12. Jeesan, S.A.; Seo, H.-S. Color-Induced Aroma Illusion: Color Cues Can Modulate Consumer Perception, Acceptance, and Emotional Responses toward Cooked Rice. Foods 2020, 9, 1845. [Google Scholar] [CrossRef]
  13. USDA. United States Standards for Rice; Revised; Federal Grain Inspection Service: Washington, DC, USA, 2009.
  14. Bao, J. Rice milling quality. In Rice; Elsevier: Amsterdam, The Netherlands, 2019; pp. 339–369. [Google Scholar]
  15. Marschalek, R.; Silva, M.C.; dos Santos, S.B.; Manke, J.R.; Bieging, C.; Porto, G.; Wickert, E.; de Andrade, A. Image—Rice Grain Scanner: A Three-Dimensional Fully Automated Assessment of Grain Size and Quality Traits. Crop Breed. Appl. Biotechnol. 2017, 17, 89–97. [Google Scholar] [CrossRef] [Green Version]
  16. Santos, M.V.; Cuevas, R.P.O.; Sreenivasulu, N.; Molina, L. Measurement of rice grain dimensions and chalkiness, and rice grain elongation using image analysis. In Rice Grain Quality; Springer: Berlin/Heidelberg, Germany, 2019; pp. 99–108. [Google Scholar]
  17. Tikapunya, T.; Fox, G.; Furtado, A.; Henry, R. Grain Physical Characteristic of the Australian Wild Rices. Plant Genet. Resour. Charact. Util. 2017, 15, 409–420. [Google Scholar] [CrossRef]
  18. Sreenivasulu, N.; Misra, G.; Bandyopadhyay, A.; Badoni, S. Project Breeding for Low Chalk in Rice; ACIAR: Canberra, Australia, 2019. [Google Scholar]
  19. Misra, G.; Badoni, S.; Parween, S.; Singh, R.K.; Leung, H.; Ladejobi, O.; Mott, R.; Sreenivasulu, N. Genome-wide Association Coupled Gene to Gene Interaction Studies Unveil Novel Epistatic Targets among Major Effect Loci Impacting Rice Grain Chalkiness. Plant Biotechnol. J. 2020, 19, 910–925. [Google Scholar] [CrossRef]
  20. Senthil Kumar, M.; Javeed, M. An Efficient Rice Variety Identification Scheme Using Shape, Harlick & Color Feature Extraction and Multiclass SVM. Int. J. Eng. Adv. Technol. 2019, 8, 3629–3632. [Google Scholar] [CrossRef]
  21. Zareiforoush, H.; Minaei, S.; Alizadeh, M.R.; Banakar, A. Qualitative Classification of Milled Rice Grains Using Computer Vision and Metaheuristic Techniques. J. Food Sci. Technol. 2016, 53, 118–131. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Sun, C.; Liu, T.; Ji, C.; Jiang, M.; Tian, T.; Guo, D.; Wang, L.; Chen, Y.; Liang, X. Evaluation and Analysis the Chalkiness of Connected Rice Kernels Based on Image Processing Technology and Support Vector Machine. J. Cereal Sci. 2014, 60, 426–432. [Google Scholar] [CrossRef]
  23. Parveen, Z.; Alam, M.A.; Shakir, H. Assessment of quality of rice grain using optical and image processing technique. In 2017 International Conference on Communication, Computing and Digital Systems (C CODE); IEEE: Manhattan, NY, USA, 2017; pp. 265–270. [Google Scholar] [CrossRef]
  24. Itsarawisut, J.; Kanjanawanishkul, K. Neural Network-Based Classification of Germinated Hang Rice Using Image Processing. IETE Tech. Rev. 2019, 36, 375–381. [Google Scholar] [CrossRef]
  25. Xue, J.R.; Fuentes, S.; Poblete-Echeverria, C.; Viejo, C.G.; Tongson, E.; Du, H.J.; Su, B. Automated Chinese Medicinal Plants Classification Based on Machine Learning Using Leaf Morpho-Colorimetry, Fractal Dimension and Visible/near Infrared Spectroscopy. Int. J. Agric. Biol. Eng. 2019, 12, 123–131. [Google Scholar] [CrossRef]
  26. De-la-Torre, M.; Zatarain, O.; Avila-George, H.; Muñoz, M.; Oblitas, J.; Lozada, R.; Mejía, J.; Castro, W. Multivariate Analysis and Machine Learning for Ripeness Classification of Cape Gooseberry Fruits. Processes 2019, 7, 928. [Google Scholar] [CrossRef] [Green Version]
  27. Azarmdel, H.; Jahanbakhshi, A.; Mohtasebi, S.S.; Muñoz, A.R. Evaluation of Image Processing Technique as an Expert System in Mulberry Fruit Grading Based on Ripeness Level Using Artificial Neural Networks (ANNs) and Support Vector Machine (SVM). Postharvest Biol. Technol. 2020, 166, 111201. [Google Scholar] [CrossRef]
  28. Duan, L.; Han, J.; Guo, Z.; Tu, H.; Yang, P.; Zhang, D.; Fan, Y.; Chen, G.; Xiong, L.; Dai, M.; et al. Novel Digital Features Discriminate between Drought Resistant and Drought Sensitive Rice under Controlled and Field Conditions. Front. Plant Sci. 2018, 9, 492. [Google Scholar] [CrossRef] [Green Version]
  29. Ruslan, R.; Khairunniza-Bejo, S.; Rukunudin, I.H.; Jahari, M.; Ibrahim, M. Development of a Machine Vision System for Rice Seed Inspection System. Food Res. 2020, 4, 150–156. [Google Scholar] [CrossRef]
  30. Sung, H.-J.; Park, M.-K.; Choi, J.W. Automatic Grader for Flatfishes Using Machine Vision. Int. J. Control. Autom. Syst. 2020, 18, 3073–3082. [Google Scholar] [CrossRef]
  31. Kaisaat, K.; Keawdonree, N.; Chomkokard, S.; Jinuntuya, N.; Pattanasiri, B. Colour Measurements of Pigmented Rice Grain Using Flatbed Scanning and Image Analysis. J. Phys. Conf. Ser. 2017, 901, 2–5. [Google Scholar] [CrossRef]
  32. Wu, W.; Liu, T.; Zhou, P.; Yang, T.; Li, C.; Zhong, X.; Sun, C.; Liu, S.; Guo, W. Image Analysis-Based Recognition and Quantification of Grain Number per Panicle in Rice. Plant Methods 2019, 15, 1–14. [Google Scholar] [CrossRef] [PubMed]
  33. Komyshev, E.; Genaev, M.; Afonnikov, D. Evaluation of the Seed Counter, a Mobile Application for Grain Phenotyping. Front. Plant Sci. 2017, 7, 1–9. [Google Scholar] [CrossRef] [Green Version]
  34. Kapur, S. Computer Vision with Python 3; Packt Publishing Ltd.: Birmingham, UK, 2017. [Google Scholar]
  35. Viejo, C.G.; Fuentes, S.; Hernandez-Brenes, C. Smart Detection of Faults in Beers Using Near-Infrared Spectroscopy, a Low-Cost Electronic Nose and Artificial Intelligence. Fermentation 2021, 7, 117. [Google Scholar] [CrossRef]
  36. Fuentes, S.; Tongson, E.; Unnithan, R.R.; Viejo, C.G. Early Detection of Aphid Infestation and Insect-Plant Interaction Assessment in Wheat Using a Low-Cost Electronic Learning Modeling. Sensors 2021, 21, 5948. [Google Scholar] [CrossRef]
  37. Calingacion, M.; Laborte, A.; Nelson, A.; Resurreccion, A.; Concepcion, J.C.; Daygon, V.D.; Mumm, R.; Reinke, R.; Dipti, S.; Bassinello, P.Z.; et al. Diversity of Global Rice Markets and the Science Required for Consumer-Targeted Rice Breeding. PLoS ONE 2014, 9, e85106. [Google Scholar] [CrossRef] [Green Version]
  38. Gunaratne, A.; Wu, K.; Kong, X.; Gan, R.Y.; Sui, Z.; Kumara, K.; Ratnayake, U.K.; Senarathne, K.; Kasapis, S.; Corke, H. Physicochemical Properties, Digestibility and Expected Glycaemic Index of High Amylose Rice Differing in Length-Width Ratio in Sri Lanka. Int. J. Food Sci. Technol. 2020, 55, 74–81. [Google Scholar] [CrossRef]
  39. Mittal, S.; Dutta, M.K.; Issac, A. Non-Destructive Image Processing Based System for Assessment of Rice Quality and Defects for Classification According to Inferred Commercial Value. Measurement 2019, 148, 106969. [Google Scholar] [CrossRef]
  40. Fuentes, S.; Hernández-Montes, E.; Escalona, J.M.; Bota, J.; Gonzalez Viejo, C.; Poblete-Echeverría, C.; Tongson, E.; Medrano, H. Automated Grapevine Cultivar Classification Based on Machine Learning Using Leaf Morpho-Colorimetry, Fractal Dimension and near-Infrared Spectroscopy Parameters. Comput. Electron. Agric. 2018, 151, 311–318. [Google Scholar] [CrossRef]
  41. Verdú, S.; Barat, J.M.; Grau, R. Fresh-Sliced Tissue Inspection: Characterization of Pork and Salmon Composition Based on Fractal Analytics. Food Bioprod. Process. 2019, 116, 20–29. [Google Scholar] [CrossRef]
  42. Rahimi, J.; Baur, J.; Singh, A. Digital Imaging as a Tool to Study the Structure of Porous Baked Foods. J. Cereal Sci. 2020, 95, 103084. [Google Scholar] [CrossRef]
  43. De Bei, R.; Fuentes, S.; Gilliham, M.; Tyerman, S.; Edwards, E.; Bianchini, N.; Smith, J.; Collins, C. Viticanopy: A Free Computer App to Estimate Canopy Vigor and Porosity for Grapevine. Sensors 2016, 16, 585. [Google Scholar] [CrossRef] [Green Version]
  44. Plantix. The Smart Crop Assistant on Your Smartphone. Available online: https://plantix.net/en/ (accessed on 6 March 2021).
  45. Wang, Z.; Koirala, A.; Walsh, K.; Anderson, N.; Verma, B. In Field Fruit Sizing Using a Smart Phone Application. Sensors 2018, 18, 3331. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Yang, M.; Hsu, Y.; Tseng, W.; Lu, C.; Yang, C.; Lai, M.; Wu, D. Assessment of Grain Harvest Moisture Content Using Machine Learning on Smartphone Images for Optimal Harvest Timing. Sensors 2021, 21, 5875. [Google Scholar] [CrossRef]
  47. Mezgec, S.; Koroušić Seljak, B. NutriNet: A Deep Learning Food and Drink Image Recognition System for Dietary Assessment. Nutrients 2017, 9, 657. [Google Scholar] [CrossRef] [PubMed]
  48. Yu, Z.; Ustin, S.L.; Zhang, Z.; Liu, H.; Zhang, X.; Meng, X.; Cui, Y.; Guan, H. Estimation of a New Canopy Structure Parameter for Rice Using Smartphone Photography. Sensors 2020, 20, 4011. [Google Scholar] [CrossRef] [PubMed]
  49. Rhim, J.W.; Wu, Y.; Weller, C.L.; Schnepf, M. Physical Characteristics of a Composite Film of Soy Protein Isolate and Propyleneglycol Alginate. J. Food Sci. 1999, 64, 149–152. [Google Scholar] [CrossRef]
  50. Lurstwut, B.; Pornpanomchai, C. Image Analysis Based on Color, Shape and Texture for Rice Seed (Oryza sativa L.) Germination Evaluation. Agric. Nat. Resour. 2017, 51, 383–389. [Google Scholar] [CrossRef]
  51. Richardson, M. Principal Component Analysis. Available online: http//people.maths.ox.ac.uk/richardsonm/SignalProcPCA.pdf (accessed on 3 September 2021).
  52. Abdi, H.; Williams, L.J. Principal Component Analysis. Wiley Interdiscip. Rev. Comput. Stat. 2010, 2, 433–459. [Google Scholar] [CrossRef]
  53. Wold, S.; Esbensen, K.; Geladi, P. Principal Component Analysis. Chemom. Intell. Lab. Syst. 1987, 2, 37–52. [Google Scholar] [CrossRef]
  54. Alkarkhi, A.F.M.; Alqaraghuli, W.A.A. Principal components analysis. In Easy Statistics for Food Science with R; Academic Press: Oxford, UK, 2019; pp. 125–141. [Google Scholar]
  55. Syms, C. Principal components analysis. In Encyclopedia of Ecology; Jørgensen, S.E., Fath, B.D., Eds.; Academic Press: Oxford, UK, 2008; pp. 2040–2949. [Google Scholar]
  56. Gonzalez Viejo, C.; Torrico, D.D.; Dunshea, F.R.; Fuentes, S. Emerging Technologies Based on Artificial Intelligence to Assess the Quality and Consumer Preference of Beverages. Beverages 2019, 5, 62. [Google Scholar] [CrossRef] [Green Version]
  57. Gonzalez Viejo, C.; Fuentes, S.; Li, G.J.; Collmann, R.; Condé, B.; Torrico, D. Development of a Robotic Pourer Constructed with Ubiquitous Materials, Open Hardware and Sensors to Assess Beer Foam Quality Using Computer Vision and Pattern Recognition Algorithms: RoboBEER. Food Res. Int. 2016, 89, 504–513. [Google Scholar] [CrossRef]
  58. Gonzalez Viejo, C.; Fuentes, S.; Torrico, D.D.; Howell, K.; Dunshea, F.R. Assessment of Beer Quality Based on a Robotic Pourer, Computer Vision, and Machine Learning Algorithms Using Commercial Beers. J. Food Sci. 2018, 83, 1381–1388. [Google Scholar] [CrossRef] [PubMed]
  59. Gonzalez Viejo, C.; Torrico, D.D.; Dunshea, F.R.; Fuentes, S. Development of Artificial Neural Network Models to Assess Beer Acceptability Based on Sensory Properties Using a Robotic Pourer: A Comparative Model Approach to Achieve an Artificial Intelligence System. Beverages 2019, 5, 33. [Google Scholar] [CrossRef] [Green Version]
  60. Beale, M.H.; Hagan, M.T.; Demuth, H.B. Deep Learning Toolbox User’s Guide; Mathworks Inc.: Natick, MA, USA, 2020. [Google Scholar]
  61. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
  62. Kayri, M. Predictive Abilities of Bayesian Regularization and Levenberg-Marquardt Algorithms in Artificial Neural Networks: A Comparative Empirical Study on Social Data. Math. Comput. Appl. 2016, 21, 20. [Google Scholar] [CrossRef]
  63. Tokan, F.; Türker, N.; Yıldırım, T. ROC analysis as a useful tool for performance evaluation of artificial neural networks. In International Conference on Artificial Neural Networks; Kollias, S., Stafylopatis, A., Duch, W., Oja, E., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 923–931. [Google Scholar]
  64. Jinorose, M.; Prachayawarakorn, S.; Soponronnarit, S. A Novel Image-Analysis Based Approach to Evaluate Some Physicochemical and Cooking Properties of Rice Kernels. J. Food Eng. 2014, 124, 184–190. [Google Scholar] [CrossRef]
  65. Huggins, T.D.; Chen, M.; Fjellstrom, R.G.; Jackson, A.K.; McClung, A.M.; Edwards, J.D. Association Analysis of Three Diverse Rice (Oryza sativa L.) Germplasm Collections for Loci Regulating Grain Quality Traits. Plant Genome 2019, 12, 1–21. [Google Scholar] [CrossRef] [Green Version]
  66. Saikia, S.; Dutta, H.; Saikia, D.; Mahanta, C.L. Quality Characterisation and Estimation of Phytochemicals Content and Antioxidant Capacity of Aromatic Pigmented and Non-Pigmented Rice Varieties. Food Res. Int. 2012, 46, 334–340. [Google Scholar] [CrossRef]
  67. Melini, V.; Acquistucci, R. Health-Promoting Compounds in Pigmented Thai and Wild Rice. Foods 2017, 6, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Lin, Z.; Zheng, D.; Zhang, X.; Wang, Z.; Lei, J.; Liu, Z.; Li, G.; Wang, S.; Ding, Y. Chalky Part Differs in Chemical Composition from Translucent Part of Japonica Rice Grains as Revealed by a Notched-Belly Mutant with White-Belly. J. Sci. Food Agric. 2016, 96, 3937–3943. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Lamberts, L.; de Bie, E.; Vandeputte, G.E.; Veraverbeke, W.S.; Derycke, V.; de Man, W.; Delcour, J.A. Effect of Milling on Colour and Nutritional Properties of Rice. Food Chem. 2007, 100, 1496–1503. [Google Scholar] [CrossRef]
  70. Cruz, N.D.; Khush, G.S. Rice Grain Quality Evaluation Procedures. Aromat. Rices 2000, 3, 15–28. [Google Scholar]
  71. Castillo, E.; Iglesias, A.; Ruiz-Cobo, R. Functional Equations in Applied Sciences; Elsevier: Amsterdam, The Netherlands, 2004. [Google Scholar]
  72. Kambo, R.; Yerpude, A. Classification of Basmati Rice Grain Variety Using Image Processing and Principal Component Analysis. Int. J. Comput. Trends Technol. 2014, 11, 80–85. [Google Scholar] [CrossRef] [Green Version]
  73. Anami, B.S.; Malvade, N.N.; Palaiah, S. Automated Recognition and Classification of Adulteration Levels from Bulk Paddy Grain Samples. Inf. Process. Agric. 2019, 6, 47–60. [Google Scholar] [CrossRef]
  74. Summerson, V.; Viejo, C.G.; Torrico, D.D.; Pang, A. Detection of Smoke-Derived Compounds from Bushfires in Cabernet-Sauvignon Grapes, Must, and Wine Using Near-Infrared Spectroscopy and Machine Learning Algorithms. OENO One 2020, 54, 1105–1119. [Google Scholar] [CrossRef]
  75. Vemireddy, L.R.; Satyavathi, V.V.; Siddiq, E.A.; Nagaraju, J. Review of Methods for the Detection and Quantification of Adulteration of Rice: Basmati as a Case Study. J. Food Sci. Technol. 2015, 52, 3187–3202. [Google Scholar] [CrossRef] [Green Version]
  76. Gonzalez Viejo, C.; Caboche, C.H.; Kerr, E.D.; Pegg, C.L.; Schulz, B.L.; Howell, K.; Fuentes, S. Development of a Rapid Method to Assess Beer Foamability Based on Relative Protein Content Using RoboBEER and Machine Learning Modeling. Beverages 2020, 6, 28. [Google Scholar] [CrossRef]
Figure 1. Experimental setup for image acquisition consisting of (1) a lightbox; (2) 70 pieces of two LED light strips; (3) a platform; (4) cardboard to place the grain for image acquisition; (5) rice samples; (6) top opening of the lightbox used to acquire the images using a smartphone. LED = light-emitting diode.
Figure 1. Experimental setup for image acquisition consisting of (1) a lightbox; (2) 70 pieces of two LED light strips; (3) a platform; (4) cardboard to place the grain for image acquisition; (5) rice samples; (6) top opening of the lightbox used to acquire the images using a smartphone. LED = light-emitting diode.
Sensors 21 06354 g001
Figure 2. Examples of the 15 commercial rice sample images used in the study. The details of the rice samples based on the labeled class ID correspond to the list in Table 1. Rice sample images shown in the figure were cropped for presentation purposes only.
Figure 2. Examples of the 15 commercial rice sample images used in the study. The details of the rice samples based on the labeled class ID correspond to the list in Table 1. Rice sample images shown in the figure were cropped for presentation purposes only.
Sensors 21 06354 g002
Figure 3. Flow diagram of the method used to extract the morpho-colorimetric features from the rice grain image.
Figure 3. Flow diagram of the method used to extract the morpho-colorimetric features from the rice grain image.
Sensors 21 06354 g003
Figure 4. Diagram of a neural network model (Model 1) of the Bayesian Regularization algorithm with seven hidden neurons and sigmoid function showing nine inputs of morpho-colorimetric parameters and 15 outputs of commercial rice grains. The abbreviations for the morpho-colorimetric parameters (inputs) and commercial rice grains (outputs) are shown in Table 1 and Table 2. w = weight; b = bias.
Figure 4. Diagram of a neural network model (Model 1) of the Bayesian Regularization algorithm with seven hidden neurons and sigmoid function showing nine inputs of morpho-colorimetric parameters and 15 outputs of commercial rice grains. The abbreviations for the morpho-colorimetric parameters (inputs) and commercial rice grains (outputs) are shown in Table 1 and Table 2. w = weight; b = bias.
Sensors 21 06354 g004
Figure 5. Diagram of a neural network model (Model 2) of the Bayesian Regularization algorithm with seven hidden neurons and sigmoid function showing nine inputs of morpho-colorimetric parameters and 15 outputs of commercial rice grains. The abbreviation for morpho-colorimetric parameters (inputs) and commercial rice grains (outputs) are shown in Table 1 and Table 2. w = weight; b = bias.
Figure 5. Diagram of a neural network model (Model 2) of the Bayesian Regularization algorithm with seven hidden neurons and sigmoid function showing nine inputs of morpho-colorimetric parameters and 15 outputs of commercial rice grains. The abbreviation for morpho-colorimetric parameters (inputs) and commercial rice grains (outputs) are shown in Table 1 and Table 2. w = weight; b = bias.
Sensors 21 06354 g005
Figure 6. Flow diagram of the rice classification using computer vision and machine learning analysis. ANN = artificial neural network.
Figure 6. Flow diagram of the rice classification using computer vision and machine learning analysis. ANN = artificial neural network.
Sensors 21 06354 g006
Figure 7. The multivariate data analysis for (a) principal component analysis (PCA) biplot for morpho-colorimetric parameters of 17 commercial rice types, where PC1 = principal component one, PC2 = principal component two; and (b) cluster analysis of the commercial rice samples based on morpho-colorimetric parameters. The abbreviations for morpho-colorimetric parameters and rice samples are shown in Table 1 and Table 2.
Figure 7. The multivariate data analysis for (a) principal component analysis (PCA) biplot for morpho-colorimetric parameters of 17 commercial rice types, where PC1 = principal component one, PC2 = principal component two; and (b) cluster analysis of the commercial rice samples based on morpho-colorimetric parameters. The abbreviations for morpho-colorimetric parameters and rice samples are shown in Table 1 and Table 2.
Sensors 21 06354 g007
Figure 8. The receiver operating characteristic (ROC) curve of the artificial neural network models to classify 15 commercial rice samples using morpho-colorimetric parameters as inputs for (a) Model 1 and (b) Model 2. The abbreviations for rice samples are shown in Table 1.
Figure 8. The receiver operating characteristic (ROC) curve of the artificial neural network models to classify 15 commercial rice samples using morpho-colorimetric parameters as inputs for (a) Model 1 and (b) Model 2. The abbreviations for rice samples are shown in Table 1.
Sensors 21 06354 g008
Table 1. Details of commercial rice samples, including class ID, product category, type, origin, abbreviation, and the number of grains per rice type in triplicates obtained using Lightbox 1.
Table 1. Details of commercial rice samples, including class ID, product category, type, origin, abbreviation, and the number of grains per rice type in triplicates obtained using Lightbox 1.
Class IDProduct CategoryTypeOriginAbbreviationNumber of Grains
1White rice
(Polished rice)
Khoshihikari aJapanKHO243
2Sushi rice aJapanSRS276
3Bomba aSpainBMB210
4Calasparra aSpainCLP263
5Arborio bItalyARB126
6Calrose bAustraliaCLS230
7Long-grain cThailandLGW195
8Jasmine cThailandJAS297
9Basmati cPakistanBAS330
10Whole grain rice
(Unpolished rice)
Biodynamic rice bAustraliaBDM224
11Medium grain bAustraliaMGB212
12Medium-grain—organic bIndiaMOB274
13Doongara cAustraliaDGR308
14Black rice cThailandBKR317
15Wild rice—organic cUSAWRO335
a short-grain; b medium-grain; c long-grain.
Table 2. The adimensional morpho-colorimetric features and indices that were used to develop the ML (machine learning) model to classify 15 types of commercial rice grains.
Table 2. The adimensional morpho-colorimetric features and indices that were used to develop the ML (machine learning) model to classify 15 types of commercial rice grains.
ParametersAbbreviationDescription
Fractal dimensionFDFractal dimension obtained from the box-counting method [40].
CircularityCirDegree of object roundness, which returned the value between 0 to 1. The value 1 indicates a perfect circle.
Aspect ratioARThe ratio between major and minor axis length [50].
ExtentExtThe ratio between the rice grain area and bounding box area.
Area-Perimeter Ratio IndexAPIdxAPIdx = [(A/P) − (A/P)min]/(A/P)max
CIELab color scaleL, a and bLightness (L), red to green color range (a), and yellow to blue color range (b) [40].
Yellowness IndexYIDegree of yellowness [49].
Table 3. The mean and ± standard error (SE) values for morpho-colorimetric parameters of the commercial rice samples.
Table 3. The mean and ± standard error (SE) values for morpho-colorimetric parameters of the commercial rice samples.
Rice SamplenFDSECirSEARSEExtSEAPIdxSELSEaSEbSEYISE
KHO2431.82 a±0.080.89 b±0.021.63 h±0.090.72 a±0.040.70 c,d±0.0855.96 d±1.611.93 e±0.55−1.61 f±1.81−4.17 i±4.69
SRS2761.82 a±0.080.90 a,b±0.021.64 h±0.100.72 a,b±0.040.71 c,d±0.0853.80 f±1.812.05 e±0.17−0.91 e±0.42−2.42 g,h±1.12
BMB2101.83 a±0.100.91 a,b±0.041.61 h±0.110.73 a±0.050.75 c±0.2055.20 e±1.751.89 e±0.380.70 c±0.651.78 e,f±1.64
CLP2621.82 a±0.120.91 a±0.081.58 h±0.160.74 a±0.050.69 d±0.3154.62 e±2.092.52 c±0.37−0.34 d±0.40−0.88 g±1.05
ARB1261.66 c,d±0.040.83 c±0.021.94 g±0.100.67 c±0.051.17 a±0.1054.87 e±1.711.34 f,g±0.31−0.16 d±0.82−0.40 f,g±2.16
CLS2301.67 c±0.180.83 c±0.041.97 g±0.200.68 c±0.070.75 c±0.0756.71 c±3.311.27 g±0.52−0.35 d±1.01−0.91 g±2.52
LGW1951.57 f±0.100.61 g±0.043.32 c±0.340.57 d,e±0.130.46 f±0.0859.39 a±1.061.31 g±0.19−1.82 f±0.31−4.38 i±0.76
JAS2971.59 e,f±0.100.64 f±0.093.18 d±0.470.59 d±0.140.50 f±0.2159.50 a±3.411.30 g±0.40−1.43 f±0.36−3.56 h,i±0.50
BAS3301.58 f±0.170.59 h±0.113.59 b±0.660.55 e±0.150.27 h±0.1859.16 a±1.471.52 f±0.25−1.53 f±1.60−3.71 h,i±3.87
BDM2241.62 d,e±0.160.82 c±0.032.06 g±0.180.69 b,c±0.080.83 b±0.1153.10 g±1.932.27 d±0.303.91 b±1.0310.55 d±2.88
MGB2121.59 e,f±0.120.79 d±0.062.27 f±0.260.6 c±0.100.71 c,d±0.2955.20 e±1.461.97 e±0.274.72 a±2.0212.22 c,d±5.23
MOB2741.71 b±0.170.78 d±0.062.33 f±0.310.67 c±0.100.49 f±0.1158.01 b±0.971.99 e±0.201.08 c±0.632.67 e±1.57
DGR3081.59 e,f±0.090.67 e±0.033.06 e±0.230.60 d±0.120.63 e±0.0952.99 g±1.651.51 f±0.294.86 a±1.8113.18 c±5.09
BKR3171.59 e,f±0.090.60 g,h±0.053.56 b±0.390.56 e±0.140.50 f±0.1127.54 i±2.2214.64 a±1.324.97 a±2.3525.11 a±10.43
WRO3351.66 c±0.120.46 i±0.055.19 a±0.820.45 f±0.160.39 g±0.1229.60 h±2.327.95 b±0.944.19 b±2.8819.59 b±13.50
Mean values with different letters for each parameter indicate significant differences based on ANOVA (p < 0.001) and Tukey’s honestly significant difference (HSD) test (α < 0.05). Abbreviations of samples are found in Table 1 and morpho-colorimetric parameters in Table 2.
Table 4. Statistical results of the ANN model using Bayesian Regularization algorithm of the artificial neural network.
Table 4. Statistical results of the ANN model using Bayesian Regularization algorithm of the artificial neural network.
StageSample (n)Accuracy (%)Error (%)Performance (MSE)
Model 1 (Neurons = 7)
Training268795.05.0<0.01
Validation----
Testing115287.812.20.01
Overall383992.97.1-
Model 2 (Neurons = 10)
Training688791.68.4<0.01
Validation----
Testing295288.511.50.01
Overall983990.79.3
Table 5. Results of deployment accuracy when tested on the new data set obtained from 2D PhotoBench 120 lightbox. The abbreviations for the rice samples are shown in Table 1.
Table 5. Results of deployment accuracy when tested on the new data set obtained from 2D PhotoBench 120 lightbox. The abbreviations for the rice samples are shown in Table 1.
Rice SampleClassDeployment Accuracy (%)
KHO196.0
SRS294.2
BMB396.0
CLP482.0
ARB594.0
CLS694.0
LGW787.8
JAS898.0
BAS988.0
BDM1094.0
MGB1198.0
MOB1292.0
DGR1396.0
BKR1498.0
WRO15100.0
Mean (%)93.9
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Aznan, A.; Gonzalez Viejo, C.; Pang, A.; Fuentes, S. Computer Vision and Machine Learning Analysis of Commercial Rice Grains: A Potential Digital Approach for Consumer Perception Studies. Sensors 2021, 21, 6354. https://doi.org/10.3390/s21196354

AMA Style

Aznan A, Gonzalez Viejo C, Pang A, Fuentes S. Computer Vision and Machine Learning Analysis of Commercial Rice Grains: A Potential Digital Approach for Consumer Perception Studies. Sensors. 2021; 21(19):6354. https://doi.org/10.3390/s21196354

Chicago/Turabian Style

Aznan, Aimi, Claudia Gonzalez Viejo, Alexis Pang, and Sigfredo Fuentes. 2021. "Computer Vision and Machine Learning Analysis of Commercial Rice Grains: A Potential Digital Approach for Consumer Perception Studies" Sensors 21, no. 19: 6354. https://doi.org/10.3390/s21196354

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop