Skip to main content

Advertisement

SpringerLink
Log in

Appraisal of kappa-based metrics and disagreement indices of accuracy assessment for parametric and nonparametric techniques used in LULC classification and change detection

  • Original Article
  • Published:
Modeling Earth Systems and Environment Aims and scope Submit manuscript

Abstract

In the face of rapid urbanization, monitoring urban expansion has gained importance to sustainably manage the land resources and minimize the impact on the environment. Monitoring urban growth using satellite data involves computing the state of land use–land cover and their change over time. A number of computing methods have been developed to process and interpret the satellite results for an urban environment. However, due to a large number of parametric and nonparametric algorithms used for land-use–land-cover classification, there is uncertainty regarding choosing the best algorithm to measure the urban processes. In this study, several parametric (maximum likelihood) and nonparametric (support vector machine, spectral angle mapper, artificial neural network and decision tree) algorithms were used. The study was aimed at finding out the best available classification technique for land-use–land-cover classification and change detection. Landsat 8, the latest in Landsat series, and Landsat 7 and 5 freely available satellite data were used. Due to the redundancy reported for the traditional kappa-based indices, we applied modern disagreement indices to assess the accuracy of the classification process. Artificial neural network for Landsat 8 image had the highest kappa coefficient, while spectral angle mapper had the highest overall agreement (97%) and least quantity allocation error (1%). Spectral Angle Mapper gave the highest accuracy, while maximum likelihood classification gave the least for allocation and spatial disagreement indices. We found that spectral angle mapper gave the best results for land-use–land-cover change analysis in terms of least omission and commission errors (2.5% each) and highest overall agreement, whereas artificial neural network performed better in land-use–land-cover classification studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aghsaei H, Dinan NM, Moridi A, Asadolahi Z, Delavar M, Fohrer N, Wagner PD (2020) Effects of dynamic land use/land cover change on water resources and sediment yield in the Anzali wetland catchment, Gilan, Iran. Sci Total Environ 712:136449

    Google Scholar 

  • Aithal BH, Ramachandra TV (2016) Visualization of urban growth pattern in Chennai using geoinformatics and spatial metrics. J Indian Soc Remote Sens 44(4):617–633

    Google Scholar 

  • Anasuya B, Swain D, Vinoj V (2019) Rapid urbanization and associated impacts on land surface temperature changes over Bhubaneswar Urban District, India. Environ Monit Assess 191(3):790

    Google Scholar 

  • Bryant MM (2006) Urban landscape conservation and the role of ecological greenways at local and metropolitan scales. Landsc Urban Plan 76(1):23–44

    Google Scholar 

  • Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167

    Google Scholar 

  • Chatterjee R, Verma P, Shaw R, Raghubanshi AS (2015) Climate and disaster resilience of Varanasi. Kyoto University, Banaras Hindu University, Varanasi Nagar Nigam, Kyoto, p 76

    Google Scholar 

  • Comber A, Fisher P, Brunsdon C, Khmag A (2012) Spatial analysis of remote sensing image classification accuracy. Remote Sens Environ 127:237–246

    Google Scholar 

  • Deilami K, Kamruzzaman M, Liu Y (2018) Urban heat island effect: a systematic review of spatio-temporal factors, data, methods, and mitigation measures. Int J Appl Earth Obs Geoinf 67:30–42

    Google Scholar 

  • Dingle Robertson L, King DJ (2011) Comparison of pixel-and object-based classification in land cover change mapping. Int J Remote Sens 32(6):1505–1529

    Google Scholar 

  • Dixon B, Candade N (2008) Multispectral landuse classification using neural networks and support vector machines: one or the other, or both? Int J Remote Sens 29(4):1185–1206

    Google Scholar 

  • Eck DL (1982) Banaras, city of light. Columbia University Press, Columbia

    Google Scholar 

  • Foody G (2003) Remote sensing of tropical forest environments: towards the monitoring of environmental resources for sustainable development. Int J Remote Sens 24(20):4035–4046

    Google Scholar 

  • Friedl MA, Brodley CE (1997) Decision tree classification of land cover from remotely sensed data. Remote Sens Environ 61(3):399–409

    Google Scholar 

  • Gašparović M, Zrinjski M, Gudelj M (2019) Automatic cost-effective method for land cover classification (ALCC). Comput Environ Urban Syst 76:1–10

    Google Scholar 

  • Gopal S, Woodcock C (1996) Remote sensing of forest change using artificial neural networks. Geosci Remote Sens IEEE Trans 34(2):398–404

    Google Scholar 

  • Grimm NB, Faeth SH, Golubiewski NE, Redman CL, Wu J, Bai X, Briggs JM (2008) Global change and the ecology of cities. Science 319(5864):756–760

    Google Scholar 

  • He Q, Dai L, Zhang W, Wang H, Liu S, He S (2013) An unsupervised classifier for remote-sensing imagery based on improved cellular automata. Int J Remote Sens 34(21):7821–7837

    Google Scholar 

  • Huang C, Davis L, Townshend J (2002) An assessment of support vector machines for land cover classification. Int J Remote Sens 23(4):725–749

    Google Scholar 

  • Huang C, Song K, Kim S, Townshend JR, Davis P, Masek JG, Goward SN (2008) Use of a dark object concept and support vector machines to automate forest cover change analysis. Remote Sens Environ 112(3):970–985

    Google Scholar 

  • Jiang Z, Sun X, Liu F, Shan R, Zhang W (2019) Spatio-temporal variation of land use and ecosystem service values and their impact factors in an urbanized agricultural basin since the reform and opening of China. Environ Monit Assess 191(12):739

    Google Scholar 

  • Johnson BA, Iizuka K, Bragais MA, Endo I, Magcale-Macandog DB (2017) Employing crowdsourced geographic data and multi-temporal/multi-sensor satellite imagery to monitor land cover change: a case study in an urbanizing region of the Philippines. Comput Environ Urban Syst 64:184–193

    Google Scholar 

  • Kandrika S, Roy PS (2008) Land use land cover classification of Orissa using multi-temporal IRS-P6 awifs data: a decision tree approach. Int J Appl Earth Obs Geoinf 10(2):186–193

    Google Scholar 

  • Kavzoglu T, Reis S (2008) Performance analysis of maximum likelihood and artificial neural network classifiers for training sets with mixed pixels. GISci Remote Sens 45(3):330–342

    Google Scholar 

  • Kumar M, Mukherjee N, Sharma GP, Raghubanshi A (2010) Land use patterns and urbanization in the holy city of Varanasi, India: a scenario. Environ Monit Assess 167(1–4):417–422

    Google Scholar 

  • Kumar P, Gupta DK, Mishra VN, Prasad R (2015) Comparison of support vector machine, artificial neural network, and spectral angle mapper algorithms for crop classification using LISS IV data. Int J Remote Sens 36(6):1604–1617

    Google Scholar 

  • Li G, Lu D, Moran E, Sant’Anna SJ (2012) Comparative analysis of classification algorithms and multiple sensor data for land use/land cover classification in the Brazilian Amazon. J Appl Remote Sens 6(1):061706

    Google Scholar 

  • Li X, Chen G, Liu J, Chen W, Cheng X, Liao Y (2017) Effects of RapidEye imagery’s red-edge band and vegetation indices on land cover classification in an arid region. Chin Geogr Sci 27(5):827–835

    Google Scholar 

  • Li W, MacBean N, Ciais P, Defourny P, Lamarche C, Bontemps S, Peng S (2018) Gross and net land cover changes in the main plant functional types derived from the annual ESA CCI land cover maps (1992–2015)

  • Lu D, Mausel P, Brondizio E, Moran E (2004) Change detection techniques. Int J Remote Sens 25(12):2365–2401

    Google Scholar 

  • McKinney ML (2002) Urbanization, biodiversity, and conservation the impacts of urbanization on native species are poorly studied, but educating a highly urbanized human population about these impacts can greatly improve species conservation in all ecosystems. Bioscience 52(10):883–890

    Google Scholar 

  • Mondal A, Kundu S, Chandniha SK, Shukla R, Mishra P (2012) Comparison of support vector machine and maximum likelihood classification technique using satellite imagery. Int J Remote Sens GIS 1(2):116–123

    Google Scholar 

  • Mountrakis G, Im J, Ogole C (2011) Support vector machines in remote sensing: a review. ISPRS J Photogramm Remote Sens 66(3):247–259

    Google Scholar 

  • Nagendra H, Gopal D (2010) Street trees in Bangalore: density, diversity, composition and distribution. Urban For Urban Green 9(2):129–137

    Google Scholar 

  • Nagendra H, Nagendran S, Paul S, Pareeth S (2012) Graying, greening and fragmentation in the rapidly expanding Indian city of Bangalore. Lands Urban Plan 105(4):400–406

    Google Scholar 

  • Obeidat M, Awawdeh M, Lababneh A (2019) Assessment of land use/land cover change and its environmental impacts using remote sensing and GIS techniques, Yarmouk River Basin, north Jordan. Arab J Geosci 12(22):685

    Google Scholar 

  • Ornetsmüller C, Heinimann A, Verburg PH (2018) Operationalizing a land systems classification for Laos. Landsc Urban Plan 169:229–240

    Google Scholar 

  • Otukei JR, Blaschke T (2010) Land cover change assessment using decision trees, support vector machines and maximum likelihood classification algorithms. Int J Appl Earth Obs Geoinf 12:S27–S31

    Google Scholar 

  • Pal M, Mather PM (2003) An assessment of the effectiveness of decision tree methods for land cover classification. Remote Sens Environ 86(4):554–565

    Google Scholar 

  • Paliwal M, Kumar UA (2009) Neural networks and statistical techniques: a review of applications. Expert Syst Appl 36(1):2–17

    Google Scholar 

  • Petropoulos GP, Kalivas DP, Georgopoulou IA, Srivastava PK (2015) Urban vegetation cover extraction from hyperspectral imagery and geographic information system spatial analysis techniques: case of Athens, Greece. J Appl Remote Sens 9(1):096088

    Google Scholar 

  • Pontius RG Jr, Millones M (2011) Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. Int J Remote Sens 32(15):4407–4429

    Google Scholar 

  • Pontius RG Jr, Santacruz A (2014) Quantity, exchange, and shift components of difference in a square contingency table. Int J Remote Sens 35(21):7543–7554

    Google Scholar 

  • Pradhan B (2013) A comparative study on the predictive ability of the decision tree, support vector machine and neuro-fuzzy models in landslide susceptibility mapping using GIS. Comput Geosci 51:350–365

    Google Scholar 

  • Schalkoff RJ (1997) Artificial neural networks. McGraw-Hill Higher Education

  • Shao Y, Lunetta RS (2012) Comparison of support vector machine, neural network, and CART algorithms for the land-cover classification using limited training data points. ISPRS J Photogramm Remote Sens 70:78–87

    Google Scholar 

  • Sharma R, Ghosh A, Joshi P (2013) Decision tree approach for classification of remotely sensed satellite data using open source support. J Earth Syst Sci 122(5):1237–1247

    Google Scholar 

  • Singh SK, Srivastava PK, Gupta M, Thakur JK, Mukherjee S (2014) Appraisal of land use/land cover of mangrove forest ecosystem using support vector machine. Environ Earth Sci 71(5):2245–2255

    Google Scholar 

  • Singh SK, Mustak S, Srivastava PK, Szabó S, Islam T (2015) Predicting spatial and decadal LULC changes through cellular automata Markov chain models using earth observation datasets and geo-information. Environ Process 2(1):61–78

    Google Scholar 

  • Sloan S, Pelletier J (2012) How accurately may we project tropical forest-cover change? a validation of a forward-looking baseline for REDD. Global Environmen Change 22(2):440–453

    Google Scholar 

  • Sohl TL, Sleeter BM, Sayler KL, Bouchard MA, Reker RR, Bennett SL, Zhu Z (2012) Spatially explicit land-use and land-coverscenarios for the Great Plains of the United States. Agric Ecosyst Environ 153:1–15

    Google Scholar 

  • Srivastava PK, Han D, Rico-Ramirez MA, Bray M, Islam T (2012) Selection of classification techniques for land use/land cover change investigation. Adv Space Res 50(9):1250–1265

    Google Scholar 

  • Srivastava PK, Suman S, Pandey S (2017) Monitoring changes in urban cover using landsat satellite images and demographical information. In: Singh RP, Singh A, Srivastava V (eds) Environmental issues surrounding human overpopulation. IGI Global, Hershey, pp 89–103

    Google Scholar 

  • UN DESA (2018) 2018 revision of world urbanization prospects. Avaialble at: https://www.un.org/development/desa/publications/2018-revisionof-world-urbanization-prospects.html. Accessed 15 Jan 2019

  • Vapnik V (2013) The nature of statistical learning theory. Springer, Berlin

    Google Scholar 

  • Verma P, Raghubanshi AS (2018) Urban sustainability indicators: challenges and opportunities. Ecol Indic 93:282–291

    Google Scholar 

  • Verma P, Raghubanshi AS (2019) Rural development and land use land cover change in a rapidly developing agrarian South Asian landscape. Remote Sen Appl Soc Environ 14:138–147

    Google Scholar 

  • Wentz EA, Nelson D, Rahman A, Stefanov WL, Roy SS (2008) Expert system classification of urban land use/cover for Delhi, India. Int J Remote Sens 29(15):4405–4427

    Google Scholar 

  • Whitford V, Ennos AR, Handley JF (2001) “City form and natural process”—indicators for the ecological performance of urban areas and their application to Merseyside, UK. Landsc Urban Plan 57(2):91–103

    Google Scholar 

  • Yuan F, Sawaya KE, Loeffelholz BC, Bauer ME (2005) Land cover classification and change analysis of the Twin Cities (Minnesota) Metropolitan Area by multitemporal Landsat remote sensing. Remote Sens Environ 98(2):317–328

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Integrative Ecology Laboratory (IEL), Institute of Environment and Sustainable Development, Banaras Hindu University, Varanasi, 221005, India

    Pramit Verma & A. S. Raghubanshi

  2. Department of Computer Sciences, Viterbi School of Engineering, University of Southern California, Los Angeles, USA

    Aditya Raghubanshi

  3. Remote Sensing Laboratory, Institute of Environment and Sustainable Development, Banaras Hindu University, Varanasi, 221005, India

    Prashant K. Srivastava

Authors
  1. Pramit Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aditya Raghubanshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prashant K. Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. S. Raghubanshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. S. Raghubanshi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 14 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Verma, P., Raghubanshi, A., Srivastava, P.K. et al. Appraisal of kappa-based metrics and disagreement indices of accuracy assessment for parametric and nonparametric techniques used in LULC classification and change detection. Model. Earth Syst. Environ. 6, 1045–1059 (2020). https://doi.org/10.1007/s40808-020-00740-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40808-020-00740-x

Keywords

Search

Navigation