Elsevier

Pattern Recognition

Volume 34, Issue 12, December 2001, Pages 2259-2281
Pattern Recognition

Color image segmentation: advances and prospects

https://doi.org/10.1016/S0031-3203(00)00149-7Get rights and content

Abstract

Image segmentation is very essential and critical to image processing and pattern recognition. This survey provides a summary of color image segmentation techniques available now. Basically, color segmentation approaches are based on monochrome segmentation approaches operating in different color spaces. Therefore, we first discuss the major segmentation approaches for segmenting monochrome images: histogram thresholding, characteristic feature clustering, edge detection, region-based methods, fuzzy techniques, neural networks, etc.; then review some major color representation methods and their advantages/disadvantages; finally summarize the color image segmentation techniques using different color representations. The usage of color models for image segmentation is also discussed. Some novel approaches such as fuzzy method and physics-based method are investigated as well.

Introduction

Image segmentation is the first step in image analysis and pattern recognition. It is a critical and essential component of image analysis and/or pattern recognition system, is one of the most difficult tasks in image processing, and determines the quality of the final result of analysis. Image segmentation is a process of dividing an image into different regions such that each region is, but the union of any two adjacent regions is not, homogeneous. A formal definition of image segmentation is as follows [1]: If P() is a homogeneity predicate defined on groups of connected pixels, then segmentation is a partition of the set F into connected subsets or regions (S1,S2,…,Sn) such thati=1nSi=FwithSi∩Sj=Φ(i≠j).

The uniformity predicate P(Si)=true for all regions, Si, and P(SiSj)=false, when ij and Si and Sj are neighbors.

According to [2], “the image segmentation problem is basically one of psychophysical perception, and therefore not susceptible to a purely analytical solution”. There are many papers and several surveys on monochrome image segmentation techniques. Color image segmentation attracts more and more attention mainly due to the following reasons: (1) color images can provide more information than gray level images; (2) the power of personal computers is increasing rapidly, and PCs can be used to process color images now. The segmentation techniques for monochrome images can be extended to segment color images by using R, G and B or their transformations (linear/non-linear). However, comprehensive surveys on color image segmentation are few. Ref. [3] analyzed the problem when applying edge-based and region-based segmentation techniques to color images with complex texture, and Ref. [4] discussed the properties of several color representations, the segmentation methods and color spaces.

This paper provides a summary of color image segmentation techniques available at present, and describes the properties of different kinds of color representation methods and problems encountered when applying the color models to image segmentation. Some novel approaches such as fuzzy and physics-based approaches will be discussed as well.

Section 2 briefly introduces the major segmentation approaches for processing monochrome images: histogram thresholding, characteristic feature clustering, edge detection, region-based methods, fuzzy techniques and neural networks. Section 3 reviews some major color representations and their advantages/disadvantages. Section 4 investigates the segmentation techniques applied to color images using different color representations, and the summary is given in Section 5.

Section snippets

Monochrome image segmentation

Monochrome image segmentation approaches are based on either discontinuity and/or homogeneity of gray level values in a region. The approach based on discontinuity tends to partition an image by detecting isolated points, lines and edges according to abrupt changes in gray levels. The approaches based on homogeneity include thresholding, clustering, region growing, and region splitting and merging. Several survey papers on monochrome image segmentation [1], [2], [5], [6], [7], [8] cover the

Color features

Color is perceived by humans as a combination of tristimuli R (red), G (green), and B (blue) which are usually called three primary colors. From R,G,B representation, we can derive other kinds of color representations (spaces) by using either linear or nonlinear transformations. Several color spaces, such as RGB, HSI, CIE L∗u∗v∗ are utilized in color image segmentation, but none of them can dominate the others for all kinds of color images. Selecting the best color space still is one of the

Color image segmentation

It has long been recognized that human eye can discern thousands of color shades and intensities but only two-dozen shades of gray. It is quite often when the objects cannot be extracted using gray scale but can be extracted using color information. Compared to gray scale, color provides information in addition to intensity. Color is useful or even necessary for pattern recognition and computer vision. Also the acquisition and processing hardwares for color images have become more available and

Summary

There is no universal theory on color image segmentation yet. All of the existing color image segmentation approaches are, by nature, ad hoc. They are strongly application dependent, in other words, there are no general algorithms and color space that are good for all color images. An image segmentation problem is basically one of psychophysical perception, and it is essential to supplement any mathematical solutions by a priori knowledge about the picture knowledge. Most gray level image

About the Author—HENG-DA CHENG received Ph.D. degree in Electrical Engineering from Purdue University, West Lafayette, IN, in 1985 (supervisor: K.S. Fu). Now he is Full Professor, Department of Computer Science, and an Adjunct Full Professor, Department of Electrical Engineering, Utah State University, Logan, UT. Dr. Cheng also holds a Visiting Professor position of Northern Jiaotong University, China.

Dr. Cheng has published more than 180 technical papers, is the co-editor of the book, Pattern

References (139)

  • K.B. Eom

    Segmentation of monochrome and color texture using moving average modeling approach

    Image Vision Comput.

    (1999)
  • A. Sarabi et al.

    Segmentation of chromatic images

    Pattern Recognition

    (1981)
  • S.A. Underwood et al.

    Interactive computer analysis of aerial color infrared photographs

    Comput. Graphics Image Process.

    (1977)
  • R. Ohlander et al.

    Picture segmentation using a recursive region splitting method

    Comput. Graphics Image Process.

    (1978)
  • M. Celenk

    A color clustering technique for image segmentation

    Graphical Models Image Process.

    (1990)
  • F. Ferri et al.

    Color image segmentation and labeling through multi-edit condensing

    Pattern Recognition Lett.

    (1992)
  • A. Tremeau et al.

    A region growing and merging algorithm to color segmentation

    Pattern Recognition

    (1997)
  • N. Ito

    The combination of edge detection and region extraction in non-parametric color image segmentation

    Inform. Sci.

    (1996)
  • L.K. Huang et al.

    Image thresholding by minimizing the measures of fuzziness

    Pattern Recognition

    (1995)
  • A. De Luca et al.

    A definition of a nonprobabilistic entropy in the setting of fuzzy set theory

    Inform. Control

    (1972)
  • M.P. Windham

    Geometrical fuzzy clustering algorithms

    Fuzzy Sets and Systems

    (1983)
  • D.N. Chun et al.

    Robust image segmentation using genetic algorithm with a fuzzy measure

    Pattern Recognition

    (1996)
  • S.K. Pal et al.

    Image enhancement and thresholding by optimization of fuzzy compactness

    Pattern Recognition Lett.

    (1988)
  • M.P. Windham

    Geometrical fuzzy clustering algorithms

    Fuzzy Sets and Systems

    (1983)
  • S.K. Pal

    A review on image segmentation techniques

    Pattern Recognition

    (1993)
  • W. Skarbek, A. Koschan, Colour image segmentation – a survey, Technical Report, Tech. Univ. of Berlin, October...
  • L. Spirkovska, A summary of image segmentation techniques, NASA Technical Memorandum 104022, June...
  • C.H. Chen, On the statistical image segmentation techniques, Proceedings of IEEE Conference on Pattern Recognition and...
  • A. Rosenfeld et al.

    Edge and curve detection for visual scene analysis

    IEEE Trans. Comput.

    (1971)
  • A. Rosenfeld et al.

    Edge and curve detection further experiments

    IEEE Trans. Comput.

    (1972)
  • M. Hueckel

    An operator which locates edges in digitized pictures

    J. Assoc. Comput. Mach.

    (1971)
  • M. Hueckel

    A local visual operator which recognizes edges and lines

    J. Assoc. Comput. Mach.

    (1973)
  • J. Gauch, Chi-Wan Hsia, A comparison of three color image segmentation algorithm in four color spaces, SPIE Vol. 1818...
  • D.E.P. Hoy

    On the use of color imaging in experimental applications

    Exp. Tech.

    (1997)
  • M. Chapron, A new chromatic edge detector used for color image segmentation, IEEE International Conference on Pattern...
  • M.T. Orchard et al.

    Color quantization of images

    IEEE Trans. Signal Process.

    (1991)
  • D. Comaniciu, P. Meer, Robust analysis of feature spaces: color image segmentation, IEEE Conference on Computer Vision...
  • M. Pietikainen et al., Accurate color discrimination with classification based on feature distributions International...
  • E. Littmann et al.

    Adaptive color segmentation – a comparison of neural and statistical methods

    IEEE Trans. Neural Network

    (1997)
  • G.S. Robinson

    Color edge detection

    Opt. Eng.

    (1977)
  • Nevatia, A color edge detector and its use in scene segmentation, IEEE Trans. System Man Cybernet. SMC-7 (11) (1977)...
  • J.C. Terrillon, M. David, S. Akamatsu, Detection of human faces in complex scene images by use of a skin color model...
  • T.L. Huntsberger et al.

    Iterative fuzzy image segmentation

    Pattern Recognition

    (1985)
  • T. Carron, P. Lambert, Color edge detector using jointly hue, saturation and intensity, IEEE International Conference...
  • Y. Rui, A.C. She, T.S. Huang, Automated region segmentation using attraction-based grouping in spatial-color-texture...
  • W.S. Kim, R.H. Park, Color image palette construction based on the HSI color system for minimizing the reconstruction...
  • P.W.M. Tsang, W.H. Tsang, Edge detection on object color, IEEE International Conference on Image Processing, C, 1996,...
  • E. Tepichin et al.

    Hue, brightness, and saturation manipulation of diffractive colors

    Opt. Eng.

    (1995)
  • K.M. Kim, C.S. Lee, Y.H. Ha, Color image quantization using weighted distortion measure of HVS color activity, IEEE...
  • D.C. Tseng, C.H. Chang, Color segmentation using perceptual attributes, IEEE International Conference on Pattern...
  • Cited by (1486)

    View all citing articles on Scopus

    About the Author—HENG-DA CHENG received Ph.D. degree in Electrical Engineering from Purdue University, West Lafayette, IN, in 1985 (supervisor: K.S. Fu). Now he is Full Professor, Department of Computer Science, and an Adjunct Full Professor, Department of Electrical Engineering, Utah State University, Logan, UT. Dr. Cheng also holds a Visiting Professor position of Northern Jiaotong University, China.

    Dr. Cheng has published more than 180 technical papers, is the co-editor of the book, Pattern Recognition: Algorithms, Architectures and Applications (World Scientific, 1991), and the editor for a couple conference proceedings. His research interests include image processing, pattern recognition, computer vision, artificial intelligence, medical information processing, fuzzy logic, genetic algorithms, neural networks, parallel processing, parallel algorithms, and VLSI architectures.

    Dr. Cheng was the general chair and program chair of the Third International Conference on Computer Vision, Pattern Recognition and Image Precessing (CVPRIP’2000), 2000; was the general chair and program chair of the First International Workshop on Computer Vision, Pattern Recognition and Image Processing (CVPRIP’98), 1998, and was the Program Co-Chairman of Vision Interface ’90, 1990. He served as a program committee member and session chair for many conferences, and as a reviewer for many scientifical journals and conferences.

    Dr. Cheng has been listed in Whos Who in the World, Whos Who in America, Whos Who in Communications and Media, Whos Who in Finance and Industry, Whos Who in Science and Engineering, Men of Achievement, 2000 Notable American Men, International Leaders in Achievement, Five Hundred Leaders of Influence, International Dictionary of Distinguished Leadership, etc. He is appointed as a Member of the Advisory Council, the International Biographical Center, England, and a Member of the Board of Advisors, the American Biographical Institute, USA.

    Dr. Cheng is a Senior Member of the IEEE Society, and a Member of the Association of Computing Machinery. Dr. Cheng is also an Associate Editor of Pattern Recognition and an Associate Editor of Information Sciences.

    About the Author—XIHUA JIANG received the Bachelor of Science degree from the Department of Computer and System Science, Nankai University, Tianjin, China, in 1988, the Master of Science degree from the Department of Computer Science and Technology, Peking University, Beijing, China, in 1997, and the Master of Science degree from the Department of Computer Science, Utah State University, Logan, UT, in 1999.

    From 1988 to 1994, he was a research assistant and software engineer in the Department of Computer and System Science, Nankai University, Tianjin, China, and participated in the design, development, and implementation of several robot control systems for the national hi-tech plan in China. Since 1999, he has been with the Software Engineering Department at Citrix Systems, Inc. as a software engineer. His research interests are fuzzy logic, image processing and server-client computing.

    About the Author—YING SUN received the Bachelor of Science degree in computer software in 1989 from the Department of Computer and System Science, Nankai University, Tianjin, China, and the Master of Science degree in computer science from Utah State University, Logan, UT, in 1999.

    From 1991 to 1995, she was a software engineer in the Computing Center of Tianjin Foreign Trade and Economic Relations Committee, Tianjin, China, and participated in the design, development, and implementation of several financial management information systems for the state-run foreign trade companies in Tianjin, China. From 1996 to 1997, she joined Motorola (China) Electronics Ltd. and worked in Management Information System Department, Cellular Subscriber Sector, as an application analyst. She conducted software project design, plan, program, and trouble-shooting. Since 1999, she has been with the information Technology Group, Personal Communication Systems, Motorola Inc. in Libertyville, ILL, as a software engineer. Her research interests include computer algorithms and image processing methodology.

    About the Author—JINGLI WANG received the bachelor of science degree in computer science in 1990 from the Computer Science Department, Zhengzhou University, China. From 1990 to 1999, she was a software engineer in Chinese Academy of Sciences, participated in remote sensing instruments design, development, and implementation, worked on remote sensing information processing, such as real-time control and data processing, simulation, and calibration for the satellite and airborne altimeter and conducted software project designing, planning and programming. Since 1999, she has been a graduate student in Utah State University with a major in Computer Science. Her research interests include software engineering and image processing methodology.

    View full text