Lei Zhu
Abstract
The success of text-based retrieval motivates us to investigate analogous techniques which can support the querying and browsing of image data. However, images differ significantly from text both syntactically and semantically in their mode of representing and expressing information. Thus, the generalization of information retrieval from the text domain to the image domain is non-trivial. This paper presents a framework for information retrieval in the image domain which supports content-based querying and browsing of images. A critical first step to establishing such a framework is to construct a codebook of "keywords" for images which is analogous to the dictionary for text documents. We refer to such "keywords" in the image domain as "keyblocks." In this paper, we first present various approaches to generating a codebook containing keyblocks at different resolutions. Then we present a keyblock-based approach to content-based image retrieval. In this approach, each image is encoded as a set of one-dimensional index codes linked to the keyblocks in the codebook, analogous to considering a text document as a linear list of keywords. Generalizing upon text-based information retrieval methods, we then offer various techniques for image-based information retrieval. By comparing the performance of this approach with conventional techniques using color and texture features, we demonstrate the effectiveness of the keyblock-based approach to content-based image retrieval.
- Ahuja, N. and Rosenfeld, A. 1981. Mosaic models for texture. IEEE Trans. Patt. Anal. Mach. Intell. 3, 1, 1--11.Google Scholar
- Bach, J., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Jain, R., and Shu, C. 1996. The virage image search engine: An open framework for image management. In Proceedings of SPIE, Storage and Retrieval for Still Image and Video Databases IV. San Jose, CA, USA, 76--87.Google Scholar
- Baeza-Yates, R. and Ribiero-Neto, B. 1999. Modern Information Retrieval. Addison Wesley. Google Scholar
- Brodatz, P. 1966. Textures: A Photographic Album for Artists and Designers. Dover, New York.Google Scholar
- Dougherty, E. and Pelz, J. 1989. Texture-based segmentation by morphological granulometrics. In Advanced Printing of Paper Summaries, Electronic Imaging '89. Vol. 1. Boston, Massachusetts, 408--414.Google Scholar
- Ester, M., Kriegel, H., Sander, J., and Xu, X. 1996. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the 2nd International Conference on KDD. Portland, OR, 226--231.Google Scholar
- Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., and Equitz, W. 1994. Efficient and effective querying by image content. J. Intell. Inf. Syst. 3, 3/4 (July), 231--262. Google Scholar
- Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., and Dom., B. 1995. Query by Image and Video Content: The QBIC System. IEEE Comput. 28, 9 (Sept.), 23--32. Google Scholar
- Gersho, A. and Gray, R. M. 1992. Vector Quantization and Signal Compression. Kluwer Academic Publishers. Google Scholar
- Hirata, K. and Kato, T. 1993. Rough sketch-based image information retrieval. NEC Res. Dev. 34, 2, 263--273.Google Scholar
- Horn, B. K. P. 1988. Robot Vision, Fourth ed. The MIT Press. Google Scholar
- Hsu, W., Chua, T., and Pung, H. K. 1995. An integrated color-spatial approach to content-based image retrieval. Proceedings of the ACM Multimedia Conference, 305--313. Google Scholar
- Huang, J. 1998. Color-spatial image indexing and applications. Ph.D. dissertation, Cornell University. Google Scholar
- Huang, J., Kumar, S., Mitra, M., Zhu, W., and Zabih, R. 1997. Image indexing using color correlograms. In IEEE Conference on Computer Vision and Pattern Recognition. 762--768. Google Scholar
- Hunt, R. W. G. 1989. Measuring Color. Ellis Horwood series in applied science and industrial technology. Halsted Press, New York, NY.Google Scholar
- Idris, F. and Panchanathan, S. 1996. Algorithms for indexing of compressed images. In Proceedings of the International Conference on Visual Information Systems. Melbourne, 303--308.Google Scholar
- Jurafsky, D. and Martin, J. H. 2000. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice Hall. Google Scholar
- Kaufman, L. and Rousseeuw, P. J. 1990. Finding Groups in Data: an Introduction to Cluster Analysis. John Wiley & Sons.Google Scholar
- Kohonen, T., Hynninen, J., Kangas, J., Laaksonen, J., and Torkkola, K. 1995. Lvq pak: The learning vector quantization program package.Google Scholar
- Korn, F., Sidiropoulos, N., Faloutsos, C., Siegel, E., and Protopapas, Z. 1996. Fast nearest-neighbor search in medical image databases. In Conference on Very Large Data Bases (VLDB96). Google Scholar
- Lu, G. and Teng, S. 1999. A novel image retrieval technique based on vector quantization. In Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation. Vienna, Austria, 36--41.Google Scholar
- Mandelbrot, B. 1977. Fractals---Form, Chance, Dimension. W. H. Freeman, San Francisco, California.Google Scholar
- Manjunath, B. and Ma, W. 1996. Texture Features for Browsing and Retrieval of Image Data. IEEE Trans. Patt. Anal. Mach. Intell. 18, 8 (August), 837--842. Google Scholar
- Mehrotra, R. and Gary, J. E. 1995. Similar-shape retrieval in shape data management. IEEE Computer 28, 9 (September), 57--62. Google Scholar
- Modestino, J., Fries, R., and Vickers, A. 1981. Texture discrimination based upon an assumed stochastic texture model. IEEE Trans. Patt. Anal. Mach. Intell. 3, 5, 557--580.Google Scholar
- Mokhtarian, F., Abbasi, S., and Kittler, J. 1996a. Efficient and Robust Retrieval by Shape Content through Curvature Scale Space. In Proceedings of the International Workshop on Image Databases and MultiMedia Search. Amsterdam, The Netherlands, 35--42.Google Scholar
- Mokhtarian, F., Abbasi, S., and Kittler, J. 1996b. Robust and efficient shape indexing through curvature scale space. In Proceedings of the British Machine Vision Conference. Edinburgh, UK, 53--62.Google Scholar
- Netravali, A. N. and Haskell, B. G. 1988. Digital Pictures: representation and compression. Applications of Communications Theory. Plenum Press, New York, NY. Google Scholar
- Ng, R. T. and Han, J. 1994. Efficient and Effective Clustering Methods for Spatial Data Mining. In Proceedings of the 20th VLDB Conference. Santiago, Chile, 144--155. Google Scholar
- Niblack, W., Barker, R., Equitz, W., Flickner, M., Glasman, E., and Petkovic, D. P. 1993. The qbic project: Querying images by content using color, texture, and shape, IBM Tech. Rep.Google Scholar
- Orchard, M. T. and Bouman, C. A. 1991. Color Quantization of Images. IEEE Trans. Signal Proc. 39, 12 (December), 2677--2690.Google Scholar
- Pass, G. and Zabih, R. 1996. Histogram refinement for content-based image retrieval. IEEE Workshop on Applications of Computer Vision, 96--102. Google Scholar
- Pass, G., Zabih, R., and Miller, J. 1996. Comparing images using color coherence vectors. In Proceedings of ACM Multimedia 96. Boston, MA, USA, 65--73. Google Scholar
- Pauwels, E., Fiddelaers, P., and Gool, L. V. 1997. DOG-based unsupervized clustering for CBIR. In Proceedings of the 2nd International Conference on Visual Information Systems. San Diego, California, 13--20.Google Scholar
- Pentland, A., Picard, R., and Sclaroff, S. 1994. Photobook: Tools for Content-based Manipulation of Image Databases. In Proceedings of the SPIE Conference on Storage and Retrieval of Image and Video Databases II. 34--47.Google Scholar
- Picard, R. 1996. A society of models for video and image libraries. Tech. Rep. 360, MIT Media Laboratory Perceptual Computing.Google Scholar
- Rao, A., Srihari, R. K., and Zhang, Z. 1999. Spatial color histograms for content-based image retrieval. Proceedings of the Eleventh IEEE International Conference on Tools with Artificial Intelligence. Google Scholar
- Rao, A., Srihari, R. K., and Zhang, Z. 2000. Geometric histogram: A distribution of geometric configurations of color subsets. In Proceedings of SPIE---Internet Imaging. Vol. 3964. San Joes, California, 91--101.Google Scholar
- Rickman, R. and Stonham, J. 1996. Content-based image retrieval using color tuple histograms. SPIE proceedings: Symposium on Electronic Imaging: Science and Technology---Storage and Retrieval for Image and Video Databases IV 2670, 2--7.Google Scholar
- Russ, J. C. 1995. The Image Processing Handbook. CRC Press, Boca Raton. Google Scholar
- Safar, M., Shahabi, C., and Sun, X. 2000. Image retrieval by shape: A comparative study. In Proceedings of the IEEE International Conference on Multimedia and Exposition ICME. USA.Google Scholar
- Shahabi, C. and Safar, M. 1999. Efficient retrieval and spatial querying of 2d objects. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems (ICMCS99). Florence, Italy, 611--617. Google Scholar
- Sheikholeslami, G. 1999. Multi-resolution content-based image retrieval and clustering in large visual databases. Ph.D. dissertation, Department of Computer Science and Engineering, State University of New York at Buffalo. Google Scholar
- Sheikholeslami, G., Chatterjee, S., and Zhang, A. 1998. WaveCluster: A Multi-Resolution Clustering Approach for Very Large Spatial Databases. In Proceedings of the 24th VLDB conference. 428--439. Google Scholar
- Sheikholeslami, G., Chatterjee, S., and Zhang, A. 2000. WaveCluster: A Wavelet-Based Clustering Approach for Multidimensional Data in Very Large Databases. The VLDB Journal 8, 4 (February), 289--304. Google Scholar
- Sheikholeslami, G. and Zhang, A. 1997. An Approach to Clustering Large Visual Databases Using Wavelet Transform. In Proceedings of the SPIE Conference on Visual Data Exploration and Analysis IV. San Jose, 322--333.Google Scholar
- Smith, J. R. and Chang, S.-F. 1994. Transform Features For Texture Classification and Discrimination in Large Image Databases. In Proceedings of the IEEE International Conference on Image Processing. 407--411.Google Scholar
- Smith, J. and Chang, S.-F. 1996a. Tools and techniques for color image retrieval. SPIE proceedings 2670, 1630--1639.Google Scholar
- Smith, J. R. and Chang, S.-F. 1996b. VisualSeek: a fully automated content-based image query system. In Proceedings of ACM Multimedia 96. Boston, MA, USA, 87--98. Google Scholar
- Strang, G. and Nguyen, T. 1996. Wavelets and Filter Banks. Wellesley-Cambridge Press, Wellesley, MA.Google Scholar
- Stricker, M. and Dimai, A. 1996. Color indexing with weak spatial constraints. SPIE Proceedings 2670, 29--40.Google Scholar
- Stromberg, W. and Farr, T. 1986. A Fourier-based textural feature extraction procedure. IEEE Trans. Geosc. Rem. Sens. 24, 5, 722--732.Google Scholar
- Swain, M. and Ballard, D. 1991. Color Indexing. Int J. Comput. Vis. 7, 1, 11--32. Google Scholar
- Syeda-Mahmood, T. 1996. Finding shape similarity using a constrained non-rigid transform. In International Conference on Pattern Recognition. Google Scholar
- Tao, Y. and Grosky, W. 1999. Delaunay triangulation for image object indexing: A novel method for shape representation. In Proceedings of the Seventh SPIE Symposium on Storage and Retrieval for Image and Video Databases. San Jose, California, 631--942.Google Scholar
- Wang, J. and Acharya, R. 1998a. Efficient access to and retrieval from a shape image database. In IEEE Workshop on Content Based Access of Image and Video Libraries (CBAIL 98). Santa Barbara. Google Scholar
- Wang, J. and Acharya, R. 1998b. A vertex based shape coding approach for similar shape retrieval. In ACM Symposium on Applied Computing. Atlanta, GA, 520--524. Google Scholar
- Wang, W., Yang, J., and Muntz, R. 1997. STING: A Statistical Information Grid Approach to Spatial Data Mining. In Proceedings of the 23rd VLDB Conference. Athens, Greece, 186--195. Google Scholar
- Zhang, A. and Zhu, L. 2001. Metadata generation and retrieval of geographical imagery. In Proceedings of the National Conference for Digital Government Research (dg.o2001). Los Angeles, California, USA, 76--83.Google Scholar
- Zhang, T., Ramakrishnan, R., and Livny, M. 1996. BIRCH: An Efficient Data Clustering Method for Very Large Databases. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data. Montreal, Canada, 103--114. Google Scholar
- Zhu, L., Rao, A., and Zhang, A. 2000a. Advanced feature extraction for keyblock-based image retrieval. In Proceedings of the International Workshop on Multimedia Information Retrieval (MIR2000). Los Angeles, California, USA, 179--183. Google Scholar
- Zhu, L., Rao, A., and Zhang, A. 2000b. Keyblock: An approach for content-based geographic image retrieval. In Proceedings of the First International Conference on Geographic Information Science (GIScience 2000). Savannah, Georgia, USA, 286--287. Google Scholar
- Zhu, L. and Zhang, A. 2000. Supporting multi-example image queries in image databases. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME2000). New York City, NY, USA, 697--700.Google Scholar
- Zhu, L., Zhang, A., Rao, A., and Srihari, R. 2000. Keyblock: An approach for content-based image retrieval. In Proceedings of ACM Multimedia 2000. Los Angeles, California, USA, 157--166. Google Scholar
Index Terms
- Theory of keyblock-based image retrieval
Recommendations
Advanced feature extraction for keyblock-based image retrieval
Keyblock, which is a new framework we proposed for content-based image retrieval, is a generalization of the text-based information retrieval technology in the image domain. In this framework, keyblocks, which are analogous to keywords in text document ...
Codebook design of keyblock based image retrieval
ICEC'07: Proceedings of the 6th international conference on Entertainment ComputingThis paper presents an image retrieval method based on keyblocks combing with interest points, furthermore the generation of codebook is also utilized to enhance the retrieval performance, where the balance between the retrieved precision and time cost ...
Image retrieval based on bag of images
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingConventional relevance feedback schemes may not be suitable to all practical applications of content-based image retrieval (CBIR), since most ordinary users would like to complete their search in a single interaction, especially on the web search. In ...
Reviews
Donald Harris Kraft
This research is very interesting, applying a paradigm of text retrieval to two-dimensional content-based image retrieval. The authors use the construct of keyblocks, based on a codebook approach. This strategy is innovative, and the authors get good results. The framework of the paradigm includes the generation of codebooks, where images are encoded, features are extracted, and codebooks are then constructed via clustering. This work is based on some previous work, involving compression via vector quantization, and a code vector histogram as an image feature, which is seen as analogous to aspects of text retrieval. The authors present two clustering algorithms, and eventually recommend a hybrid based on both. Moreover, they consider the inclusion of a knowledge base for applications where domain knowledge is present. The authors also look at a vector (space) model and a Boolean model of the image features for their keyblock representations. They view a query as an image, too. One serious contribution is their use of models for context-sensitive information, analogous to n -grams in text retrieval. The authors present uni-block, bi-block, and tri-block models, as well as a feature combination model. Another delight is the authors’ testing of their approach with a variety of experiments on small and larger databases of images, controlling for a variety of factors, such as block size. They find a relationship between retrieval performance and average distortion. Their approach has merit and deserves serious consideration. There are two minor drawbacks to this paper. One is that it is not an easy read for those not familiar with image retrieval issues. The second is that the authors are slightly naive in terms of the terminology of text retrieval (referring to the vector space model as the vector model, and not mentioning the use of stopword lists and stemming for keywords in text retrieval). However, this in no way negates the contributions of the paper. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments