Elsevier

Image and Vision Computing

Volume 21, Issue 11, October 2003, Pages 977-1000
Image and Vision Computing

Image registration methods: a survey

https://doi.org/10.1016/S0262-8856(03)00137-9Get rights and content

Abstract

This paper aims to present a review of recent as well as classic image registration methods. Image registration is the process of overlaying images (two or more) of the same scene taken at different times, from different viewpoints, and/or by different sensors. The registration geometrically align two images (the reference and sensed images). The reviewed approaches are classified according to their nature (area-based and feature-based) and according to four basic steps of image registration procedure: feature detection, feature matching, mapping function design, and image transformation and resampling. Main contributions, advantages, and drawbacks of the methods are mentioned in the paper. Problematic issues of image registration and outlook for the future research are discussed too. The major goal of the paper is to provide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas.

Introduction

Image registration is the process of overlaying two or more images of the same scene taken at different times, from different viewpoints, and/or by different sensors. It geometrically aligns two images—the reference and sensed images. The present differences between images are introduced due to different imaging conditions. Image registration is a crucial step in all image analysis tasks in which the final information is gained from the combination of various data sources like in image fusion, change detection, and multichannel image restoration. Typically, registration is required in remote sensing (multispectral classification, environmental monitoring, change detection, image mosaicing, weather forecasting, creating super-resolution images, integrating information into geographic information systems (GIS)), in medicine (combining computer tomography (CT) and NMR data to obtain more complete information about the patient, monitoring tumor growth, treatment verification, comparison of the patient's data with anatomical atlases), in cartography (map updating), and in computer vision (target localization, automatic quality control), to name a few.

During the last decades, image acquisition devices have undergone rapid development and growing amount and diversity of obtained images invoked the research on automatic image registration. A comprehensive survey of image registration methods was published in 1992 by Brown [26]. The intention of our article is to cover relevant approaches introduced later and in this way map the current development of registration techniques. According to the database of the Institute of Scientific Information (ISI), in the last 10 years more than 1000 papers were published on the topic of image registration. Methods published before 1992 that became classic or introduced key ideas, which are still in use, are included as well to retain the continuity and to give complete view of image registration research. We do not contemplate to go into details of particular algorithms or describe results of comparative experiments, rather we want to summarize main approaches and point out interesting parts of the registration methods.

In Section 2 various aspects and problems of image registration will be discussed. Both area-based and feature-based approaches to feature selection are described in Section 3. Section 4 reviews the existing algorithms for feature matching. Methods for mapping function design are given in Section 5. Finally, Section 6 surveys main techniques for image transformation and resampling. Evaluation of the image registration accuracy is covered in Section 7. Section 8 concludes main trends in the research on registration methods and offers the outlook for the future.

Section snippets

Image registration methodology

Image registration, as it was mentioned above, is widely used in remote sensing, medical imaging, computer vision etc. In general, its applications can be divided into four main groups according to the manner of the image acquisition:

Different viewpoints (multiview analysis). Images of the same scene are acquired from different viewpoints. The aim is to gain larger a 2D view or a 3D representation of the scanned scene.

Examples of applications: Remote sensing—mosaicing of images of the surveyed

Feature detection

Formerly, the features were objects manually selected by an expert. During an automation of this registration step, two main approaches to feature understanding have been formed.

Feature matching

The detected features in the reference and sensed images can be matched by means of the image intensity values in their close neighborhoods, the feature spatial distribution, or the feature symbolic description. Some methods, while looking for the feature correspondence, simultaneously estimate the parameters of mapping functions and thus merge the second and third registration steps.

In the following paragraphs, the two major categories (area-based and feature-based methods, respectively), are

Transform model estimation

After the feature correspondence has been established the mapping function is constructed. It should transform the sensed image to overlay it over the reference one. The correspondence of the CPs from the sensed and reference images together with the fact that the corresponding CP pairs should be as close as possible after the sensed image transformation are employed in the mapping function design.

The task to be solved consists of choosing the type of the mapping function (see Fig. 5) and its

Image resampling and transformation

The mapping functions constructed during the previous step are used to transform the sensed image and thus to register the images. The transformation can be realized in a forward or backward manner. Each pixel from the sensed image can be directly transformed using the estimated mapping functions. This approach, called a forward method, is complicated to implement, as it can produce holes and/or overlaps in the output image (due to the discretization and rounding). Hence, the backward approach

Evaluation of the image registration accuracy

Regardless of the particular images, the used registration method, and the application area, it is highly desirable to provide the user with an estimate how accurate the registration actually is. The accuracy evaluation is a non-trivial problem, partially because the errors can be dragged into the registration process in each of its stages and partially because it is hard to distinguish between registration inaccuracies and actual physical differences in the image contents. In this Section, we

Current trends and outlook for the future

Image registration is one of the most important tasks when integrating and analyzing information from various sources. It is a key stage in image fusion, change detection, super-resolution imaging, and in building image information systems, among others. This paper gives a survey of the classical and up-to-date registration methods, classifying them according to their nature as well as according to the four major registration steps. Although a lot of work has been done, automatic image

Acknowledgements

This work has been supported by the grant No. 102/01/P065 of the Grant Agency of the Czech Republic.

References (224)

  • F.H. Cheng

    Point pattern matching algorithm invariant to geometrical transformation and distortion

    Pattern Recognition Letters

    (1996)
  • J.K. Cheng et al.

    Image registration by matching relational structures

    Pattern Recognition

    (1984)
  • P. Dani et al.

    Automated assembling of images: Image montage preparation

    Pattern Recognition

    (1995)
  • L. Ding et al.

    Volume image registration by template matching

    Image and Vision Computing

    (2001)
  • J. Flusser

    An adaptive method for image registration

    Pattern Recognition

    (1992)
  • J. Flusser

    Object matching by means of matching likelihood coefficients

    Pattern Recognition Letters

    (1995)
  • J. Flusser et al.

    Pattern recognition by affine moment invariants

    Pattern Recognition

    (1993)
  • M. Fornefett et al.

    Radial basis functions with compact support for elastic registration of medical images

    Image and Vision Computing

    (2001)
  • A. Goshtasby

    Piecewise linear mapping functions for image registration

    Pattern Recognition

    (1986)
  • A. Goshtasby

    Piecewise cubic mapping functions for image registration

    Pattern Recognition

    (1987)
  • A. Goshtasby

    Image registration by local approximation methods

    Image and Vision Computing

    (1988)
  • L. Greengard et al.

    A fast algorithm for particle simulations

    Journal of Computers and Physics

    (1987)
  • E. Gülch

    Results of test on image matching of ISPRS WG

    ISPRS Journal of Photogrammetry and Remote Sensing

    (1991)
  • J.W. Hsieh et al.

    Image registration using a new edge-based approach

    Computer Vision and Image Understanding

    (1997)
  • M. Jenkinson et al.

    A global optimisation method for robust affine registration of brain images

    Medical Image Analysis

    (2001)
  • S. Kaneko et al.

    Robust image registration by increment sign correlation

    Pattern Recognition

    (2002)
  • S. Kaneko et al.

    Using selective correlation coefficient for robust image registration

    Pattern Recognition

    (2003)
  • S. Abdelsayed, D. Ionescu, D. Goodenough, Matching and registration method for remote sensing images, Proceedings of...
  • W.S.I. Ali et al.

    Registering coronal histological 2-D sections of a rat brain with coronal sections of a 3-D brain atlas using geometric curve invariants and B-spline representation

    IEEE Transactions on Medical Imaging

    (1998)
  • R.J. Althof et al.

    A rapid and automatic image registration algorithm with subpixel accuracy

    IEEE Transactions on Medical Imaging

    (1997)
  • P.R. Andersen et al.

    Non-rigid registration by geometry-constrained diffusion

    Medical Image Analysis

    (2001)
  • P.E. Anuta

    Spatial registration of multispectral and multitemporal digital imagery using Fast Fourier Transform

    IEEE Transactions on Geoscience Electronics

    (1970)
  • C.R. Appledorn

    A new approach to the interpolation of sampled data

    IEEE Transactions on Medical Imaging

    (1996)
  • D.I. Barnea et al.

    A class of algorithms for fast digital image registration

    IEEE Transactions on Computing

    (1972)
  • H.G. Barrow, J.M. Tenenbaum, R.C. Bolles, H.C. Wolf., Parametric correspondence and chamfer matching: Two new...
  • R. Berthilsson, Affine correlation. Proceedings of the International Conference on Pattern Recognition ICPR'98,...
  • P.J. Besl et al.

    A method for registration of 3D shapes

    IEEE Transactions on Pattern Analysis and Machine Intellinegce

    (1992)
  • S.S. Beuchemin et al.

    The computation of optical flow

    ACM Computing Surveys

    (1995)
  • F.L. Bookstein

    Principal warps: Thin-plate splines and the decomposition of deformations

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1989)
  • G. Borgefors

    Hierarchical chamfer matching: a parametric edge matching algorithm

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1988)
  • R.N. Bracewell

    The Fourier Transform and Its Applications

    (1965)
  • P.A. Brivio et al.

    Automatic selection of control points from shadow structures

    International Journal of Remote Sensing

    (1992)
  • M. Bro-Nielsen, C. Gramkow, Fast fluid registration of medical images, In Proceedings Visualization in Biomedical...
  • L.G. Brown

    A survey of image registration techniques

    ACM Computing Surveys

    (1992)
  • S.C. Cain et al.

    Projection-based image registration in the presence of fixed-pattern noise

    IEEE Transactions on Image Processing

    (2001)
  • J. Canny

    A computational approach to edge detection

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1986)
  • E.D. Castro et al.

    Registration of translated and rotated images using finite Fourier transform

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1987)
  • Q. Chen et al.

    Symmetric phase-only matched filtering of Fourier–Mellin transform for image registration and recognition

    IEEE Transactions on Pattern Analysis and Machine Intellingence

    (1994)
  • A.V. Cideciyan

    Registration of ocular fundus images

    IEEE Engineering in Medicine and Biology

    (1995)
  • X. Dai et al.

    A feature-based image registration algorithm using improved chain-code representation combined with invariant moments

    IEEE Transactions on Geoscience and Remote Sensing

    (1999)
  • Cited by (5867)

    • Feature matching based on Gaussian kernel convolution and minimum relative motion

      2024, Engineering Applications of Artificial Intelligence
    View all citing articles on Scopus
    View full text