Image registration methods: a survey
Introduction
Image registration is the process of overlaying two or more images of the same scene taken at different times, from different viewpoints, and/or by different sensors. It geometrically aligns two images—the reference and sensed images. The present differences between images are introduced due to different imaging conditions. Image registration is a crucial step in all image analysis tasks in which the final information is gained from the combination of various data sources like in image fusion, change detection, and multichannel image restoration. Typically, registration is required in remote sensing (multispectral classification, environmental monitoring, change detection, image mosaicing, weather forecasting, creating super-resolution images, integrating information into geographic information systems (GIS)), in medicine (combining computer tomography (CT) and NMR data to obtain more complete information about the patient, monitoring tumor growth, treatment verification, comparison of the patient's data with anatomical atlases), in cartography (map updating), and in computer vision (target localization, automatic quality control), to name a few.
During the last decades, image acquisition devices have undergone rapid development and growing amount and diversity of obtained images invoked the research on automatic image registration. A comprehensive survey of image registration methods was published in 1992 by Brown [26]. The intention of our article is to cover relevant approaches introduced later and in this way map the current development of registration techniques. According to the database of the Institute of Scientific Information (ISI), in the last 10 years more than 1000 papers were published on the topic of image registration. Methods published before 1992 that became classic or introduced key ideas, which are still in use, are included as well to retain the continuity and to give complete view of image registration research. We do not contemplate to go into details of particular algorithms or describe results of comparative experiments, rather we want to summarize main approaches and point out interesting parts of the registration methods.
In Section 2 various aspects and problems of image registration will be discussed. Both area-based and feature-based approaches to feature selection are described in Section 3. Section 4 reviews the existing algorithms for feature matching. Methods for mapping function design are given in Section 5. Finally, Section 6 surveys main techniques for image transformation and resampling. Evaluation of the image registration accuracy is covered in Section 7. Section 8 concludes main trends in the research on registration methods and offers the outlook for the future.
Section snippets
Image registration methodology
Image registration, as it was mentioned above, is widely used in remote sensing, medical imaging, computer vision etc. In general, its applications can be divided into four main groups according to the manner of the image acquisition:
Different viewpoints (multiview analysis). Images of the same scene are acquired from different viewpoints. The aim is to gain larger a 2D view or a 3D representation of the scanned scene.
Examples of applications: Remote sensing—mosaicing of images of the surveyed
Feature detection
Formerly, the features were objects manually selected by an expert. During an automation of this registration step, two main approaches to feature understanding have been formed.
Feature matching
The detected features in the reference and sensed images can be matched by means of the image intensity values in their close neighborhoods, the feature spatial distribution, or the feature symbolic description. Some methods, while looking for the feature correspondence, simultaneously estimate the parameters of mapping functions and thus merge the second and third registration steps.
In the following paragraphs, the two major categories (area-based and feature-based methods, respectively), are
Transform model estimation
After the feature correspondence has been established the mapping function is constructed. It should transform the sensed image to overlay it over the reference one. The correspondence of the CPs from the sensed and reference images together with the fact that the corresponding CP pairs should be as close as possible after the sensed image transformation are employed in the mapping function design.
The task to be solved consists of choosing the type of the mapping function (see Fig. 5) and its
Image resampling and transformation
The mapping functions constructed during the previous step are used to transform the sensed image and thus to register the images. The transformation can be realized in a forward or backward manner. Each pixel from the sensed image can be directly transformed using the estimated mapping functions. This approach, called a forward method, is complicated to implement, as it can produce holes and/or overlaps in the output image (due to the discretization and rounding). Hence, the backward approach
Evaluation of the image registration accuracy
Regardless of the particular images, the used registration method, and the application area, it is highly desirable to provide the user with an estimate how accurate the registration actually is. The accuracy evaluation is a non-trivial problem, partially because the errors can be dragged into the registration process in each of its stages and partially because it is hard to distinguish between registration inaccuracies and actual physical differences in the image contents. In this Section, we
Current trends and outlook for the future
Image registration is one of the most important tasks when integrating and analyzing information from various sources. It is a key stage in image fusion, change detection, super-resolution imaging, and in building image information systems, among others. This paper gives a survey of the classical and up-to-date registration methods, classifying them according to their nature as well as according to the four major registration steps. Although a lot of work has been done, automatic image
Acknowledgements
This work has been supported by the grant No. 102/01/P065 of the Grant Agency of the Czech Republic.
References (224)
- et al.
Virtual circles: a new set of features for fast image registration
Pattern Recognition Letters
(2003) - et al.
Image warping by radial basis functions: application to facial expressions
CVGIP: Graphical Models and Image Processing
(1994) - et al.
An algorithmic overview of surface registration techniques for medical imaging
Medical image Analysis
(2000) - et al.
Multiresolution elastic matching
Computer Vision, Graphics and Image Processing
(1989) - et al.
Point landmarks for registration of CT and NMR images
Pattern Recognition Letters
(1995) - et al.
Warping digital images using thin plate splines
Pattern Recognition
(1993) - et al.
Fast evaluation of radial basis functions
Computers Mathematical Applications
(1992) - et al.
An invariant approach for image registration in digital subtraction angiography
Pattern Recognition
(2002) - et al.
Invariance of stereo images via theory of complex moments
Pattern Recognition
(1997) - et al.
Fast algorithm for point pattern matching: Invariant to translations, rotations and scale changes
Pattern Recognition
(1997)
Point pattern matching algorithm invariant to geometrical transformation and distortion
Pattern Recognition Letters
Image registration by matching relational structures
Pattern Recognition
Automated assembling of images: Image montage preparation
Pattern Recognition
Volume image registration by template matching
Image and Vision Computing
An adaptive method for image registration
Pattern Recognition
Object matching by means of matching likelihood coefficients
Pattern Recognition Letters
Pattern recognition by affine moment invariants
Pattern Recognition
Radial basis functions with compact support for elastic registration of medical images
Image and Vision Computing
Piecewise linear mapping functions for image registration
Pattern Recognition
Piecewise cubic mapping functions for image registration
Pattern Recognition
Image registration by local approximation methods
Image and Vision Computing
A fast algorithm for particle simulations
Journal of Computers and Physics
Results of test on image matching of ISPRS WG
ISPRS Journal of Photogrammetry and Remote Sensing
Image registration using a new edge-based approach
Computer Vision and Image Understanding
A global optimisation method for robust affine registration of brain images
Medical Image Analysis
Robust image registration by increment sign correlation
Pattern Recognition
Using selective correlation coefficient for robust image registration
Pattern Recognition
Registering coronal histological 2-D sections of a rat brain with coronal sections of a 3-D brain atlas using geometric curve invariants and B-spline representation
IEEE Transactions on Medical Imaging
A rapid and automatic image registration algorithm with subpixel accuracy
IEEE Transactions on Medical Imaging
Non-rigid registration by geometry-constrained diffusion
Medical Image Analysis
Spatial registration of multispectral and multitemporal digital imagery using Fast Fourier Transform
IEEE Transactions on Geoscience Electronics
A new approach to the interpolation of sampled data
IEEE Transactions on Medical Imaging
A class of algorithms for fast digital image registration
IEEE Transactions on Computing
A method for registration of 3D shapes
IEEE Transactions on Pattern Analysis and Machine Intellinegce
The computation of optical flow
ACM Computing Surveys
Principal warps: Thin-plate splines and the decomposition of deformations
IEEE Transactions on Pattern Analysis and Machine Intelligence
Hierarchical chamfer matching: a parametric edge matching algorithm
IEEE Transactions on Pattern Analysis and Machine Intelligence
The Fourier Transform and Its Applications
Automatic selection of control points from shadow structures
International Journal of Remote Sensing
A survey of image registration techniques
ACM Computing Surveys
Projection-based image registration in the presence of fixed-pattern noise
IEEE Transactions on Image Processing
A computational approach to edge detection
IEEE Transactions on Pattern Analysis and Machine Intelligence
Registration of translated and rotated images using finite Fourier transform
IEEE Transactions on Pattern Analysis and Machine Intelligence
Symmetric phase-only matched filtering of Fourier–Mellin transform for image registration and recognition
IEEE Transactions on Pattern Analysis and Machine Intellingence
Registration of ocular fundus images
IEEE Engineering in Medicine and Biology
A feature-based image registration algorithm using improved chain-code representation combined with invariant moments
IEEE Transactions on Geoscience and Remote Sensing
Cited by (5867)
Shared contents alignment across multiple granularities for robust SAR-optical image matching
2024, Information FusionGLS–MIFT: A modality invariant feature transform with global-to-local searching
2024, Information FusionImproving RGB-infrared object detection with cascade alignment-guided transformer
2024, Information FusionFeature matching based on Gaussian kernel convolution and minimum relative motion
2024, Engineering Applications of Artificial IntelligenceDeformable 3D/3D CT-to-digital-tomosynthesis image registration in image-guided bronchoscopy interventions
2024, Computers in Biology and Medicine