Hybrid DDCT-PCA based multi sensor image fusion

Abstract

Multi sensor image fusion algorithm based on directional Discrete Cosine Transform (DDCT) - Principal Component Analysis (PCA) hybrid technique has been developed and evaluated. The input images were divided into non-overlapping square blocks and the fusion process was carried out on the corresponding blocks. The algorithm works in two stages. In first stage, modes 0 to 8 were performed on images to be fused. For each mode, the coefficients from the images to be fused are used in the fusion process. The same procedure is repeated for other modes. Three different fusion rules are used in fusion process viz., 1. Averaging the corresponding coefficients (DDCTav), 2. Choosing the corresponding frequency band with maximum energy (DDCTek) and 3. Choosing the corresponding coefficient with maximum absolute value (DDCTmx) between the images. After this stage, there are eight fused images, one from each mode. In second stage, these eight fused images are fused using PCA. Performance of these algorithms were compared using fusion quality evaluation metrics such as root mean square error (RMSE), quality index (QI), spatial frequency and fusion quality index (FQI). It was concluded from the results that DDCTav performs poor and DDCTek performs slightly better than DDCTmx. Moreover, DDCTek is computationally simple and easily implementable on target hardware. Matlab code has been provided for better understanding.