Scene representation and view synthesis in image-based rendering

Abstract

Image-Based Rendering is a technology that can be used to develop systems for navigation in real-image-based virtual environments, potentially providing a high-quality realistic experience. Scene representation and view synthesis are major research topics in this field and form the main focus of this thesis. Three major topics investigated in detail are high-quality panorama generation, view interpolation and a simplified Concentric Mosaics technique. First, a novel optimization model is proposed for registration of the overlap-area between two adjacent images taken by a camera mounted and rotated on a tripod, and a new algorithm is given to significantly reduce the accumulated errors when stitching multiple images to generate 360° panoramas. The algorithms have been implemented based on matching features to significantly reduce the computations. Second, a matching-feature-based view interpolation algorithm is proposed and the triangulation of the images, combined with an affine transformation model, has been applied for the texture mapping. In addition, a novel disparity estimation algorithm is studied for view interpolation. Special transformations determined from the physical imaging conditions to minimize the difference between two source views are used in the algorithm. Finally, a simplified implementation of the Concentric Mosaics technique is proposed. The camera positions, where the pre-captured images are taken, are estimated from the pre-captured images. The mathematical equations and the optimal solution of such large scale linear equations with noisy coefficients are given, together with a set of associated methods such as a closed-loop constraint, a ratio fitting technique, and an angle grouping method to improve the estimation precision. Simulation results demonstrate that each proposed view synthesis method can generate valid views of good quality and satisfy Image-Based Rendering application requirements.