Automatic Camera Localization, Reconstruction and Segmentation of Multi-planar Scenes Using Two Views

Abstract

This work addresses two main problems: (i) localization of two cameras observing a 3D scene composed by planar structures; (ii) recovering of the original structure of the scene, i.e. the scene reconstruction and segmentation stages. Although there exist some work intending to deal with these problems, most of them are based on: epipolar geometry, non-linear optimization, or linear systems that do not incorporate geometrical consistency.

In this paper, we propose an iterative linear algorithm exploiting geometrical and algebraic constraints induced by rigidity and planarity in the scene. Instead of solving a complex multi-linear problem, we solve iteratively several linear problems: coplanar features segmentation, planar projective transferring, epipole computation, and all plane intersections. Linear methods allow our approach to be suitable for real-time localization and 3D reconstruction. Furthermore, our approach does not compute the fundamental matrix; therefore it does not face stability problems commonly associated with explicit epipolar geometry computation.