Carsten Griwodz
ABSTRACT
This paper introduces the Meshroom software and its underlying 3D computer vision framework AliceVision. This solution provides a photogrammetry pipeline to reconstruct 3D scenes from a set of unordered images. It also features other pipelines for fusing multi-bracketing low dynamic range images into high dynamic range, stitching multiple images into a panorama and estimating the motion of a moving camera. Meshroom's node-graph architecture allows the user to customize the different pipelines to adjust them to their domain specific needs. The user can interactively add other processing nodes to modify a pipeline, export intermediate data to analyze the result of the algorithms and easily compare the outputs given by different sets of parameters. The software package is released in open source and relies on open file formats. These features enable researchers to conveniently run the pipelines, access and visualize the data at each step, thus promoting the sharing and the reproducibility of the results.
Supplemental Material
Available for Download
zip
p241-griwodz.zip (10 MB)
Supplemental material.
- Jong-Beom Jeong, Soonbin Lee, Il-Woong Ryu, Tuan Thanh Le, and Eun-Seok Ryu. Towards Viewport-dependent 6DoF 360 Video Tiled Streaming for Virtual Reality Systems. In Proc. 28th ACM Int. Conf. Multimed., pages 3687--3695, New York, NY, USA, oct 2020. ACM.Google Scholar
Digital Library
- Savino Dambra, Giuseppe Samela, Lucile Sassatelli, Romaric Pighetti, Ramon Aparicio-Pardo, and Anne-Marie Pinna-Déry. Film editing. In Proc. 9th ACM Multimed. Syst. Conf., pages 27--39, New York, NY, USA, jun 2018. ACM.Google ScholarDigital Library
- Mohammad Hosseini and Christian Timmerer. Dynamic Adaptive Point Cloud Streaming. In Proc. 23rd Pack. Video Work., pages 25--30, New York, NY, USA, jun 2018. ACM.Google Scholar
- Suraj Raghuraman, Kanchan Bahirat, and Balakrishnan Prabhakaran. A Visual Latency Estimator for 3D Tele-Immersion. In Proc. 8th ACM Multimed. Syst. Conf., pages 272--283, New York, NY, USA, jun 2017. ACM.Google ScholarDigital Library
- Cédric Portaneri, Pierre Alliez, Michael Hemmer, Lukas Birklein, and Elmar Schoemer. Cost-driven framework for progressive compression of textured meshes. In Proc. 10th ACM Multimed. Syst. Conf., pages 175--188, New York, NY, USA, jun 2019. ACM.Google ScholarDigital Library
- Pierre Moulon. Positionnement robuste et précis de réseaux d'images. PhD thesis, Laboratoire d'Informatique Gaspard-Monge, École des Ponts ParisTech, 2014.Google Scholar
- Michal Jančošek. Large Scale Surface Reconstruction based on Point Visibility. PhD thesis, Czech Technical University in Prague, 2015.Google Scholar
- Lilian Calvet. Méthodes de reconstruction tridimensionnelle intégrant des points cycliques : application au suivi d'une caméra. PhD thesis, Institut National Polytechnique de Toulouse, 2014.Google Scholar
- Carsten Griwodz, Lilian Calvet, and Pål Halvorsen. Popsift. In Proc. 9th ACM Multimed. Syst. Conf., pages 415--420, New York, NY, USA, jun 2018. ACM.Google Scholar
- Toby Collins, Daniel Pizarro, Simone Gasparini, Nicolas Bourdel, Pauline Chauvet, Michel Canis, Lilian Calvet, and Adrien Bartoli. Augmented Reality Guided Laparoscopic Surgery of the Uterus. IEEE Transactions on Medical Imaging, 40(1):371--380, January 2021.Google ScholarCross Ref
- Asla Medeiros e Sá, Adolfo Bartolome Ibañez Vila, Karina Rodriguez Echavarria, Ricardo Marroquim, and Vivian Luiz Fonseca. Accessible Digitisation and Visualisation of Open Cultural Heritage Assets. In Selma Rizvic and Karina Rodriguez Echavarria, editors, Eurographics Workshop on Graphics and Cultural Heritage. The Eurographics Association, 2019.Google Scholar
- J. Milàn, P. Falkingham, and Inken Juliane Mueller-Töwe. Small ornithopod dinosaur tracks and crocodilian remains from the middle jurassic bagå formation, bornholm, denmark: Important additions to the rare middle jurassic vertebrate faunas of northern europe. Bulletin of The Geological Society of Denmark, 68:245--253, 2020.Google ScholarCross Ref
- Jens Lallensack, Michael Buchwitz, and Anthony Romilio. Photogrammetry in ichnology: 3d model generation, visualisation, and data extraction. November 2020.Google Scholar
- Shah Ariful Hoque Chowdhury, Chuong Nguyen, Hengjia Li, and Richard Hartley. Fixed-lens camera setup and calibrated image registration for multifocus multiview 3d reconstruction. Neural Computing and Applications, April 2021.Google ScholarDigital Library
- Marco Wallner, Daniel Steininger, Verena Widhalm, Matthias Schörghuber, and Csaba Beleznai. RGB-d railway platform monitoring and scene understanding for enhanced passenger safety. In Pattern Recognition. ICPR International Workshops and Challenges, pages 656--671. Springer International Publishing, 2021.Google ScholarDigital Library
- Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-Motion Revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.Google Scholar
- Ewelina Rupnik, Mehdi Daakir, and Marc Pierrot Deseilligny. MicMac -- a free, open-source solution for photogrammetry. Open Geospatial Data, Softw. Stand., 2(1):14, dec 2017.Google Scholar
- Simon Fuhrmann, Fabian Langguth, and Michael Goesele. MVE - A Multi-View Reconstruction Environment. In Reinhard Klein and Pedro Santos, editors, Eurographics Workshop on Graphics and Cultural Heritage. The Eurographics Association, 2014.Google Scholar
- OpenDroneMap Authors. Odm - a command line toolkit to generate maps, point clouds, 3d models and dems from drone, balloon or kite images.Google Scholar
- Regard3D Authors. Regard3d.Google Scholar
- C. Strecha, W. von Hansen, L. Van Gool, P. Fua, and U. Thoennessen. On benchmarking camera calibration and multi-view stereo for high resolution imagery. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, June 2008.Google ScholarCross Ref
- David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vis., 60(2):91--110, nov 2004.Google ScholarDigital Library
- Jingming Dong and Stefano Soatto. Domain-size pooling in local descriptors: DSP-SIFT. In 2015 IEEE Conf. Comput. Vis. Pattern Recognit., pages 5097--5106. IEEE, jun 2015.Google ScholarCross Ref
- P. F. Alcantarilla, J. Nuevo, and A. Bartoli. Fast explicit diffusion for accelerated features in nonlinear scale spaces. In British Machine Vision Conf. (BMVC), 2013.Google ScholarCross Ref
- Lilian Calvet, Pierre Gurdjos, Carsten Griwodz, and Simone Gasparini. Detection and Accurate Localization of Circular Fiducials under Highly Challenging Conditions. In 2016 IEEE Conf. Comput. Vis. Pattern Recognit., pages 562--570. IEEE, jun 2016.Google ScholarCross Ref
- Edwin Olson. AprilTag: A robust and flexible visual fiducial system. In 2011 IEEE Int. Conf. Robot. Autom., pages 3400--3407. IEEE, may 2011.Google ScholarCross Ref
- D. Nister and H. Stewenius. Scalable Recognition with a Vocabulary Tree. In 2006 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. - Vol. 2, volume 2, pages 2161--2168. IEEE.Google Scholar
- Jian Cheng, Cong Leng, Jiaxiang Wu, Hainan Cui, and Hanqing Lu. Fast and Accurate Image Matching with Cascade Hashing for 3D Reconstruction. In 2014 IEEE Conf. Comput. Vis. Pattern Recognit., pages 1--8. IEEE, jun 2014.Google ScholarDigital Library
- Marius Muja and David G Lowe. Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration. In Alpesh Ranchordas and Helder Araújo, editors, {VISAPP} 2009 - Proc. Fourth Int. Conf. Comput. Vis. Theory Appl. Lisboa, Port. Febr. 5-8, 2009 - Vol. 1, pages 331--340. {INSTICC} Press, 2009.Google Scholar
- M. Slaney and M. Casey. Locality-Sensitive Hashing for Finding Nearest Neighbors [Lecture Notes]. IEEE Signal Process. Mag., 25(2):128--131, mar 2008.Google ScholarCross Ref
- Martin A Fischler and Robert C Bolles. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. Assoc. Comput. Mach., 24(6):381--395, jun 1981.Google ScholarDigital Library
- Changchang Wu. Towards Linear-Time Incremental Structure from Motion. In 2013 Int. Conf. 3D Vis., pages 127--134. IEEE, jun 2013.Google ScholarDigital Library
- Pierre Moulon, Pascal Monasse, and Renaud Marlet. Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion. In 2013 IEEE Int. Conf. Comput. Vis., pages 3248--3255. IEEE, dec 2013.Google ScholarDigital Library
- Michal Havlena, Akihiko Torii, and Tomáš Pajdla. Efficient Structure from Motion by Graph Optimization. pages 100--113. 2010.Google Scholar
- Roberto Toldo, Riccardo Gherardi, Michela Farenzena, and Andrea Fusiello. Hierarchical structure-and-motion recovery from uncalibrated images. Comput. Vis. Image Underst., 140:127--143, nov 2015.Google ScholarDigital Library
- Rajvi Shah, Aditya Deshpande, and P. J. Narayanan. Multistage SFM: Revisiting Incremental Structure from Motion. In 2014 2nd Int. Conf. 3D Vis., pages 417--424. IEEE, dec 2014.Google ScholarDigital Library
- H. Hirschmuller. Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Trans. Pattern Anal. Mach. Intell., 30(2):328--341, feb 2008.Google ScholarDigital Library
- Xing Mei, Xun Sun, Mingcai Zhou, Shaohui Jiao, Haitao Wang, and Xiaopeng Zhang. On building an accurate stereo matching system on graphics hardware. In 2011 IEEE Int. Conf. Comput. Vis. Work. (ICCV Work., pages 467--474. IEEE, nov 2011.Google ScholarCross Ref
- Jose Luis Blanco and Pranjal Kumar Rai. nanoflann: a C++ header-only fork of FLANN, a library for nearest neighbor (NN) with kd-trees. https://github.com/jlblancoc/nanoflann, 2014.Google Scholar
- Bruno Lévy and Alain Filbois. Geogram: a library for geometric algorithms. In Int. Conf. Adapt. Model. Simul., Nantes, France, 2015. CNIME.Google Scholar
- Michal Jancosek and Tomas Pajdla. Multi-view reconstruction preserving weakly-supported surfaces. In CVPR 2011, pages 3121--3128. IEEE, jun 2011.Google ScholarDigital Library
- Michal Jancosek and Tomas Pajdla. Exploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces. Int. Sch. Res. Not., 2014:1--20, aug 2014.Google Scholar
- Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124--1137, sep 2004.Google ScholarDigital Library
- Bruno Lévy, Sylvain Petitjean, Nicolas Ray, and Jérome Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph., 21(3):362--371, jul 2002.Google ScholarDigital Library
- Peter J. Burt and Edward H. Adelson. A multiresolution spline with application to image mosaics. ACM Trans. Graph., 2(4):217--236, oct 1983.Google ScholarDigital Library
- Adam Baumberg. Blending Images for Texturing 3D Models. Proc. Br. Mach. Vis. Conf., 2003.Google Scholar
- Cedric Allene, Jean-Philippe Pons, and Renaud Keriven. Seamless image-based texture atlases using multi-band blending. In 2008 19th Int. Conf. Pattern Recognit., pages 1--4. IEEE, dec 2008.Google ScholarCross Ref
- Simone Gasparini, Fabien Castan, and Yann Lanthony. Buddha dataset, January 2017. https://github.com/alicevision/datasetbuddha.Google Scholar
- Richard dataset. pi3dscan. https://www.pi3dscan.com/index.php/download/item/example-model-richard.Google Scholar
- Paul E. Debevec and Jitendra Malik. Recovering high dynamic range radiance maps from photographs. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH 97. ACM Press, 1997.Google Scholar
- M.D. Grossberg and S.K. Nayar. Modeling the space of camera response functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10):1272--1282, October 2004.Google ScholarDigital Library
- Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. "GrabCut". ACM Transactions on Graphics, 23(3):309--314, August 2004.Google ScholarDigital Library
- Matthew Brown and David G. Lowe. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1):59--73, December 2006.Google ScholarDigital Library
Index Terms
- AliceVision Meshroom: An open-source 3D reconstruction pipeline
Recommendations
Motion estimation during photographing towards minimal solution for mobile panoramic images
Panoramic images, which are often produced off-line a while after the photos are taken, might be an entertaining application if they were generated on mobile devices immediately. Since producing a panorama requires feature extraction and matching that ...
Omnivergent Stereo
The notion of a virtual camera for optimal 3D reconstruction is introduced. Instead of planar perspective images that collect many rays at a fixed viewpoint, omnivergent cameras collect a small number of rays at many different viewpoints. The resulting ...
Camera Calibration by Vanishing Lines for 3-D Computer Vision
A novel approach to camera calibration by vanishing lines is proposed. Calibrated parameters include the orientation, position, and focal length of a camera. A hexagon is used as the calibration target to generate a vanishing line of the ground plane ...
Comments