Seung-Hwan Baek
Min H. Kim
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a novel, compact single-shot hyperspectral imaging method. It enables capturing hyperspectral images using a conventional DSLR camera equipped with just an ordinary refractive prism in front of the camera lens. Our computational imaging method reconstructs the full spectral information of a scene from dispersion over edges. Our setup requires no coded aperture mask, no slit, and no collimating optics, which are necessary for traditional hyperspectral imaging systems. It is thus very cost-effective, while still highly accurate. We tackle two main problems: First, since we do not rely on collimation, the sensor records a projection of the dispersion information, distorted by perspective. Second, available spectral cues are sparse, present only around object edges. We formulate an image formation model that can predict the perspective projection of dispersion, and a reconstruction method that can estimate the full spectral information of a scene from sparse dispersion information. Our results show that our method compares well with other state-of-the-art hyperspectral imaging systems, both in terms of spectral accuracy and spatial resolution, while being orders of magnitude cheaper than commercial imaging systems.
Supplemental Material
Available for Download
zip
a217-baek.zip (31.4 MB)
Supplemental material.
- Manya V. Afonso, José M. Bioucas-Dias, and Mário A. T. Figueiredo. 2011. An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems. IEEE Trans. Image Processing 20, 3 (2011), 681--695. Google Scholar
Digital Library
- Sara Alvarez, Timo Kunkel, and Belen Masia. 2016. Practical Low-Cost Recovery of Spectral Power Distributions. Computer Graphics Forum 35, 1 (2016), 166--178. Google ScholarDigital Library
- Seung-Hwan Baek, Diego Gutierrez, and Min H Kim. 2016. Birefractive stereo imaging for single-shot depth acquisition. ACM Transactions on Graphics 35, 6 (2016), 194. Google ScholarDigital Library
- Nicola Brusco, S Capeleto, M Fedel, Anna Paviotti, Luca Poletto, Guido Maria Cortelazzo, and G Tondello. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Applications 17, 6 (2006), 373--393. Google ScholarDigital Library
- Xun Cao, Hao Du, Xin Tong, Qionghai Dai, and Stephen Lin. 2011a. A prism-mask system for multispectral video acquisition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (2011), 2423--2435. Google ScholarDigital Library
- Xun Cao, Xin Tong, Qionghai Dai, and S. Lin. 2011b. High Resolution Multispectral Video Capture with a Hybrid Camera System. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11). 297--304. Google ScholarDigital Library
- Inchang Choi, Daniel S. Jeon, Giljoo Nam, Diego Gutierrez, and Min H. Kim. 2017. High-Quality Hyperspectral Reconstruction Using a Spectral Prior. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2017) 36, 6 (2017). Google ScholarDigital Library
- J. M. Bioucas Dias and M. A. T. Figueiredo. 2007. A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration. IEEE Trans. Image Processing 16, 12 (Dec. 2007), 2992--3004. Google ScholarDigital Library
- Piotr Dollár and C Lawrence Zitnick. 2015. Fast edge detection using structured forests. IEEE transactions on pattern analysis and machine intelligence 37, 8 (2015), 1558--1570.Google Scholar
- Liang Gao, Robert T. Kester, Nathan Hagen, and Tomasz S. Tkaczyk. 2010. Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy. Opt. Express 18, 14 (Jul 2010), 14330--14344.Google ScholarCross Ref
- Nahum Gat. 2000. Imaging spectroscopy using tunable filters: a review. In AeroSense 2000. International Society for Optics and Photonics, 50--64.Google Scholar
- M E Gehm, R John, D J Brady, R M Willett, and T J Schulz. 2007. Single-shot compressive spectral imaging with a dual-disperser architecture. OSA OE 15, 21 (2007), 14013--27.Google ScholarCross Ref
- Ralf Habel, Michael Kudenov, and Michael Wimmer. 2012. Practical spectral photography. In Computer graphics forum, Vol. 31. Wiley Online Library, 449--458. Google ScholarDigital Library
- Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 6 (2013), 1397--1409. Google ScholarDigital Library
- Felix Heide, Mushfiqur Rouf, Matthias B Hullin, Bjorn Labitzke, Wolfgang Heidrich, and Andreas Kolb. 2013. High-quality computational imaging through simple lenses. ACM Transactions on Graphics 32, 5 (2013), 149. Google ScholarDigital Library
- Daniel S Jeon, Inchang Choi, and Min H Kim. 2016. Multisampling Compressive Video Spectroscopy. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 467--477.Google Scholar
- Jun Jiang, Dengyu Liu, Jinwei Gu, and Sabine Süsstrunk. 2013. What is the space of spectral sensitivity functions for digital color cameras?. In Applications of Computer Vision (WACV), 2013 IEEE Workshop on. IEEE, 168--179. Google ScholarDigital Library
- William R Johnson, Daniel W Wilson, Wolfgang Fink, Mark Humayun, and Greg Bearman. 2007. Snapshot hyperspectral imaging in ophthalmology. Journal of biomedical optics 12, 1 (2007), 014036--014036.Google ScholarCross Ref
- R. Kawakami, Y. Matsushita, J. Wright, M. Ben-Ezra, Y. W. Tai, and K. Ikeuchi. 2011. High-resolution hyperspectral imaging via matrix factorization. In CVPR 2011. 2329--2336. Google ScholarDigital Library
- Min H Kim. 2013. 3D Graphics Techniques for Capturing and Inspecting Hyperspectral Appearance. In Ubiquitous Virtual Reality (ISUVR), 2013 Int. Symp. on. IEEE, 15--18. Google ScholarDigital Library
- Min H Kim, Todd Alan Harvey, David S Kittle, Holly Rushmeier, Julie Dorsey, Richard O Prum, and David J Brady. 2012a. 3D imaging spectroscopy for measuring hyper-spectral patterns on solid objects. ACM Transactions on Graphics 31, 4 (2012), 38. Google ScholarDigital Library
- Min H Kim, Holly Rushmeier, John ffrench, and Irma Passeri. 2012b. Developing Open-Source Software for Art Conservators. In VAST12: The 13th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage. Eurographics Association, Brighton, England, 97--104.Google Scholar
- Min H Kim, Holly Rushmeier, John ffrench, Irma Passeri, and David Tidmarsh. 2014. Hyper3D: 3D Graphics Software for Examining Cultural Artifacts. ACM Journal on Computingand Cultural Heritage 7, 3 (2014), 1:1--19. Google ScholarDigital Library
- David Kittle, Kerkil Choi, Ashwin Wagadarikar, and David J Brady. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied Optics 49, 36 (2010), 6824--6833.Google ScholarCross Ref
- Haebom Lee and Min H Kim. 2014. Building a Two-Way Hyperspectral Imaging System with Liquid Crystal Tunable Filters. In Proc. Int. Conf. Image and Signal Processing (ICISP 2014) (LNCS), Vol. 8509. Springer, Normandy, France, 26--34.Google ScholarCross Ref
- Chengbo Li. 2011. Compressive sensing for 3D data processing tasks: applications, models and algorithms. Ph.D. Dissertation. Rice University. Google ScholarDigital Library
- Xing Lin, Yebin Liu, Jiamin Wu, and Qionghai Dai. 2014. Spatial-spectral encoded compressive hyperspectral imaging. ACM Transactions on Graphics 33, 6 (2014), 233. Google ScholarDigital Library
- Rang MH Nguyen, Dilip K Prasad, and Michael S Brown. 2014. Training-Based Spectral Reconstruction from a Single RGB Image. In Computer Vision---ECCV 2014. Springer, 186--201.Google Scholar
- Takayuki Okamoto, Akinori Takahashi, and Ichirou Yamaguchi. 1993. Simultaneous Acquisition of Spectral and Spatial Intensity Distribution. Appl. Spectrosc. 47, 8 (Aug 1993), 1198--1202.Google ScholarCross Ref
- Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. In ACM Transactions on Graphics, Vol. 22. ACM, 313--318. Google ScholarDigital Library
- Wallace M Porter and Harry T Enmark. 1987. A system overview of the airborne visible/infrared imaging spectrometer (AVIRIS). In 31st Annual Technical Symposium. International Society for Optics and Photonics, 22--31.Google ScholarCross Ref
- Konstantinos Rapantzikos and Costas Balas. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In IEEE International Conference on Image Processing 2005, Vol. 2. IEEE, II-618.Google ScholarCross Ref
- Brian Smits. 1999. An RGB-to-spectrum conversion for reflectances. Journal of Graphics Tools 4, 4 (1999), 11--22. Google ScholarDigital Library
- Tsuyoshi Takatani, Takahito Aoto, and Yasuhiro Mukaigawa. 2017. One-shot Hyperspectral Imaging using Faced Reflectors. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). TBD.Google ScholarCross Ref
- Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. 2008. Single disperser design for coded aperture snapshot spectral imaging. Applied optics 47, 10 (2008), B44--B51.Google Scholar
- Richard A Waltz, José Luis Morales, Jorge Nocedal, and Dominique Orban. 2006. An interior algorithm for nonlinear optimization that combines line search and trust region steps. Mathematical programming 107, 3 (2006), 391--408.Google Scholar
- Fumihito Yasuma, Tomoo Mitsunaga, Daisuke Iso, and Shree K Nayar. 2010. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing 19, 9 (2010), 2241--2253. Google ScholarDigital Library
- Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence 22, 11 (2000), 1330--1334. Google ScholarDigital Library
Index Terms
- Compact single-shot hyperspectral imaging using a prism
Recommendations
Compact snapshot hyperspectral imaging with diffracted rotation
Traditional snapshot hyperspectral imaging systems include various optical elements: a dispersive optical element (prism), a coded aperture, several relay lenses, and an imaging lens, resulting in an impractically large form factor. We seek an ...
High-quality hyperspectral reconstruction using a spectral prior
We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn ...
Catadioptric hyperspectral imaging, an unmixing approach
Hyperspectral imaging systems provide dense spectral information on the scene under investigation by collecting data from a high number of contiguous bands of the electromagnetic spectrum. The low spatial resolutions of these sensors frequently give rise ...
Comments