Abstract
Image processing has become a critical technology in a variety of science and engineering disciplines. Although most image processing is performed digitally, optical analog processing has the advantages of being low-power and high-speed, but it requires a large volume. Here, we demonstrate flat optics for direct image differentiation, allowing us to significantly shrink the required optical system size. We first demonstrate how the differentiator can be combined with traditional imaging systems such as a commercial optical microscope and camera sensor for edge detection with a numerical aperture up to 0.32. We next demonstrate how the entire processing system can be realized as a monolithic compound flat optic by integrating the differentiator with a metalens. The compound nanophotonic system manifests the advantage of thin form factor as well as the ability to implement complex transfer functions, and could open new opportunities in applications such as biological imaging and computer vision.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.
References
Marr, D. & Hildreth, E. Theory of edge detection. Proc. R. Soc. Lond. B 207, 187–217 (1980).
Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986).
Hsu, H.-S. & Tsai, W.-H. Moment-preserving edge detection and its application to image data compression. Opt. Eng. 32, 1596 (1993).
Brosnan, T. & Sun, D.-W. Improving quality inspection of food products by computer vision—a review. J. Food Eng. 61, 3–16 (2004).
Fürhapter, S., Jesacher, A., Bernet, S. & Ritsch-Marte, M. Spiral phase contrast imaging in microscopy. Opt. Express 13, 689–694 (2005).
Gebäck, T. & Koumoutsakos, P. Edge detection in microscopy images using curvelets. BMC Bioinformatics 10, 75 (2009).
Cardullo, R. A. Fundamentals of image processing in light microscopy. Methods Cell Biol. 72, 217–242 (2003).
Haralick, R. M. & Shapiro, L. G. Computer and robot vision. IEEE Robot. Autom. Mag. 1, 28–48 (1991).
Solli, D. R. & Jalali, B. Analog optical computing. Nat. Photon. 9, 704–706 (2015).
Yu, N. & Capasso, F. Flat optics with designer metasurfaces. Nat. Mater. 13, 139–150 (2014).
Joannopoulos, J. D., Villeneuve, P. R. & Fan, S. Photonic crystals putting a new twist on light. Nature 386, 143–149 (1997).
Silva, A. et al. Performing mathematical operations with metamaterials. Science 343, 160–163 (2014).
Kwon, H., Sounas, D., Cordaro, A., Polman, A. & Alù, A. Nonlocal metasurfaces for optical signal processing. Phys. Rev. Lett. 121, 173004 (2018).
Bykov, D. A., Doskolovich, L. L., Bezus, E. A. & Soifer, V. A. Optical computation of the Laplace operator using phase-shifted Bragg grating. Opt. Express 22, 25084–25092 (2014).
Guo, C., Xiao, M., Minkov, M., Shi, Y. & Fan, S. Photonic crystal slab Laplace operator for image differentiation. Optica 5, 251 (2018).
Cordaro, A. et al. High-index dielectric metasurfaces performing mathematical operations. Nano Lett. 19, 8418–8423 (2019).
Zhu, T. et al. Generalized spatial differentiation from the spin Hall effect of light and its application in image processing of edge detection. Phys. Rev. Appl. 11, 034043 (2019).
Zhu, T. et al. Plasmonic computing of spatial differentiation. Nat. Commun. 8, 15391 (2017).
Zhou, J. et al. Optical edge detection based on high-efficiency dielectric metasurface. Proc. Natl Acad. Sci. USA 116, 11137–11140 (2019).
Bracewell, R. N. The Fourier Transform and its Applications (McGraw Hill, 2000).
Krivenkov, V. I. Guided modes in photonic crystal fibers. Dokl. Phys. 48, 414–417 (2003).
Fan, S. & Joannopoulos, J. D. Analysis of guided resonances in photonic crystal slabs. Phys. Rev. B 65, 235112 (2002).
Zhou, W. et al. Progress in 2D photonic crystal Fano resonance photonics. Prog. Quantum Electron. 38, 1–74 (2014).
Liu, Z. S., Tibuleac, S., Shin, D., Young, P. P. & Magnusson, R. High-efficiency guided-mode resonance filter. Opt. Lett. 23, 1556–1558 (1998).
Suh, W., Yanik, M. F., Solgaard, O. & Fan, S. Displacement-sensitive photonic crystal structures based on guided resonance in photonic crystal slabs. Appl. Phys. Lett. 82, 1999–2001 (2003).
Winn, J. N., Fink, Y., Fan, S. & Joannopoulos, J. D. Omnidirectional reflection from a one-dimensional photonic crystal. Opt. Lett. 23, 1573–1575 (1998).
Hsu, C. W., Zhen, B., Stone, A. D., Joannopoulos, J. D. & Soljacic, M. Bound states in the continuum. Nat. Rev. Mater 1, 16048 (2016).
Xu, L. et al. Dynamic nonlinear image tuning through magnetic dipole quasi‐BIC ultrathin resonators. Adv. Sci. 6, 1802119 (2019).
Oskooi, A. F. et al. Meep: a flexible free-software package for electromagnetic simulations by the FDTD method. Comput. Phys. Commun. 181, 687–702 (2010).
Lee, J. et al. Observation and differentiation of unique high-Q optical resonances near zero wave vector in macroscopic photonic crystal slabs. Phys. Rev. Lett. 109, 067401 (2012).
Moitra, P. et al. Large-scale all-dielectric metamaterial perfect reflectors. ACS Photon. 2, 692–698 (2015).
Zhou, Y. et al. Multilayer noninteracting dielectric metasurfaces for multiwavelength metaoptics. Nano Lett. 18, 7529–7537 (2018).
Zhou, Y. et al. Multifunctional metaoptics based on bilayer metasurfaces. Light Sci. Appl. 8, 80 (2019).
Phan, T. et al. High-efficiency, large-area, topology-optimized metasurfaces. Light Sci. Appl. 8, 48 (2019).
Molesky, S. et al. Outlook for inverse design in nanophotonics. Nat. Photon. 12, 659–670 (2018).
Arbabi, A., Horie, Y., Ball, A. J., Bagheri, M. & Faraon, A. Subwavelength-thick lenses with high numerical apertures and large efficiency based on high-contrast transmitarrays. Nat. Commun. 6, 7069 (2015).
Khorasaninejad, M. et al. Metalenses at visible wavelengths: diffraction-limited focusing and subwavelength resolution imaging. Science 352, 1190–1194 (2016).
Lin, Z., Groever, B., Capasso, F., Rodriguez, A. W. & Lončar, M. Topology-optimized multilayered metaoptics. Phys. Rev. Appl. 9, 044030 (2018).
Sell, D., Yang, J., Doshay, S., Yang, R. & Fan, J. A. Large-angle, multifunctional metagratings based on freeform multimode geometries. Nano Lett. 17, 3752–3757 (2017).
Acknowledgements
We acknowledge support from the Office of Naval Research under award no. N00014-18-1-2563 and DARPA under the NLM programme, award no. HR001118C0015. Part of the fabrication process was conducted at the Center for Nanophase Materials Sciences, which is a DOE Office of Science User Facility. The remainder of the fabrication process took place in the Vanderbilt Institute of Nanoscale Science and Engineering (VINSE) and we thank the staff, particularly K. Heinrich, for their support.
Author information
Authors and Affiliations
Contributions
Y.Z. and J.V. developed the idea. Y.Z. conducted the modelling and theoretical analysis. Y.Z. and H.Z. fabricated the samples with small die size (less than 1 mm2) and H.Z. fabricated the samples based on self-assembled masks. I.I.K. provided the substrates and fabricated the larger die size samples not based on self-assembled masks. Y.Z. performed all of the experimental measurements and data analysis, with assistance from H.Z. Y.Z. and J.V. wrote the manuscript with input from all of the authors. The project was supervised by J.V.
Corresponding author
Ethics declarations
Competing interests
Y.Z., H.Z. and J.V. have submitted a patent application for this work, assigned to Vanderbilt University.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Discussion and Figs. 1–8.
Rights and permissions
About this article
Cite this article
Zhou, Y., Zheng, H., Kravchenko, I.I. et al. Flat optics for image differentiation. Nat. Photonics 14, 316–323 (2020). https://doi.org/10.1038/s41566-020-0591-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41566-020-0591-3
This article is cited by
-
Diffractive optical computing in free space
Nature Communications (2024)
-
Broadband angular spectrum differentiation using dielectric metasurfaces
Nature Communications (2024)
-
Poincaré sphere trajectory encoding metasurfaces based on generalized Malus’ law
Nature Communications (2024)
-
Compact meta-differentiator for achieving isotropically high-contrast ultrasonic imaging
Nature Communications (2024)
-
Multichannel meta-imagers for accelerating machine vision
Nature Nanotechnology (2024)