Computational imaging1 is an emerging field that seeks to push the fundamental limits in imaging systems by synergistically integrating optics and computation. In recent years, significant advances have been made by leveraging deep learning in computational imaging2. By combining novel deep neural network (DNN) architectures and domain knowledge in optical physics, the performance limits in various systems are continuously being re-defined, including spatial resolution3,4, depth-of-field5, space-bandwidth product6, imaging speed6,7, sensitivity to low-photon count8, and resilience to random scattering9,10. Of particular interest by Luo et al.11 is the ability to overcome random scattering by a DNN.

A notable advance demonstrated in this work is how the DNN is being designed and implemented to enable “computer-free” computational imaging. The majority of the deep learning-based techniques rely on a modern computer (or a computer cluster) to perform both training and deploying the DNNs. As such, the “cost” associated with the deployment (e.g., power consumption, data bandwidth, size, and weight) is fundamentally limited by the computing hardware requirement (e.g., GPUs). In the past few years, optical computing solutions, in particular optical neural networks (ONNs), emerge as a promising alternative to enable highly efficient “computing” at the speed of light using only optical and photonic components12. While optical computing and ONNs have been extensively studied about 30 years ago13, including impressive demonstrations, such as face recognition14, recent advances in photonic devices and the emergence of extremely large DNNs have re-fueled the research interest in ONNs12. Such novel optical computing devices promise to significantly reduce the power, bandwidth, size, and weight and enable “edge computing” directly on systems, such as surveillance cameras and autonomous vision systems.

In this work, the diffractive deep neural network (D2NN) pioneered by the Ozcan group15 is implemented for performing a challenging task for imaging through random diffusers. The D2NN is physically constructed by a series of transmissive diffractive surfaces (Fig. 1), in which the “neurons” are embodied by the phase profiles of the “pixels” on the diffractive surfaces and the interconnections are described by the physics of optical diffraction. The D2NN is first trained to perform the task of optically reconstructing images of arbitrary objects that are covered by an unknown, random phase diffuser. The result of the training stage consists of a set of phase profiles that are subsequentially fabricated to construct the ONN. A remarkable result demonstrated by the authors is that the trained D2NN can all-optically reconstruct the unknown objects through an unknown, new diffuser (that has never been used during training). As such, it implies that this ONN architecture has sufficient generalization power, similar to its digital counterpart10, to be robust to random unknown scattering changes. This makes a major advancement in optical computing towards many applications that require robustness against degraded visions, such as astronomical imaging, surveillance, and autonomous driving.

Fig. 1: Diffractive deep neural network for computer-free computational imaging through random diffusers.
figure 1

Speed-of-light inference of the letter “2” through the unknown “phase diffuser” using the trained D2NN, which consists of multiple diffractive layers.

Full size image