Introduction

More than 80 % of all cancers originate from the epithelium [1]. Current methods of diagnosing these cancers rely on histological and cytological examination of tissue or body fluids. For this purpose, brushes or biopsies are harvested, which have to be fixed and stained before diagnosis. To circumvent this time-intensive procedure, new optical imaging modalities that enable real-time (pre)malignant lesions detection in vivo are being investigated at the moment. Optical coherence tomography (OCT) is a technology developed in the early 1990s for ophthalmological applications [2, 3] and is profoundly used in that setting. OCT is the optical equivalent of ultrasound, using light instead of sound, to produce images of tissue. Resolutions up to 1–2 μm can be achieved, being 100–250 times higher than high-resolution ultrasound [4] and approaching that of microscopy. However, due to light scattering by the sample, imaging depth is usually limited to ∼2 mm. An image produced by OCT resembles the tissue architecture observed in histology and can therefore be considered as an “optical biopsy”. This “optical biopsy” has high potential in epithelial tumor diagnosis as it is non-invasive and real-time [4]. Moreover, functional quantitative information can be extracted, i.e., flow information, layer thickness, and attenuation coefficient of the OCT signal [5, 6]. Given the match between OCT imaging and histology in epithelial tissues, OCT can play an important role in the diagnosis of tumorous lesions.

After our pioneering work in kidney cancer [7, 8], bladder cancer [9], and premalignant vulvar intraepithelial neoplasia [10], we aim to review the diagnostic potential of OCT in (pre)cancerous lesions of the human epithelium. As most investigators study qualitative OCT, we focus on qualitative, morphological imaging of human epithelium, especially of the skin, the oropharyngeal and laryngeal area, the esophagus, the genital area, and the bladder. At the end, we focus on quantitative, e.g., functional OCT and we outline the potential future application of functional OCT imaging in diagnosing (pre)cancer.

Background of technique

OCT is based on depth resolved detection of elastic light scattering. When light is directed at a tissue sample, it will be partially back scattered. This back-scattered light is measured at different depths at a particular location on the tissue using low-coherence interferometry resulting in a reflection profile in the depth (z−) direction. The magnitude of the OCT signal at each depth is determined by the different cellular structures in the imaged volume (typically in the order of 103 μm3 or 1 pl) and as a result it differs per tissue type. Several adjacent depth profiles can be acquired in the lateral (x−) direction and displayed as a gray-scale image in real time which is known as an OCT B scan. Subsequently, the OCT beam can be scanned across a tissue sample in the other lateral (y−) direction, resulting in a 3D image representation with acquisition speed reported up to several volumes per second.

System setup

The OCT system (Fig. 1) consists of an interferometer, generally constructed from fiber optic components, illuminated by a broad wavelength–range light source operating in the near-infrared (typically 1,250–1,350 nm for non-ophthalmic applications). A small fraction of the light is guided towards a “reference” mirror; the majority is directed to the tissue, using handheld XY-scanning devices or miniaturized endoscopic probes (Fig. 2). Both fractions are combined and directed towards a detection unit and subsequent computer processing. Disposable in vivo OCT probes with a diameter less than 1 mm are developed and already used in several medical settings, enabling OCT to be used in endoscopes or in combination with 16G/18G needles to access internal tissue. Clinical value of OCT images depends on obvious factors such as high resolution, high-imaging speed, and adequate contrast to discriminate between benign and malignant tissues.

Fig. 1
figure 1

Schematic overview of the OCT system. An optical beam (from the light source) is split into two arms. One arm is directed at the tissue (scanning sample arm), the other at a mirror (the reference arm). The reflected light from these two paths is recombined and the differences between these two paths can be shown in a greyscale image

Full size image
Fig. 2
figure 2

Clinical OCT system employed at our medical center. Left shows the system on a cart allowing the system to be transferred from different clinical settings easily. The system can be interfaced with either an x/y scanning handheld device (upper right) or a rotating endoscopic scanning device (lower right). The latter scans its environment by fast rotational (lighthouse like) movements. 3D imaging in this case is achieved by a pullback of the probe

Full size image

Contrast in OCT is caused by spatial differences in refractive index of different tissue constituents, e.g., contrast originates from reflection of different structures. It is known that the refractive index is proportional to the density of the cells and cell structure. Because malignant cells display an increased number, larger and more irregularly shaped nuclei with a higher refractive index and more active mitochondria, OCT images are expected to be different in malignant tissue compared to normal and benign tissue [11].

Methods

For this systematic review, we performed a literature search using PubMed and Embase for original and review articles written in English, French, or German using the search terms optical coherence tomography and skin cancer or melanoma (rendering 85 and 16 articles, respectively), optical coherence tomography and oral, pharyngeal, or laryngeal cancer (rendering 30 and 8 articles, respectively), optical coherence tomography and esophageal diseases (rendering 74 and 27 articles, respectively). Optical coherence tomography and urinary bladder, vulvar, cervical, or penile cancer (rendering 95 and 42 articles, respectively) This search resulted in a total of 377 hits, from which we selected 57 articles based on relevant contribution in describing technology and clinical evaluation of these methods in skin cancer, oral, laryngeal and esophageal cancer and genital and bladder cancer. Selected articles originated from 1991 to 2012.

Results

Skin cancer

Nonmelanoma skin cancer

Nonmelanoma skin cancer (NMSC), including basal cell carcinomas (BCC), squamous cell carcinomas (SCC), and the premalignant actinic keratosis (AK), is the most prevalent cancer in light-skinned populations [12]. OCT could benefit the diagnostic management of NMSC, especially in patients with multiple lesions or lesions at locations where resection results in bad cosmetic outcome [13].

Several studies are performed to assess whether OCT is useful in diagnosing tumors and defining tumor borders [14–20]. Olmedo et al. investigated 27 patients with BCC. In 20 of them, the OCT images matched with histopathology. A typical OCT image of BCC is shown in Fig. 3a. In these images, the dark lobules representing BCC appeared as a reflection density of the epidermis. A decreased reflectance was seen around the periphery of the lobules of the tumor, while the fibrous stroma closely surrounding the BCC showed increased intensity. For the seven specimens that did not match, technical issues had interfered with the examination [18]. Gambichler et al. imaged BCC as well as healthy skin. In OCT images, healthy skin showed layered skin architecture, whereas BCC showed loss of these layers and disarrangement of the epidermis and dermis [14]. Absence of well-defined layering in NMSC has been seen by others as well [16, 20]. Mogensen et al. compared OCT directly to the accuracy of pathology diagnosis. Depending on the observers, sensitivity and specificity varied from 57 to 94 % and 43 to 96 %, respectively. Experienced observers reached a sensitivity of 79 to 94 % and a specificity of 85 to 96 % [16]. Instead of observing the OCT images, Jorgensen et al. implemented a machine-learning analysis. Correct classification of 37 AK lesions and 41 BCCs compared to pathology were made with an accuracy of 73 % (AK) and 81 % (BCC) [15]. Other studies focused on measuring the thickness of BCC [19] and AK [17] in vivo. A strong correlation between tumor thickness measured in both OCT and pathology was found (p < 0.001) [19]. Overall, OCT is able to show the different skin layers and is capable to measure tumor thickness in NMSC.

Fig. 3
figure 3figure 3

OCT vs. histology of several epithelial lesions

Full size image

Malignant melanoma

In case of a malignant melanoma (MM), thickness is the most important factor for (a) staging, (b) determining the size of the excision margins, and (c) the need for sentinel lymph node biopsy. Using OCT to measure MM thickness could enable better preoperative planning by defining resection margins and the need for sentinel lymph node biopsy. Moreover, a non-invasive diagnostic tool capable of distinguishing between benign nevi (BN) and MM would decrease the amount of unnecessary harvested biopsies and excisions. Histopathologically, MM consists of atypical melanocytes (with atypical nuclei and nucleoli) that invade the epidermis and sometimes even invade the dermis. These atypical cells may have a different nuclear refractive index and thus lead to difference in scattering [11]. A typical OCT image of MM is shown in Fig. 3b.

Only a few studies are performed to investigate whether OCT is useful in detecting MM. De Giorgi et al. studied the pigment network and brown globules in melanocytic pigmented lesions as seen with dermatoscopy [21] and compared these features with OCT and histology. In 6 out of 10 cases, it was possible to compare structures found in dermatoscopy and histology to structures in OCT. However, it was impossible to differentiate MM from BN. On contrary, a study by Gambichler et al. showed significant differences in OCT imaging of BN and MM [22]. A total of 75 patients with 52 BN and 40 MM were included. MM showed a marked architectural disarray (p = 0.036) and rarely displayed a clear dermo-epidermal border (p = 0.0031) when compared to BN. The most prominent OCT feature of MM was large, vertical cone-shaped structures, which were not observed in BN (p < 0.001) [22]. Although these results are promising, a large clinical study is needed to establish robust differentiation of melanoma from benign nevi.

Oral, laryngeal, and esophageal cancer

Oral cancer

SCC, accounting for 96 % of all oral cancers, is mostly preceded by dysplasia presenting as white or red lesions on the oral mucosa (leukoplakia, erythroplakia). Currently, these potentially malignant lesions are biopsied or excised and therefore less suitable for screening of high-risk population. Only a few studies have been performed on OCT in the oral cavity. One of these studies included 143 healthy volunteers and demonstrated a varying thickness of epithelium depending on its location within the oral cavity. The largest thickness values were measured in the region of the buccal mucosa and the hard palate, whereas the thinnest epithelium was found at the floor of the mouth [23]. Wilders-Smith et al. imaged 50 patients with oral suspicious lesions with OCT, shown in Fig. 3c. After imaging, standard biopsy and histopathology were performed. Two investigators diagnosed the lesions blinded on OCT and histopathology subsequently. For detecting carcinoma in situ or SCC versus noncancer, sensitivity was 0.931 and specificity was 0.931; for detecting SCC versus all other pathologies, sensitivity was 0.931 and specificity was 0.973 [24]. Another study showed differences between normal oral mucosa, dysplasia and SCC scoring the OCT images on three indicators: epidermal layer thickness, standard deviation of an A-mode scan intensity profile in the epidermal layer, and the attenuation of light. However, the clinical value of this study is limited due to unclear methodology [25]. Recently, Adegun et al. measured the backscattered light intensity as a function of depth in OCT images of biopsy samples from the oral cavity. Hereby, they could differentiate dysplasia from normal control samples [26]. Overall, OCT seems to differentiate well between normal tissue and SCC.

Laryngeal cancer

Several studies in the human larynx have been performed. In a study of 82 patients undergoing surgery for various head and neck pathology, OCT provided information on epithelial layer thickness, integrity of the basement membrane (BM), and microstructural features. Whereas the normal epithelial thickness varied from 98 to 185 μm, the epithelial layer was thickened up to 300 μm in hyperkeratotic lesions. In hyperkeratotic lesions, the BM on OCT images was still intact in contrary to early microinvasive SCC of the larynx, where disruption of the BM was seen on OCT images. Moreover, OCT images were comparable with conventional histopathology [27].

Esophageal cancer

Over the past two decades, the incidence of esophageal adenocarcinoma (AC) has risen considerably, especially in the distal esophagus and the esophagogastric junction. Several studies showed that gastro-esophageal reflux disease increases the risk of getting esophageal AC [28] by promoting the formation of Barret’s esophagus (BE), a precursor of AC. At the moment, endoscopy and random taken biopsies are used to follow up patients with BE. Endoscopic OCT might be helpful in selecting suspicious areas in BE for further biopsy.

The use of OCT in imaging the esophagus and the esophagogastric junction has been widely studied. Ex-vivo studies showed that OCT can differentiate diverse layers in the esophageal wall [29] and can generate images corresponding to histology [30, 31]. Poneros et al. formulated different criteria for intestinal metaplasia on the basis of OCT images. When tested prospectively, these criteria had a sensitivity of 97 % and a specificity of 92 % [32]. Likewise, others acquired OCT images of the esophagus and stomach in 69 patients containing normal squamous mucosa, BE and AC. These OCT images were accurately recognized by observers unaware of their site of origin, with an accuracy of 84.6 % [33]. Isenberg determined the accuracy of endoscopic OCT in diagnosis of dysplasia in patients with BE. They reported an accuracy of 78 % for the detection of dysplasia in BE. Main limitations of the study were the variability in endoscopists’ accuracy rates, difficulty in real-time interpretation, and the need for refined criteria of dysplasia in endoscopic OCT imaging [34]. OCT of dysplasia was studied in more detail by Evens et al. They studied biopsy-correlated OCT images to establish OCT image characteristics able to differentiate intramucosal carcinoma (ICM) and high-grade dysplasia (HGD) from low-grade dysplasia, because ICM and HGD have a higher risk of developing into AC. There was a significant relationship between the histopathological diagnosis of IMC/HGD and the scores for each OCT image feature [35]. Recent improvements in endoscopic OCT made it possible to image larger areas such as the entire distal esophagus (approximately 6.0 cm) with ultrahigh resolution [36, 37]. An example of a typical OCT image of the esophagus is given in Fig. 3d. With this approach, it may be possible to accurately define patients with dysplasia who are at risk for developing AC.

Genital and bladder cancer

Vulvar cancer

In the last 30 years, the incidence of vulvar intraepithelial neoplasia (VIN), a premalignant skin disorder that often causes pruritus, pain and psychosexual dysfunction, has increased more than 400 % to approximately 2.5 cases per 100,000 women in the USA. The progression rate of VIN into VSCC is about 9 % in untreated patients and 3.3 % in patients after treatment. VIN is treated using conservative surgical excision, laser vaporization, or medical therapy. Patients are regularly examined to foresee occult invasion and check for possible new VIN lesions [38]. Definite diagnosis of a vulvar lesion of uncertain significance is obtained by punch biopsy. However, every attempt is made to avoid vulvar mutilation that may lead to psychosexual distress [39, 40]. Hence, in the present workflow in the diagnosis and treatment of VIN, there is a clear need for non-invasive diagnostic tools.

In a study by Escobar et al., three patients with Paget’s disease (a potential premalignant lesion) were imaged with OCT [41]. The authors observed clear irregularities in the epithelial layer and disruption of the basal membrane. Besides disruption of the basement membrane, OCT may be helpful in characterization of vulvar lesions by demonstrating differences in layer thickness and in the attenuation of light between different tissues. Data from our group in 16 patients with VIN show a significant difference in the thickness of the epidermal layer as well as in the attenuation of light between VIN lesions and healthy skin [10]. These differences probably occur because of the growth of neoplastic cells and changed nuclear/cytoplasma ratio in VIN lesions compared with healthy cells [11]. These data indicate the potential of OCT to distinguish between healthy and VIN lesions. In Fig. 3e, an example of an OCT image of a VIN lesion is shown.

Cervical cancer

Cervical cancer is generally preceded by intraepithelial neoplasia (CIN). Hence, the implementation of new imaging techniques that allow cheap, real-time, and non-invasive detection of premalignant abnormalities have the potential to improve the logistics and economics of screening programs for invasive cervical cancer [41, 42].

The diagnostic value of OCT in the detection of CIN has been studied by several groups. In 1999, the first ex vivo study on 32 cervical specimens concluded that an intact basement membrane—a feature of healthy epithelium in histology—can be seen with OCT [43]. The first in vivo studies were performed by Escobar et al. [41]. OCT images of patients with CIN I, II, and III lesions were directly compared to histology reports. All images of normal cervix showed a clear epithelial layer and a basal membrane that was defined as an interface between a brighter scattering (stroma) and a poorer scattering (epithelium) region in the OCT image. The authors concluded that this pattern presented normal squamous epithelium. In 16 of the 18 patients with CIN II and/or III, the images showed an unstructured homogenous area with highly backscattering region and fast attenuation of the signal; however, this was not quantified. This correlation between the intensity of the backscattering light from the epithelia of normal and abnormal tissue in the cervix, was also found by others [44]. To determine further accuracy of the OCT images, a grading scale was used to describe the OCT images: normal if a well-organized two-layered structure was seen with a sharp interface between the surface epithelium (squamous) and underlying layer (connective tissue); abnormal if the tissue was unstructured with no interface present and intermediate if irregularities on the images suggested artifacts or physiological conditions that did not meet the criteria. With this grading scale, sensitivity and specificity of OCT alone for ≥CIN II lesions was 56 and 59 %, respectively [45].

If patients with positive cervix cytology for dysplastic cells undergo colposcopy, OCT might have the potential to predict if a specific spot seen with colposcopy is CIN I, II, III or invasive cancer. In this setting, Liu investigated OCT imaging in colposcopy. OCT decreased the sensitivity to detect ≥CIN II lesions form 63 to 36 %; however, specificity increased from 83 to 93 % [46]. To improve clinical analysis, Lee looked at the degrees of circular polarization caused by the scattering changes induced by CIN. A linear fitting procedure was used to quantify the attenuation of the degrees-of-polarization signal. This approach resulted in sensitivity and specificity to detect CIN of 94.7 and 71.2 %, respectively [47]. An OCT image of a CIN I lesion is shown in Fig. 3f. Further improvements in OCT resolution are currently tested to improve accuracy of OCT for diagnosing cervical cancer and CIN.

Bladder cancer

In nondiseased tissue, the three anatomic layers of the bladder wall (urothelium, lamina propria, and muscularis propria) can be well distinguished with OCT [48, 49]. Hermes et al. evaluated OCT in 142 human bladder specimens ex vivo and demonstrated that OCT could discriminate between normal, CIS and invasive transitional cell carcinoma (TCC) with a sensitivity of 83.8 % and a specificity of 78.1 %. None of the CIS or TCC samples was classified as normal tissue based on the OCT image (no false-negatives), 6 samples of normal tissue were classified as TCC (false-positive rate 5.7 %), and 24 samples could not be classified (16.9 %) [50]. Two groups investigated the diagnostic accuracy of cystoscopic applied OCT. Manyak et al. classified bladder tissue samples as benign or malignant with an overall sensitivity and specificity of 100 and 89 %, respectively. Ten malignancies were classified by OCT as invasive (≥T1), of which nine were confirmed by histology [51]. Lerner et al. showed that differentiation of bladder tumors confined to the mucosa (Ta) was possible with a sensitivity of 90 % and a specificity of 89 %. For differentiation of T1 and T2 tumors, sensitivity was 75 and 100 %, while specificity was 97 and 90 % [52]. False-positive findings could generally be explained by inflammatory lesions of the bladder [51].

Lingley-Papadopoulos et al. developed an automated algorithm that allows recognition of texture within an OCT image to provide the urologist with a diagnosis. This algorithm was able to differentiate benign and malignant bladder tissue with a sensitivity of 92 % and a specificity of 62 % [53].

Since real-time high-resolution OCT images can be obtained during cystoscopy, the technique may especially be useful for guidance of biopsy procedures and staging of suspected tissue areas within the bladder. In Fig. 3g, an OCT image of a papillary carcinoma of the bladder is shown.

Discussion and conclusions

OCT is an optical diagnostic tool that aims to predict in vivo histopathologic diagnosis in a non-invasive way. It produces real-time high-resolution images comparable with histopathology. Several studies have shown that OCT can be used to distinguish healthy skin from NMSC. For detection of melanoma lesions, however, the accuracy is still limited. In the oral cavity and larynx, OCT is able to distinguish between normal tissue and SCC. In addition, OCT can give information about tissue thickness and basement membrane integrity. Differentiation between dysplasia and normal can still be improved. In the esophagus, large areas with high resolution can be imaged allowing tissue characterization of dysplasia as well as tumor tissue. For vulva lesions, OCT proved to be able to differentiate between healthy tissue and VIN lesions, while in the cervix OCT allowed differentiation between normal cervical tissue and CIN. However, OCT is not capable of distinguishing different CIN grades. Bladder OCT so far has shown the potential to differentiate grade and stage in small populations but it needs a larger population study to provide definite answers.

Future perspectives

OCT holds the potential to provide functional optical biopsies of epithelial cancers by combining imaging with quantification of physiological functional parameters, e.g., perfusion and oxygenation or cellular organization. This can be achieved either by advanced analysis of the OCT signal itself [54, 55] or by combining OCT with other imaging modalities such as Raman spectroscopy (RS) [56] or fluorescence spectroscopy [57]. Analysis of the spatial and temporal changes of the OCT signal allows determination of the optical attenuation coefficient (which is related to tissue organization) and blood perfusion. We have demonstrated that the attenuation coefficient discriminates between normal and (pre)cancerous lesions in vulva [10], bladder [9], and kidney [7, 8]. Advanced implementations of OCT use light that is extended from the near infrared to the visible wavelengths, allowing quantification of hemoglobin concentrations [58] and oxygen saturation ultimately allowing to map oxygen metabolism of individual locations in the lesion. Complementary information on biochemical composition can be obtained by combining OCT with RS. In that way, RS can estimate the molecular composition of tissue, while OCT can produce images. The use of OCT in combination with other techniques can hereby assist in screening and eventually even diagnose malignancies non-invasively.