Spectroscopic Approach to Correction and Visualisation of Bright-Field Light Transmission Microscopy Biological Data

Abstract

The most realistic information about a transparent sample such as a live cell can be obtained using bright-field light microscopy. Under high-intensity pulsing LED illumination, we captured a primary 12-bit-per-channel (bpc) response from an observed sample using a bright-field microscope equipped with a high-resolution (4872 × 3248) image sensor. In order to suppress data distortions originating from the light interactions with elements in the optical path, poor sensor reproduction (geometrical defects of the camera sensor and some peculiarities of sensor sensitivity), we propose a spectroscopic approach for the correction of these uncompressed 12 bpc data by simultaneous calibration of all parts of the experimental arrangement. Moreover, the final intensities of the corrected images are proportional to the photon fluxes detected by a camera sensor. It can be visualized in 8 bpc intensity depth after the Least Information Loss compression.

1. Introduction

Techniques of light microscopy give us a possibility to investigate features of various types of specimens and have been considered as the standard in observation of living cells for decades [1,2,3]. However, each of these microscopic techniques suffers from some disadvantages.

Unlike transmitted and reflected light microscopy techniques, fluorescence microscopy only allows the observation of specific structures which have been preliminarily labeled for fluorescence or excited with light for autofluorescence. Other limitations of fluorescence microscopy are associated with photobleaching [4] of applied dyes or autofluorophores; their phototoxicity influences biological processes in a live sample over time [5,6]. These aspects are avoided in contrast microscopy, e.g., phase contrast microscopy or differential interference contrast microscopy [7], where, naturally, some intracellular structures are enhanced, whereas some of them are suppressed by artificial light interferences [8]. Nevertheless, for observation of the biological experiments using all microscopic techniques mentioned above, a grayscale digital camera is mostly sufficient [9,10,11,12], since shapes of the observed objects (e.g., cells or organelles) are mainly evaluated or, as in case of fluorescence microscopy, we observe a light of known wavelengths emitted from the sample.

The limitations of classical bright-field microscopy include low contrast for weakly diffracting, absorbing, and autofluorescent samples and low resolution due to the blurry appearance of out-of-focus material. However, bright-field transmission microscopy can allow the observation of unlabeled living cells and tissues and their internal structures and is of increasing interest due to recent experiments on super-resolution using video enhancement [13] and live cell dynamics [14]. In label-free bright-field microscopy, the signal is generated mainly by diffraction of a wide range of light wavelengths in the sample. Light diffraction is a spectrally dependent phenomenon that can be utilized for determination of the transmission spectra of observed objects [15]. Therefore, in order to describe the spectral properties of the observed sample, in the proposed microscope arrangement [13,14], the object response is captured by a color digital camera.

As a result of even slight defects of the camera sensor and aberrations in the optical path, images may suffer from strong distortions which disable exact and simple data processing and analysis. Therefore, the microscope with the camera should be calibrated as any measurement device in order to achieve a more accurate representation of the object for further digital processing or visualization.

With the growing popularity of image processing automation, the question of technically correct and accurate images becomes even more acute. Different calibration approaches have been developed so far and are widely used in various microscopy techniques. The authors refer to the necessity of digital compensation of pixel-dependent noise specific to modern cameras used in fluorescent [16,17] and single molecule localization microscopy [18] as they affect the results of image analysis by software. In [19], the authors introduced a theoretical model of noise sources and proposed their correction algorithm for fluorescence microscopy. On the other hand, there are approaches for pure data-driven determination of statistical properties of imaging sensors [20]. In [21], the authors present a fast, data-driven method for correction of inhomogeneous fluorescence signals obtained via confocal microscopy when an image is multiplied by the inverse of the estimated spatially variable illumination gain. In [22], the authors discuss the impact of vignetting in application of light-field microscopy and introduce a calibration procedure to determine an optically correct reference point for microlens-based systems. Photometric calibration in spectral microscopy is discussed in [23,24,25,26], where methods are proposed to compensate a non-uniform spectral response and quantum efficiency.

In the presented paper, we describe a method for capturing, correcting, and converting 12-bit raw images of unmodified biological samples obtained using a video-enhanced bright-field microscope. The image correction is based on the spectral characteristics of light captured by each camera sensor pixel during the calibration process. Advantages of the calibration method are demonstrated in both an image data visualization algorithm and simple image processing algorithms. The possible image visualization utilizes a full 8-bit-per-channel (bpc) range for transformation of corrected images (that depicted originally spatial distributions of spectral radiant fluxes). In this way, we acquired the most informative (in the sense of color and contrast) microscopic images of cells that can be mutually visually compared at the level of the whole image series. As we show further in this paper, the described method of the microscope calibration also brings benefits to the subsequent microscopy image processing and analysis.

The calibration method may be technically improved but, from the conceptual point of view, is complete.

2. Materials and Methods

2.1. Sample Preparation

2.1.1. Preparation of Live Cell Cultures

We used two cell types for the experimental data collection: HeLa (human negroid cervix carcinoma, Sigma-Aldrich, Prague, Czech Republic, cat. No. 93021013) and L929 (mouse fibroblast, Sigma-Aldrich, Prague, Czech Republic, cat. No. 85011425). The cell lines were grown at low optical density overnight at 37 $°$C, 5% CO${}_2$, and 90% rH in a cell culture incubator. The nutrient solution for the HeLa cells consisted of 86.7% minimal essential medium with high glucose (>1 g L${}^{-1}$), 10% fetal bovine serum (FBS), 1% antibiotics and antimycotics, 1% L-glutamine, 1% non-essential amino acids, and 0.3% gentamicin. The nutrient solution for the L929 cells consisted of 87.7% Dulbecco’s Modified Eagle Medium with high glucose (>1 g L${}^{-1}$), 10% FBS, 1% antibiotics and antimycotics, 1% L-glutamine, and 0.3% gentamicin (all components purchased from Biowest Laboratories, Nuaillé, France).

For live cell imaging, fractions of live cell cultures were cultivated with relevant nutrient medium in a tissue culture dish with a cover glass bottom (Ibidi, Gräfelfing, Germany, cat. No. 81158) covered by special glass lids (Ibidi, Gräfelfing, Germany, cat. No. 80050).

2.1.2. Fluorescent Staining of HeLa Cells

The fluorescent microscopy experiment was conducted with HeLa cells on tissue culture dishes (Ibidi, Gräfelfing, Germany). The nutrient medium was sucked out and the cells were rinsed by PBS. Then, the cells were treated by glutaraldehyde (3%) for 5 min in order to fix cells in a gentle mode (without any substantial modifications in cell morphology) followed by washing in phosphate buffer (0.2 mol L${}^{-1}$, pH 7.2) two times, always for 5 min. The cell fixation was finished by dewatering the sample in a concentration gradient of ethanol (50%, 60%, and 70%) when each concentration was in contact with the sample for 5 min.

The dish with fixed HeLa cells was rinsed with distilled water, dried for approx. 1 h, then poured over the fluorescein solution (133 $\mathsf{\mu }$M) and washed on a minishaker at low rotational speed (approx. 50–100 rpm) for 30 min. After that, the dye solution was drawn off, the samples were slightly rinsed with distilled water (2×) and finally dried again.

2.2. High-Resolution Bright-Field Light Microscope

In this paper, we present the process of calibration of the optical path and digital camera sensor of a high-resolution bright-field light transmission microscope, enabling observation of microscopic objects, further called the nanoscope [13,14]. This microscope was developed by the Institute of Complex Systems (ICS) of the Faculty of Fisheries and Protection of Waters (FFPW) in collaboration with the Petr Tax–Optax company (Prague, Czech Republic). The scheme of the nanoscope components is shown in Figure 1. The optical path consisted of two Luminus 360 light emitting diodes charged by the current up to 5000 mA. This allows the video enhancement [27] by illumination of the sample by series (7×) of light flashes with a period of 0.323 s and a duty cycle of 70%; each of the series is followed by a light delay depending on the scanning frequency. In our research, two microscope objective lenses were used: 1. Nikon Plan 40×/0.65, ∞/0.17, WD 0.56; 2. Nikon LWD 20×/0.40, ∞/1.2, WD 3.1. A projective lens magnifies (2×) and projects the image on a JAI camera with a 12 bpc color Kodak KAI-16000 image sensor of 4872 × 3248 image resolution. The process of capturing the primary signal was controlled by a custom control software. The optical system of the microscope was facilitated by infrared and ultraviolet filters (Edmund Optics, Barrington, NJ, USA). This prototype microscope allowed us to conduct experiments in time-lapse mode (a capturing of the image series at one focus position) and scanning of the interior of the cell along the optical axis in the z-scan mode (capturing in different z-positions). The z-scan can be performed automatically by programmable mechanics with a step size down to 130.9 nm.

A time-lapse experiment (737 images) on mouse fibroblasts (see Section 2.1) was taken at the nanoscope set-up as follows: camera gain: 0, offset: 300, and exposure: 293.6 ms; LED current: 4500 mA; ILCX: $-106$ $\mathsf{\mu }$m; ILCY: $-201$ $\mathsf{\mu }$m; objective Nikon Plan: 40×/0.65, ∞/1.17, WD: 0.56. A nanoscope image from a time-lapse experiment on HeLa cells taken at the set-up as follows: camera gain 0, offset 300, and exposure 295 ms; LED current 3500 mA; objective Nikon LWD 20×/0.4, ∞/1.2, WD 3.1, closed NAMC2.

In order to compare our custom microscope with conventional commercial microscopes, we used

1.

A phase contrast microscope Nikon BioStation IMQ with a high-sensitivity 1.3 Mpx cooled monochrome camera, 40× objective magnification (Nikon, N.A. 0.8);

2.

A bright-field microscope Olympus IX51, 40× objective magnification (objective LUC Plan FLN 40× 0.60 Ph2, ∞/0-2/NF 22), camera Infinity 1 giving raw image data;

3.

A fluorescent microscope Nikon Eclipse 80i, 40× objective magnification (Nikon Plan Fluor 40×/0.75, Ph2 DLL, ∞/0.17, WD 0.66), mercury discharge tube, cell sample was stained by fluorescein.

The comparative biological experiments were conducted on HeLa cells (see Section 2.1).

2.3. Microscope System Calibration and Image Correction

The microscope system that was calibrated is the nanoscope (see Section 2.2). The scheme of the measuring apparatus can be seen in Figure 2a. Calibration sample, a Linear Step Filter NDL-10S-4 (Thorlabs, Newton, NJ, USA), is a set of neutral density filters with a discrete range of 10 neutral densities (ND) of optical densities (OD${}_{633}$) from 0.1 to 4. The filters themselves are thin layers of gray shades gradients of metallic (NiCrFe), light absorbing, material coated on one side of a 2 mm UV fused silica support. The optical fiber spectrometer used for the calibration of the microscope system was calibrated itself by the procedure described in Appendix A and characterized by its spectral sensitivity (Figure 2b). The input data for the calibration were also digitalized microscope camera filters’ spectra obtained by the camera producer (Figure 2c).

The camera calibration and image correction was performed in the steps as follows:

1.

Focus position determination:

(a)

During image acquisition, a light is passing through the microscope optical path and sample. The resulting signal was then captured by a camera sensor. Filters of ND 0.1, 0.2, 0.3, 0.4, 0.5, and 0.8 were scanned through their depths with a step of 130.9 nm. In this way, six sets of images were obtained. The results for neutral densities ND <0.1 and ND 0 were acquired for a path of ray with the blank support UV fused silica and for a clear path of ray without any light absorber, respectively;

(b)

The sets of images for ND <0.1–0.8 were processed by Image Info Extractor Professional software (ICS FFPW) (chosen Rényi parameter $\alpha =2$) [28]. The point information gain entropy (PIE) gives a total change of the image information after iterative removing of one pixel from an individual image. The highest value of this parameter allows one to pick an in-focus image taken from the center of the coatings (Figure 3). This reduced the diffraction-induced aberrations from the boundaries of the calibration sample.

2.

Acquisition of calibration images and transmission spectra of linear step filters:

(a)

A set of calibration data (40 images for each neutral density) was taken at the determined focus position (see item 1b). The calibration images were calculated as a pixel-wise mean through the image sets corresponding to each neutral density. In this way, we reduced a random noise in the measured data;

(b)

An optical waveguide P400-1-UV-VIS was housed inside a 3D-printed mechanical adapter at the determined focus position and connected to a spectrometer Ocean Optics USB 4000 VIS-NIR-ES. The spectral responses ND <0.1–0.8 in Figure 4a correspond to electroluminescences of relevant calibration sample filters. The spectral response ND 0 reflects electroluminescence captured without any absorber by an optical spectrophotometer during calibration sample illumination.

3.

Calculation of the radiant fluxes reaching each microscope camera pixel after passing the calibration filters.

(a)

The raw microscope diode wavelength-dependent signal obtained by a spectrophotometer (Figure 4a) was divided by a spectrophotometer sensitivity wavelength-dependent curve (Figure 2b). That gave the final calibrated microscope diode signal (Figure 4b);

(b)

After data smoothing and interpolation, the light spectra reaching each pixel of the camera sensor (Figure 4c) were obtained as multiplication of the incoming spectra (measured by the calibrated spectrometer; see item 3a and Figure 4b) by the respective (red, green, blue) quantum efficiency profile of the Kodak KAI-16000 image sensor (Figure 2c);

(c)

For each neutral density of the filter coating, the radiant flux (Figure 4d) reaching each pixel during exposure was calculated as an integral (trapezoidal rule) of the area below the respective incident spectrum.

4.

Calibration curves and image correction:

(a)

Calibration curves were obtained for each image pixel position and respective extensions of values of camera channel radiant flux (see the range of filter coatings in items 1–2). The calibration curves map the pixel intensities to the real spectra responses (e.g., Figure 5). Each pair of consecutive points was interpolated linearly;

(b)

In order to extend the range of experimental (useful) intensities, the initial calibration curves for each pixel were extrapolated linearly about 80% (Figure 5, magenta line). Statistics of the pixels’ intensity values for each calibration filter was evaluated (Figure 6) in order to see the deviations of the sensor pixels’ responses from idealities;

(c)

Hereafter, the calibrated image format (as double precision floating point type) showing the spatial distribution of radiant fluxes was transformed into a png format. The calibration curve associates the value of each calibration intensity with the respective experimental measured photon energy.

2.4. Least Information Loss (LIL) Image Conversion

The resulted calibrated images have >8 bpc png format, but nowadays the majority of digital screens can show only 8 bpc formats. For practical purposes, the LIL Convertor software was developed by the ICS FFPW. It converts >8 bpc images more informatively and comparably [29]. This program allows to convert RGB, grayscale, and raw (an appropriate Bayer mask [30] can be selected) data format to 8 bpc images. Rescaling the intensities in each color (r, g, or b) channel can be applied for image sets of any length (even for one image only) either separately (when the intensity maximum and minimum is found for each color channel separately) or together (one common intensity minimum and maximum, respectively, is found for all color channels). If color channels are normalized separately, more information in the image is preserved. In case of multi-image series, the intensities are rescaled between the maximum and minimum intensities through the whole series. The empty (unoccupied) levels are always removed in case that they are empty in all images of the image set. The basic LIL algorithm (Algorithm A1) is written in Appendix B.

The images can be cropped which is useful namely for removing dead pixels’ rows and columns. Intensities of, e.g., dead or hot pixels can be eliminated in the image by a thresholding function that, before applying the LIL conversion, replaces the undesirable intensities in an original vice-bit image by a neighbor pixel’s active intensity. The program is optimized to utilize multicore CPUs.

3. Results and Discussion

The nanoscope described in Section 2.2 was developed mainly for studying unlabeled live and fixed cells to investigate their real structures. Thus, no additional contributions to the object response, such as (imuno)fluorescent dye modifications, influence the detected signal. Investigation of the unmodified samples does not require complicated sample preparation and does not decrease the lifetime of the living cells.

In Figure 7, HeLa cells are presented in the phase contrast, bright-field, and fluorescent modes of a commercial microscope and in video-enhanced high-resolution imaging using the nanoscope at the same magnification. The detail descriptions of the optical paths for the microscopes is written in Section 2.2. In the phase contrast imaging (Figure 7a), the cell borders are surrounded by halos of light interferences. In the bright-field mode of the commercial microscope (Figure 7b), the cell structures are hardly observed, mainly due to a lower intensity of illumination and a larger size of (an object which can be projected on) a camera pixel. Moreover, due to the usage of a grayscale camera, the spectral characteristics of the standard bright-field image can be hardly analyzable. The staining for fluorescence microscopy (Figure 7c) visualizes only selected parts of the chemically fixed cells. The cells lost their viability. Moreover, information about the positions, shapes, or behavior of other organelles is also lost. In contrast, the nanoscope (Figure 7d) provides primary color images where, namely, all cell borders and condensed chromosomes during mitosis are nicely visible and the contrast can be further intensified by the method of simultaneous optical path and camera chip calibration (Section 2.3).

The color calibration of a digital camera is a process that has been already known to camera manufacturers for several decades. The fact that the microscope camera is somehow calibrated and, thus, acquired microscopy data are not commonly available in its raw format is unfortunately usually unknown to ordinary users of the digital microscopy. In the proposed calibration method (Section 2.3), the microscope is calibrated, in our case the custom-made nanoscope, as a whole (see Figure 1)—from the wavelength-dependent transmission and chromatic aberrations of the microscope objective and other optical elements in the microscope optical path up to the microscope camera sensor whose original calibration made by the camera producer was switched off in order to obtain the primary camera signal. The proposed calibration method allows to obtain more realistic spectral properties of a micro-object. The image intensity values in the original image are transformed into values of spectral radiant flux (Figure 8). In addition, it is evident (Figure 8) that the image histogram after correction is much more structured than that before the correction. Even in absence of the sample, the incoming light is distorted by the optical path and non-idealities of the camera sensor. Moreover, the corrected image shows no signs of optical vignetting (which is undesirable mainly for macroscopic camera imaging purposes; the fields of view of most microscopes are too small to show the edge optical vignetting). Our method helps to mitigate such problems, improving any consecutive method of analysis.

For the correct assessment of experimental results, a proper visualization of the >8 bpc images is important. For this, we proposed a LIL algorithm (Section 2.4). Figure 9 shows images of mouse fibroblasts and corresponding image intensity histograms with and without prior calibration converted using the LIL algorithm applied to a single image and a whole time-lapse series. The conditions at which the experiment was performed using the nanoscope are described in Section 2.2. There is a visible vertical line in the middle of each uncorrected image. It is caused by sensor taps’ imbalance problem, which can be crucial for digital processing algorithms. In addition, in the corrected image, we suppressed artifacts such as grains of dust in the microscope optical path (cf. Figure 9a–d) and intensified color contrast between different cellular structures and the background. All images in Figure 9 were obtained by the separate normalization of color channels and thus there are no natural colors. Even though the information is necessarily lost in the transformation from >8 bpc into 8 bpc images, the LIL transformation of an individual image preserves the highest proportion of it and the transformation of a whole series preserves most of the information in the series and allows a mutual visual comparison of its images.

As an example of how the described method affects the subsequent image processing, we applied a standard Matlab binarization function (after color-to-grayscale image transformation) to the corrected and uncorrected images (Figure 10). Figure 10 was acquired on the same Nikon 40× objective as Figure 9. The uncorrected rgb and binarized images (Figure 10a,b) have background non-uniformities (scratches and dust in the microscope optical path shown by arrows) and also presents less compact borders of pseudopodia (shown in the square) when compared with the corrected images (Figure 10c,d).

Conceptually, the most similar to our approach of microscope calibration are procedures described in [23,25,26].

Thigpen et al. described a photometric calibration which is suitable for a fluorescence microscope or a transmitted light microscope [23]. The method works wavelength- and exposure-wise, establishing white balance in the image background. The method was demonstrated on a microscope equipped with a grating-based spectral light source and a grayscale digital camera. Absorption spectra originated from various filters placed in the computer controllable emission-side filter wheel of the microscope. Calculating the average optical density (i.e., average intensity) of the background surrounding a sample for an image at a fixed wavelength allowed the researchers to approximate the incident light to the filter. Performing this calculation for all wavelengths yielded an intensity profile of the incident light which was flat through all wavelengths. After selection of a reference image (at a wavelength of 550 nm and reference camera exposure), the similarity measure between the reference image and the testing image was calculated for an image captured at each combination of wavelength and camera exposure. Then, the exposure for the image, which was the most similar to the reference, was selected as an exposure at the relevant wavelength.

Since this calibration method [23] is wavelength-wise, it allows the segmentation of objects of a particular wavelength in a microscopy image like our spectral method (following our calibration approach, ref. [15]). Unlike our method, the calibration filters are not placed in the focal plane of the imaging system. Moreover, this method [23] can show a drift in the monochromator’s precision.

Young et al. [25] proposed a method for photocalibration of a fluorescence microscope. The authors employed a set of six spectrally different LEDs and varied the current through the LEDs in a controlled way. A controlled amount of light emitted by the LEDs and transmitted by the microscope optical path (including/excluding combination of filters of different ND) was measured by a grayscale image sensor. After that, the digitized intensity was explained as a function of the LED current. The nonlinearity in the dependency “image intensity vs. LED current” was suppressed by the calibration of the excitation light by a photodiode using a Spectralon reflector. The photodiode has a dynamic range of 10${}^5$:1 and its integration time and gain were computer controllable.

Compared with this method [25], our approach is also suitable for not only bright-field microscopy but also fluorescence microscopy. However, our method is pixel-wise. Moreover, we utilize spectral properties of color camera chip filters which allows us to obtain complete information on spectral properties of imaged objects in each individual image. In contrast the approach described in [25] requires to capture a set of images in each time step. By our method, we can thus obtain spectral information also about objects that vary quickly over time. In addition, our method allows us calibration of the optical path in light energies. Such a calibration by Young’s approach is substantially technically more complicated. For that, the authors proposed a special portable instrument of the size of a microscope slide that uses LEDs at different wavelengths and with controllable amount of current going through in order to produce calibrated amounts of light [24]. Unfortunately, for work [25], we have not found any image outputs which would allow us qualitative and quantitative comparisons with our method.

The authors of [26] described a calibration of light fluxes for a wide-field fluorescence microscope using a thermocouple power meter, after passing the light through an objective with an openable iris. The method allowed the researchers to precisely estimate excitation intensities in the object plane using commercially available opto-mechanical components, compare experimental set-ups, and highlight details in an image. The whole calibration process for one objective and six fluorescence filter cubes lasted for 75 min. This method seems to be the most similar to ours. The main difference is in the usage of the thermocouple power meter for measurement of doses of the incident light instead of a fiber optical spectrophotometer. Using this procedure [26], one cannot obtain any spectrally resolved image information.

4. Conclusions

Biological experiments are hardly reproducible and repeatable but also often sensitive to the technical provision. The approach presented in this paper allows, without any extensive technical treatment, one to reduce, for one thing, the impact of optical path inhomogeneities and, for another thing, defects of a camera sensor on color properties of images in a bright-field light transmission microscopy experiment. Thus, it allows one to maximize the yield of information obtained in the technically simplest possible experiment. The calibration procedure proposed here (if needed, followed by a correct image intensity compression) is promising for more adequate and precise transfer of color information to the computer and researchers. This experimental technology allows observation of living cells in dynamics and makes the following microscopy image processing (especially, cells’ features and edge detection) easier.

The information in the uncalibrated image is obviously dominated by the sensor and light path inhomogeneities and for proper evaluation only calibrated images may be used for assessment. This is particularly important for the eventual automatic machine analysis which will be unavoidable in case of a large field of view containing many individual objects that cannot be analyzed manually.

This method makes edge detection (extraction of the edge features) easier, which is important in image processing. In other words, the calibration method allows a better grouping of the intensities of similar spectral properties in order to separate background from foreground in a bright-field transmission image.

This calibration approach as such is independent of the light microscope construction, from all its optical components to a digital camera type (monochromatic vs. color, of different bit depths). After slight differences in the procedure, this calibration method can be also suitable for contrast microscopy or fluorescent microscopy. The only condition of a successful application of the method is a microscope camera giving a primary (raw) signal which is not distorted and modified by either electronics or software.

Nevertheless, the described image calibration and correction method in bright field is not applicable only for observation of a live cell but also for analysis of other transparent materials such as, e.g., polymer scaffolds or nanoprinted polymer structures. After some slight changes in the experimental protocol and system arrangement, the method can also be usable in light reflection or fluorescence microscopy.

It can be concluded that, compared with the microscope calibration methods published previously, our approach to calibration is the most versatile and also enables further advanced data processing including spectral image data analysis.