Introduction

With recent advances in optical and digital technology, the consumer-level digital camera has become a convenient and cost-effective instrument for acquiring images for quantitative analysis [1], [2]. One major issue with using consumer-level cameras is obtaining a linear response, which is a prerequisite for tasks such as deriving spectral sensitivity curves [3], spectral reconstruction [4]–[9] and colorimetric evaluation [10]. Furthermore, quantitative analysis on images representing the linear sensor response has applications in various biological studies including: characterisation of animal colour patterns [11], and the evolution of signaller-receiver interactions through the analysis of the spectral component of images representing naturally-occurring visual signals [12]. In particular, measurements with digital cameras can be of high value for qualifying non-visible regions of the spectrum like the ultraviolet (UV) [13]. There are also new and emerging applications of using digital images for quantifying subject matter. For example digital imaging can be useful for measuring the occurring turbidity of fluids for quantifying bacteria counts [14], measuring spectral information from different inorganic salts [15] or in forensic applications for accurately documenting bite marks on skin through the use of the various penetration levels in different wavebands of radiation [16]. Although digital cameras designed for technical purposes usually maintain the linear relationship between the incident radiance and the camera response typical of most CCD and CMOS sensors [17], consumer-level digital camera models do not necessarily maintain this relationship. Departures from linearity in the camera response may be built into the camera’s hardware and software to satisfy several purposes, such as the historical practice of gamma correction, aesthetic and perceptual considerations relating to image display, and for increasing the dynamic range of the sensor [2], [18]. Furthermore, the techniques employed by the camera manufacturers are usually proprietary, and response curves are not generally available.

Linear responses from consumer-level cameras can be recovered by fitting a function to a plot of camera response versus incident radiance, the Opto-Electronic Conversion Function curve (OECF), and subsequently inverting the fitting function via analytical or graphical methods, or look-up tables (LUTs) [19]. Polynomial, power and exponential functions have been previously suggested as fitting functions [20], [21]. Nevertheless the implementation of these functions does not guarantee an accurate fit of the entire OECF curve for all camera models. For example, for cameras with extended dynamic or spectral ranges, the OECF curve may present two distinct regions: linear and saturation separated by an ‘inflexion’ point corresponding to the amount of energy required for activating the electron drainage mechanism [22]. Consequently, there is no a priori reason to expect a particular camera sensor to obey any specific analytical function for its OECF curve. For this reason, it is necessary to carry out measurements to find a function that is able to accurately fit the entire OECF curve if high quality quantifiable data is to be recovered.

Here we compare the use of (parametric) cubic Bézier curves and biexponential functions for characterising two camera models: (i) a Canon D40 camera sensitive to visible radiation and (ii) a Nikon D70s camera modified for recording near-ultraviolet radiation. Although both methodologies allow the recovery of linear camera responses, they differ in the model assumptions, the interpretation of the recovered camera responses and the size of the uncertainty bounds associated with the recovered responses. We compare performance using both methods and provide some recommendations for selecting the appropriate method depending on the intended use of the recovered linear responses.

Materials and Methods

Results

OECF curves were reconstructed for the three different colour channels of the Canon camera and the red channel of the modified Nikon device. All the reconstructed OECF curves present a similar form that are entirely fitted by implementing either biexponential functions or cubic Bézier curves (Figure 1); nevertheless, the use of Bézier curves requires an additional normalisation step prior to fitting as these curves are solely defined in a [0, 1] interval [30]. Normalisation was carried out on the two variables defining the OECF curve: camera responses and irradiation input, with the latter defined by the selected exposure parameters as expressed by Equation 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Cubic Bézier curves (dashed lines) and biexponential functions (solid lines) fitting the camera responses (circle markers) making up the OECF curves for the red (a), green (b) and blue (c) colour channels of a Canon 40D digital camera and the red colour channel of a Nikon D70s camera modified for ultraviolet recording (d).

Exposure values corresponding to the total incident irradiance were calculated from Equation (2). Values were normalised by dividing the total amount of irradiance required for each camera response () by the amount of energy required to attain a camera response equal to the selected maximum pixel response (). See text for details.

https://doi.org/10.1371/journal.pone.0079534.g001

Pixel intensity values, representing the camera output, were normalised by dividing each camera response by the maximum intensity level attainable in the selected colour-bit depth scale. This value, , equals 255 intensity levels for the 8-bit colour encoding scheme selected for characterising the two cameras. Normalisation of the input exposure was done by dividing the exposure value corresponding to each camera response included in the OECF by the exposure required to obtain for each characterised colour channel.

The maximum exposure values () obtained for the four characterised colour channels were: 0.0122 μmol, 0.0125 μmol, 0.0124 μmol and 0.0081 μmol, corresponding to the Canon camera red, green, blue channels and the modified Nikon D70s UV-sensitive red channel respectively. values were obtained from a biexponential function fitted to the OECF curves expressed in the original (not normalised) scale; however, these values can also be directly obtained from the OECF curve either by visual inspection or by linear interpolation of the OECF data points, provided that there are enough points at the upper end of the curve up to the value. Note that a biexponential function can be fitted to the OECF curve expressed either in the original or a normalised scale.

Regardless of the method selected to fit a given OECF curve, linear camera responses, i.e. the intensity of the irradiance signal at a given pixel location corresponding to a given value, can be recovered by inverting the equation of the selected fitting function. The parameters defining the two fitting functions, namely the coefficients of the biexponential function and the coordinates of the control points for the Bézier curve, are presented in Tables 1 and 2 along with their 95% confidence intervals. Equations 1 and 2 present the general form of the two fitting functions. Whilst the biexponential function coefficients and associated 95% confidence intervals (Table 1) were obtained directly from the output of the biexponential fitting procedure, implementation of simulation techniques were necessary for obtaining the coordinates of the control point defining the Bézier and their 95% confidence intervals (Table 2) as detailed in the Methods section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Coefficients of biexponential functions fitting the OECF curves for two camera models.

https://doi.org/10.1371/journal.pone.0079534.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Coordinates and 95% confidence intervals for the four control points defining each Bézier curve fitting the OECF curves for two camera models.

https://doi.org/10.1371/journal.pone.0079534.t002

Another important difference between the two fitting functions is the minimum camera responses included in the OECF: the value. The precise value for was found to be a factor influencingg the number of Bézier segments required to accurately fit the OECF curve (Figure 2, panel B). Although complex curves can be accurately fitted using several Bézier segments rather than a single Bézier curve [30], for the purpose of camera characterisation, it is desirable to fit the entire OECF curve using a single segment in such way that the LUT required for recovering the linear camera values can be constructed applying an equation-based interpolation (Equation 3) from a single Bézier segment. values were set at 31 and 37 pixel intensity values for the Canon and Nikon camera respectively, corresponding to the first control point () on the Bézier curve. On the other hand, the biexponential function accurately fitted the entire OECF curve, eliminating the need for a value (Figure 2, panel A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Observed camera responses for the red channel of a Canon 40D digital camera (red × markers) and fitting results including values below the minimum pixel response value .

(A) Biexponential fit (black circle markers), (B) 19 Bézier segments (black squares).

https://doi.org/10.1371/journal.pone.0079534.g002

Linear camera responses recovered by implementing the two methods are presented in Figure 3. The uncertainty associated with the recovery of the linear camera responses varies with the exposure, reaching its maximum value at for the two methods. Such a behaviour is not surprising, as large changes in exposure only produce slight changes in camera responses near as expected from the asymptotic behaviour of the OECF curve (Figure 1); however, an important difference between the two methods is the number of dimensions associated with the uncertainty of the recovered linear camera responses. Whilst the uncertainty of the linear camera responses recovered by implementing a biexponential function is only associated with the recovered exposure value, i.e. variation in the y-axis (Figure 3, right column), the uncertainty of the recovered linear camera responses by using Bézier involves both the and parameters (Figure 3, left column), as these are required to define each control point of the Bézier curve (Equation 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Recovered linear camera responses and confidence bounds for the (A–B) red, (C–D) green and (E–F) blue channels of a Canon 40D digital camera and; (G–H) the red channel of a Nikon D70s camera modified for ultraviolet recording, using cubic Bézier curves (left column) and biexponential functions(right column).

Linear camera responses were obtained by inverting the biexponential fitting function (Equation 4) (squares) and implementing a look up table derived after evaluating a cubic Bézier curve (Equation 3)(circles). Confidence bounds represent the standard deviation in all cases. Standard deviation of the biexponential function for display purposes.

https://doi.org/10.1371/journal.pone.0079534.g003

The magnitude of the uncertainty associated with the recovered linear camera responses is not uniform, but varies with the different values of irrespective of the employed linearisation method (Figure 4). However, differences do exist in the total magnitude of the standard deviation obtained by implementing each method and in the precise values where it is higher. In the case of the biexponential function, the magnitude of the standard deviation increases in a relatively linear manner after reaching about 10% of and up to the saturation region where it rapidly increases until reaching (Figure 4, left column). This behaviour is also observed for the Bézier curves with an additional increase in the uncertainty of the recovered values at low irradiance levels arising from the high standard deviation associated with the parameter (Figure 4, right column).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Standard deviation of linear camera responses (cross markers) as a function of increasing values of / recovered implementing a biexponential function (dotted line left column) and cubic Bézier curves (solid and dashed lines right column) for the (A–B) red, (C–D) green and (E–F) blue channels of a Canon 40D digital camera and; (G–H) the red channel of a Nikon D70s camera modified for ultraviolet recording.

Standard deviations for each / recovered by the biexponential function were obtained after simulating 1,000 normally-distributed random coefficients within the 95% confidence intervals for each of the four parameters in Table 1. Standard deviation for each / recovered by the cubic Bézier curve were obtained from the LUTs constructed after simulating 1,000 normally-distributed pseudorandom coefficients within the 95% confidence intervals for the eight parameters in Table 2. Solid line in panels B, D, F and H corresponds to the standard deviation of the normalised camera responses (), whilst the dashed line represents the standard deviation of the recovered normalised exposure value ().

https://doi.org/10.1371/journal.pone.0079534.g004

The sum of squared errors (SSE) between the measured irradiance input (exposure) and the linear camera responses recovered by the two fitting functions is presented in the second and third columns of Table 3. Even though the implementation of biexponential functions always resulted in predicted camera response values which are closer to the exposure calculated from the measured irradiance, particularly at the extreme ends of the exposure range, a comparison of the median differences between the camera response values predicted by the two methods for the entire exposure interval did not prove significantly different (Table 3 fourth and fifth column). However, significant differences between the two methods do exist in the computational time required for applying the two methods. Calculation of the confidence bounds for 256 linear responses, as required to reconstruct the LUT employed for linearising images, took a median of 131 seconds for the biexponential function compared to a median of 4.10 seconds required for the implementation of the Bézier approach.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Statistical comparison of the linear camera responses obtained with two characterisation methods.

https://doi.org/10.1371/journal.pone.0079534.t003

Discussion

With the growing use of digital imaging for quantifying the tonal and spectral characteristics of radiations reflected from various object matter [1], [2], [11], [12], [14]–[16], it is important to have accurate methods for specifying the relationships between input irradiance signal and camera output for quantitative analyses. In spite of being sensitive to different regions of the spectrum, the OECF curves of the two tested cameras present a notable similarity in their general form (Figure 1). This result indicates a close likeness between the gain functions applied to the sensor response of the two cameras. The use of non-linear gain functions which asymptotically approach to is characteristic of different consumer-level digital cameras as a strategy for increasing their dynamic range [22], [29]; a commonly desired feature for commercial photography, but a limitation for quantitative image analysis [2]. Therefore the present method is potentially applicable to other camera models presenting a similar gain function, including those cameras capable of producing images from reflected near-ultraviolet radiation [1], [23].

Even though the two proposed characterisation and linearisation methods accurately recover the linear camera response (Figure 3, Table 3), they differ in the magnitude of the uncertainty associated with the recovered radiometric information. Irrespective of the selected linearisation method, a graphical depiction of the gain function, i.e. the OECF curve (Figure 1), in conjunction with a plot of the standard deviation as a function of exposure level (Figure 4), provides a guideline for establishing the maximum camera response included in a given image and its corresponding exposure value. By establishing these two criteria it is possible to define precise exposure parameters, f-number and shutter speeds, for attaining a standardised exposure, which in turn allows for an objective comparison among images recorded with the same camera.

Selecting values corresponding to values located before the region of increasing standard deviation has the advantage of ensuring the recovery of linear camera responses with the lowest possible uncertainty for a given camera system/colour channel combination; however, other factors such as the intensity of the signals produced by study object itself should also be considered when selecting the value.

One of the most common applications of linear camera responses is for reconstructing spectral sensitivity curves [3], defined as the ratio of linear camera response to incident energy at different wavelengths across a given spectral interval [33]. Camera characterisation by means of a biexponential function and the subsequent recovery of linear camera responses and their associated standard deviation after inverting the fitting function (Equation 4) is particularly useful in this case, as the linearised responses are expressed in the same units as the energy input (Table 2). Furthermore, the number of camera responses required for this application allows for a precise recovery of the linear camera responses whilst keeping the computational time at reasonable levels. On the other hand, the use of Bézier curves for this purpose not only requires an extra step represented by the multiplication of the recovered linear response by a separately-measured value of , but has the shortcoming of the wide uncertainty bounds associated with extremely low and high exposure values (Figure 3).

When the objective is to quantitatively analyse images representing complex scenes including large areas widely varying in irradiance levels (brightness), or when the entire photographic frame has to be analysed, a researcher faces different requirements. In these, and other biology-related studies involving imaging such as characterisation of animal colour patterns, camouflage studies, modelling non-human visual spaces and animal-plant interactions [1], [2], [11], [12], [34], [35], the efficiency of the Bézier technique may overcome the wider uncertainty levels associated with this methodology (Figure 3); in particular, when the digital images to be linearised consist of several megapixels. Yet in this case a biexponential linearisation function can be efficiently implemented if a LUT is constructed for linearising the images rather than directly inverting the function for the camera response at each pixel location as was done here.

In contrast to the biexponential fitting function, the cubic Bézier curve requires establishing a minimum pixel response value (). This value corresponds to the first control point of the fitting curve (Figure 1) and represents the lowest camera response that can be accurately linearised. Camera responses below follow a distribution different from the remaining OECF curve [2], [36], and including them may prevent attaining an adequate fit with the selected programming code. The precise value of varies from one camera to another and must be found empirically, which is again a limitation compared with the biexponential approach (Figure 2, panel A). Although it is possible to fit the entire OECF curve, including the low response region, implementing several Bézier segments rather than a single Bézier curve (Figure 2, panel B), this approach has the limitation of producing LUT tables whose values do not uniformly cover the entire OECF curve, but are clustered along different regions of varying length along the curve corresponding to the different segments (Figure 2, panel B). This arrangement of the LUT values makes it necessary to resort to interpolation techniques to recover linear values corresponding to values located on non-sampled regions of the OECF, thus introducing an additional step in the computation and increasing the uncertainty bounds of the recovered linear response. Contrary to the use of Bézier fitting techniques, the implementation of a biexponential function does not require the use of a value as it accurately fits the entire OECF curve including extremely low camera responses (Figure 2, panel A). This characteristic of the biexponential function is particularly convenient when reconstructing spectral sensitivity curves, as it removes the necessity to modify the exposure parameters to increase the camera’s response at wavelengths where the sensitivity is very low.

Even though the two methods differ in the number of parameters that need to be estimated to fit a curve, in the present application, only four parameters need to be estimated by either method. Camera characterisation by means of a biexponential function requires estimating four parameters, corresponding to the two coefficients included on each of the biexponential terms here represented by the letters b, c, d and g (Table 1 and Equation 4). Even though in principle camera characterisation by cubic Bézier curves requires finding a total of eight parameters represented by the and values for each of the four control points defining the curve in Equation (3), two of these points, and , are predefined by setting and by the highest included in the OECF, so again there are only four free parameters.

From our results it can be concluded that both biexponential functions and cubic Bézier curves overcome the limitations of power and exponential functions to completely characterise the OECF curve of cameras equipped with a Bayer filter array. Although either of the two methods can be used for accurately recovering total irradiance at a given pixel location, the selection of a particular method should be based on: (i) the final objective of using linear camera responses, and (ii) the potential implications of differences in the magnitude of the uncertainty associated with the recovered linear camera responses.

When the objective is to reconstruct spectral sensitivity curves, camera characterisation by means of biexponential functions is the best approach. These functions accurately model the entire OECF curve including the extremely low camera response and saturation region thus making unnecessary the use of ad hoc parameters, namely the value. Moreover camera characterisation by this method allows for a precise estimation of the normalisation parameters required for the implementation of Bézier fitting techniques.

On the other hand, cubic Bézier curves have the advantage of permitting the recovery of linear camera responses and their associated uncertainty bounds in a computationally-efficient manner through the implementation of a formula-based interpolation. When implementing this method, the required look-up-tables are constructed by simply inverting the axes, making unnecessary the implementation of numerical approximation algorithms such as those required for inverting the biexponential fitting function. Nevertheless when implementing this method it is still important to consider the wider uncertainty bounds compared to those obtained by implementing the biexponential approach.

Finally, by selecting adequate and values it is possible to establish precise and standardised minimum and maximum exposure parameters thus permitting the objective comparison and quantitative analysis of the reconstructed images. These images accurately reconstruct two-dimensional information from real, complex scenes, which should have high value for biological imaging and other quantitative image analysis applications.

Conclusions

Our results introduce two different methodologies for recovering irradiance information, at each pixel location, within a digital image recorded with RGB cameras sensitive to visible and UV irradiation. Both methods achieve this by fitting a mathematical function to the OECF curve (gain function) of the camera and subsequently inverting it to solve for exposure from camera responses. However the main differences between the two methods consist on the amount of uncertainty associated with the recovered irradiance and the means by which the two functions are inverted. Recovering of irradiance values by implementing biexponential functions results in consistently reduced uncertainty bounds, but the inversion of such a function requires resorting to optimisation techniques requiring longer computational times. On the other hand, recovery of irradiance values employing Bézier curves requires shorter computational times, is more intuitive and easily achieved with linear interpolation through the use of LUTs. The application of these methodologies makes it possible to accurately recover total irradiance information from complex scenes, enabling investigations such as the study of animal vision in natural settings.