Next Article in Journal
Light-Sheet Microscopy for Surface Topography Measurements and Quantitative Analysis
Next Article in Special Issue
The Quality Assessment of Different Geolocalisation Methods for a Sensor System to Monitor Structural Health of Monumental Objects
Previous Article in Journal
Extrinsic LiDAR/Ground Calibration Method Using 3D Geometrical Plane-Based Estimation
Previous Article in Special Issue
Tower of Belém (Lisbon)–Status Quo 3D Documentation and Material Origin Determination
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios

Istituto di Fisica Applicata “Nello Carrara” del Consiglio Nazionale delle Ricerche (IFAC-CNR), Via Madonna del piano 10, 50019 Firenze, Italy
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(10), 2843; https://doi.org/10.3390/s20102843
Submission received: 3 April 2020 / Revised: 8 May 2020 / Accepted: 13 May 2020 / Published: 16 May 2020
(This article belongs to the Special Issue Sensors for Cultural Heritage Monitoring)

Abstract

:
Imaging spectroscopy technique was introduced in the cultural heritage field in the 1990s, when a multi-spectral imaging system based on a Vidicon camera was used to identify and map pigments in paintings. Since then, with continuous improvements in imaging technology, the quality of spectroscopic information in the acquired imaging data has greatly increased. Moreover, with the progressive transition from multispectral to hyperspectral imaging techniques, numerous new applicative perspectives have become possible, ranging from non-invasive monitoring to high-quality documentation, such as mapping and characterization of polychrome and multi-material surfaces of cultural properties. This article provides a brief overview of recent developments in the rapidly evolving applications of hyperspectral imaging in this field. The fundamentals of the various strategies, that have been developed for applying this technique to different types of artworks are discussed, together with some examples of recent applications.

1. Introduction

In order to provide curators, scholars, conservators, archaeologists and conservation scientists with efficient tools for gaining knowledge of artifacts and archeological objects, it is important to study the materials and artists’ techniques used in creating the artworks and understand what restoration materials have subsequently been adopted in their preservation [1,2]. Presently, both invasive and non-invasive approaches can be used in the study of these materials; both have specific advantages and problems [2,3]. When dealing with cultural properties, the use of invasive techniques is discouraged since they require samples or micro-samples from investigated objects. However, these techniques offer more detailed information for identification purposes than non-invasive techniques. The latter, instead, are performed without sampling operations, and can be implemented as spot and as imaging techniques. Although they provide preliminary information on the materials and their distribution on the object’s surfaces, in most of cases, non-invasive techniques need a complimentary multimodal approach, in order to exhaustively characterize or identify all the materials. Usually, scientific investigation programs encompass a hierarchical use of analytical techniques, starting from non-invasive imaging techniques that are used for a preliminary screening and extensive evaluation of the surface, followed by non-invasive analytical spot techniques and, only when needed, as the last step, a complementary phase with micro-invasive techniques focused on investigating few suitable, selected points. Although not all these steps are always necessary, preliminary non-invasive analysis should always be recommended before starting any conservation or restoration procedure, in order to assist curators and conservators in their decision-making process. Non-invasive techniques, as such, do not alter the surface under examination, and are suitable for long-term or periodic monitoring of the same area. This approach is used to monitor the effectiveness of cleaning processes, restoration interventions, etc. In addition, monitoring over time is recommended in preventive conservation to safeguard artworks and prevent adverse actions of external agents (e.g., environmental agents, inappropriate microclimatic conditions, etc.).
When dealing with paintings, drawings, or other quasi-two dimensional (2D) art objects, imaging techniques are the preferred choice, not only for a preliminary evaluation of the conservation state of the artwork, but also for gaining knowledge on pigments and their distribution on the surface. Choosing the most suitable technique from the different available options depends on the size of the artwork, its value, the research interests and budget. The well-established traditional methodologies, such as ultraviolet-induced fluorescence (UVF), infrared reflectography (IRR), X-ray radiography (XRR) [4,5,6,7,8,9] are presently used together with more advanced techniques that enable the acquisition of multilevel information on pictorial materials, their distribution, and to some extent, also on their layering when suitable data-processing algorithms are used. These advanced imaging techniques include, multi-spectral imaging (MSI), hyper-spectral imaging (HSI), Macro X-Ray Fluorescence (Ma-XRF), and Time-Domain Terahertz imaging (THz-TDI), to name a few of the most significant [10,11,12,13,14,15,16,17,18,19,20,21].
This paper focuses on HSI, a cutting-edge technology that has nevertheless moved beyond the experimental phase, and has become quite accurate. It can be used in different applicative contexts, and can be implemented with several devices with different sensors and camera models. HSI devices for cultural heritage (CH) applications mainly operate in reflectance mode, and depending on the type of sensor, cover the Visible (Vis), Near Infrared (NIR), and Short-Wave Infrared (SWIR) regions. HSI enables the acquisition of a data-set that includes hundreds of spectral images acquired in very narrow spectral bands (bandwidth 2–10 nm). These data allow the reconstruction of a reflectance spectrum at each pixel in the scene, thereby, providing laboratory-like spectroscopic information useful for identification purposes. Algorithms that reduce the dimensionality of data and extract the required information are needed to process these data. Typically, image-processing is based on multivariate techniques, including statistical methods, such as Principal Component Analysis (PCA) and Maximum Noise Fraction (MNF), or automated classification methods, such as neural networks, t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP) [22,23,24,25]. These methods allow the grouping and mapping of artists’ materials and alteration products according to their spectral similarities [12,26,27]. This provides elaborate images where the material’s distributions are mapped, thus, enhancing aspects that are not detectable by visual inspection. The capability of visualizing hidden details is further improved when the HSI data cover the SWIR region. In this case, investigations of the painted surface is extended to the inner layers of paintings, highlighting the underdrawings, pentimenti, etc. [15].
These various potentialities have encouraged further study of HSI usability as an advanced tool for responding at once to the needs of analysis, documentation and/or timely monitoring of different classes of polychrome surfaces. In particular, high-resolution reflectance spectra, obtainable at each pixel of the imaged surface, has rendered HSI data useful for colorimetric analysis and for a comparative evaluation of chromatic changes in time.
For over a decade, numerous studies and research projects have been dedicated to adapting image spectroscopy instrumentation to the specific needs of art preservation and conservation [12]. Specifically, HSI devices made by a few specialized companies or, in very rare cases, prototypes developed or readapted by research groups, have been applied on canvas, panel, paper, and parchment paintings in laboratory and museum contexts [15,28]. Typically, these devices have mainly been designed to work on panel paintings, at a macroscopic scale and at a short distance from the target, covering areas of limited sizes that range from tens of square centimeters to few square meters. However, recent publications have demonstrated that a remote sensing-like approach could successfully be used to extend HSI applications to larger surfaces, such as frescoes and mural paintings [29,30,31,32]. Such studies report on the use of readapted avionic HSI sensors, or in other cases compact HSI cameras featuring very flexible optical features that were used for the inspection of outdoors targets at distances of several meters.
Interestingly enough, very recently another research front has developed towards the opposite direction: To apply HSI for the study of small size and finely detailed art and historical objects, such as photographic materials, including contemporary color negative and positive films. One such occasion presented itself when a corpus of photographic materials, which was considerably damaged by a flood, was studied, restored, and digitalized within the Italian Tuscan Regional Project ‘Memoria Fotografica’ (Photographic Memory). The measurements acquired in this project were performed with a modified version of high-resolution spatial and spectral HSI scanner made by IFAC-CNR, which allows non-invasive operations on surfaces of variable dimensions. This innovative IFAC-CNR HSI prototype, used to study 35 mm photographic negatives and positives, was optimized for the inspection of details featuring millimetric (or sub-millimetric) sizes [33]. Due to the excellent spatial and spectral resolution of the developed system, this novel methodology is now suitable for being used in this specific sector to support the conservation and restoration of negative and positive films. To the best of the authors’ knowledge, it was the first time such a technique was applied to the analysis of negative films.
Another important application of HSI technique is related to archival high quality documentation of works of art. In this case, matching the requested requirements by curators and conservators involves dedicated HSI instrumentation and experimental protocols. These data have to be acquired with an illumination/observation geometry configuration that follows the Commission Internationale de l’Eclairage (CIE) recommendations, such as the 2 × 45°/0° or d/0° configurations, to provide calibrated RGB images and colorimetric values (i.e., CIEXYZ, CIEL*a*b*, sRGB, etc.) [34,35,36,37,38]. The application of HSI technique for the colorimetric analysis of paintings before and after the restoration is still rare in the field due to the difficulties in having accurate, reliable, and reproducible data suitable for matching the colorimetric calculations as required by CIE.
In addition to these two pioneering approaches, some significant examples of HSI applications in the CH field are presented in the following sections, with a focus on the most recent developments that point toward new applicative directions.

2. State-of-the-Art of HSI Systems

Imaging spectrometers collect data over three dimensions: Two spatial and one spectral. For this reason the acquired data-set is called file-cube (or data-cube, image-cube), as each item/datum is associated with two spatial coordinates (x, y) locating the pixel position, and with a spectral coordinate (λ), providing the reflectance intensity at each wavelength.
The most common method for categorizing the various types of imaging spectrometers is by the method of collecting the data-cube in a single detector readout; they are categorized as; (a) Push-broom spectrometers; (b) Staring; (c) Fourier Transform Imaging Spectrometry; (d) Whisk-broom Spectrometers; (e) Snapshot Imaging Spectrometers [39].
Currently, most of the HSI systems tailored for CH field are push-broom sensors. These systems generally use prism-grating (PG or PGP) spectrographs connected to Si CCD/CMOS (400–1000 nm), InGaAs (900–1700 nm), MCT, InSb sensors (1000–2500 nm).
To acquire the data set, the imaging system is pushed across the object under investigation (HSI scanner) or vice versa, and the image-cube is built up one spatial line at a time. Alternatively, the acquisition of the data can be done by using an internal device scan mirror and without moving the HSI device in front of the object (or vice versa) [12]. Push broom spectrometers use a focal plane array (FPA) detector, and thus collect a vertical slice of the data-cube at once so that only one spatial dimension needs to be scanned to fill out the cube.
In the staring systems, a filtered camera is constructed by placing a filter wheel or a tunable spectral filter in front of a camera. This device collects several images at different wavelengths and needs to take several images of the same detail to complete the data set [40,41,42].
Fourier Transform imaging spectrometry (FTIS) uses an internal interferometer, usually a Sagnac interferometer, which images the interference pattern onto the camera sensor [43,44]. This type of imaging systems are not common, particularly not in the Vis-NIR regions.
Whisk broom spectrometers, which use a linear array of detectors, collect a single column of the data-cube at a time and thus scan across the two spatial dimensions of the data-cube [12]. However, to the knowledge of the authors, this type of system is not as commonly used for CH applications.
Snapshot imaging spectrometers collect the entire three-dimensional (3D) data-cube in a single integration period without scanning. This imaging system is faster than the previous ones as it can take spectral images at real time (several frames per second), but to date, the spatial and spectral resolutions are poorer than, for instance, the push-broom systems [45,46].

3. HSI Application on Photographic Materials

Within the “Memoria Fotografica” (Photographic Memory, 2018–2019) project, recently funded by the Tuscan Region (Italy) and IFAC-CNR Florence (Italy), HSI technique was applied, for the first time, on photographic materials, namely on contemporary negative and positive films. This project had set as its primary objective the definition of a methodology for the recovery, stabilization and restoration of photographic collection of the “Dainelli archive”, which was damaged by the flood that hit the territory of Leghorn, Italy, in September 2017. This archive included different classes of photographic material (color negatives, color slides, prints with chromogen and inkjet development) that were severely compromised by this extreme event. HSI technique was selected as an innovative scientific approach that could simultaneously face multiple needs, including: (a) analysis of the photographic materials (dyes and emulsions); (b) digitization; (c) tentative digital restoration and/or recovery of lost parts. Since the use of HSI for investigation on photographic negatives was unexplored until this pilot project, the instrumentation had to be re-designed for this specific purpose. At IFAC-CNR laboratories, a readapted version of the HSI scanner [47,48], operating both in transmission- and reflectance-mode, was devised to inspect negative and positive photographic films.
The hyper-spectral scanner for measurements in transmission on photographic material consists of a linear spectrograph connected to a camera, a mechanical part that allows the movement of photographic film pieces, a projector that uniformly illuminates a diffusing screen positioned almost in contact with the sample to be measured. The composed instrument is connected to a computer that allows its management and provides the acquisition of data and their display.
With this instrumental configuration, the scanner operates in the 400–900 nm range. The measuring head is equipped with an Opto-Engineering TC23024 telecentric lens, with a Prism-Grating type transmission spectrograph (Specim ImSpector™ -Oulu, Finland- mod. V10E) and with a Hamamatsu digital camera CCA ORCA–ERG model.
Scanning is obtained by moving the target (negative/positive film) with horizontal streaks up to about 25 cm. Prior to each horizontal scan, a calibration is performed on the illuminant itself. Lighting is provided by a 150 Watt tungsten-quartz halogen lamp (QTH) with a color temperature of approximately 3200 K. The instrument operates with a spatial sampling step of about 37 microns, resulting in the acquisition of approximately 27 points per millimeter, equivalent to almost 700 ppi. Thus, the obtained spectral images have, at half of the maximum contrast, a spatial resolution of 5 lp/mm. The spectral sampling is performed in steps of about 1.2 nm, which produces 400 bands and results in a spectral resolution of approximately 2.8 nm in the operating range.
Presently, scientific analysis applied to photographic films and their constituent materials for conservative purposes are quite fragmentary, and there are hardly any publications available dealing with this matter [49,50,51,52].
From a material point of view, the film system, as far as the color is concerned, can be considered relatively simple, with a three-layer structure. Each layer contains a specific dye, namely yellow, magenta and cyan dyes, to which other two correcting dyes have to be added. However, due to the complex and extensive gamut of color offered by the negative and positive color films, in the 400–900 nm range the identification of these colorants is far from easy. In fact, working in transmittance the three layers contribute to the final spectra producing complex spectral features (absorption bands) that are quite similar on all the pixels [52]. This makes it difficult attaining an analytical information and identification of dyes formulation (e.g., based on their commercial brand). However, it is possible to process the HSI data in order to infer some knowledge on the state of conservation of the frames, and thus to support with accurate spectroscopic information the digitalization phase and the subsequent digital restoration of the photographic material. In some cases, even partially lost parts of a scene can be reconstructed by recovering the information on colors from preserved details. As an example of the potentialities of this type of analysis, the results obtained on a color negative photogram, showing dented and un-homogeneous areas, are reported in Figure 1.
Figure 1a shows the entire sRGB image reconstructed from the HSI data of a negative color that had remained attached to the glassine envelope following the flood (after being in the water and mud for nearly two weeks) and detached during the restoration intervention.
To optimize the spectroscopic study of these frames, automated classification methods were used on these image-data in order to obtain a separation in homogeneous areas, or rather, to group pixels in classes that shared similar spectral trends. This phase of the research focused on testing a series of algorithms and procedures to define these areas. Among the various algorithms tested, a new statistical approach was proposed based on a mathematical model called “UMAP” (Uniform Manifold Approximation and Projection for Dimension Reduction) [25]. UMAP is a scalable algorithm for data dimension reduction that favors the preservation of local distances over global distance. It is built upon mathematical foundations related to the work of Belkin and Niyogi on Laplacian eigenmaps [53,54]. In the UMAP procedure, the elements (pixels) that make up the region of interest (ROI) are grouped based on the similarity of their spectral trend (Figure 1b). In this map, lightest areas correspond to the most represented areas. In the distribution map there is a rather compact central body (reported in white), while a peninsula and two detached areas are distinguishable in its periphery. To recognize the pixels on the sRGB image grouped by spectral similarity in these areas, an arbitrary color has been assigned to these zones (or classes). In Figure 1b five classes of pixels are highlighted in red, green, blue, cyan, and yellow. The corresponding average transmittance spectra, representative of these classes, are shown in Figure 1c. It can be observed that the classes represented with red, green, and blue colors have similar spectral trends, which are mainly dominated by spectral features of the magenta color present in part of the tram body (Figure 1d). The spectral differences in these three classes are instead related to the darker tone of the portion of the tram (blue curve), to the areas that are partially overshadowed by the trees (green curve) and to the remaining part under the sunrays (red curve). The red area in Figure 1d could also be related to the portion of the frame that was damaged by water. The other two classes, cyan and yellow, map the distribution in the frame sections that are lightly and severely damaged, respectively. The yellow class, particularly, groups pixels with spectra that show weak spectral features originating from the chromatic emulsions of the film during the harsh washing process during and after the flood.
Strengths and weaknesses UMAP algorithm are summarized in [25]. In particular, in Section 6 it is remarked that whenever the interpretability of the reduced dimension results is of critical importance, algorithms like PCA must be preferred with respect to t-SNE and UMAP. Nevertheless, methods like t-SNE and UMAP tend to keep the local structure with respect to the global structure of data, which in HSI application means that pixels that contribute to different clusters not necessarily have so distant spectra as the distance among the clusters may suggest. Here, to the authors’ knowledge, UMAP resulted to be better scalable than t-SNE, which means that its results depend less on data dimensions. On IFAC-CNR data, UMAP has worked well up to about 7,000,000 spectral pixels. To give an idea of the speeds, UMAP processes 429,165 pixels with 399 bands in 857.47 s, while t-SNE version optimized for Multicore CPUs by Dmitry Ulyanov (https://github.com/DmitryUlyanov/Multicore-TSNE) employs 2905.28 s.
As regards to the digitalization and digital restoration, HSI can help detect whether the dyes of the emulsions have maintained their chromatic characteristics or have undergone alterations based on the transmittance spectra extracted from the cube-file. These data need to be complemented with the technical data reported by the producers of photographic film. To this aim, HSI data-analysis based on PCA (Principal Component Analysis) demonstrated to be effective. Specifically, PCA was successfully applied to reconstruct the absorbance curves of each individual dye from their mixtures [52]. Using this procedure, the absorption curves of the three basic layers could be obtained for each frame, or groups of frames, to be compared with the data provided by the manufacturer when the films were produced/purchased. After an iterative process, the reported procedure converges toward an output data set that can be used to guide the digitalization of the degraded altered frames and their subsequent digital restoration. These preliminary results indicate that HSI can play a prominent role in the long-term preservation and development of photo inventory in archives and image collections [55].

4. HSI Application for Colorimetric Analysis of Paintings

Non-invasive diagnostics as well as accurate color acquisitions on paintings are possible with the HSI scanner developed at IFAC-CNR. Its orthogonal pair of linear motion actuators move a compact line-spectrograph in a vertical-plane. The adjacent vertical strips, which slightly overlap, perform the actual scan movement. The Specim Ltd. spectrographs ensure geometrical deformations that can be considered negligible, while the additional filters compensate any internal stray-light. This system’s spectral range is in the 400–1650 nm, with two separate spectrographic heads covering the VNIR (400–900 nm), which is also used for colorimetric applications, and SWIR (950–1650 nm) ranges with a spectral resolution of 2.5 nm and 8 nm, respectively. The devices are pushbroom HSI models based on PGP spectrographic heads. The technical details have been previously described in [47,48].
A fiber-optic illuminator (Schott-Fostec) which directs the radiation of a 3300 K QTH lamp onto a fiber-optic bundle forms the basis of the illumination module that terminates with a pair of light-lines with cylindrical lenses. The geometry of the illumination system is conformed to CIE standards for colorimetric measurements, projecting their beams around the viewed line-segment, symmetrically at 45° with respect to the surface [47]. The internal stray-light is one of the main cause of colorimetric errors that can be compensated only by a flat subtraction from all the wavelengths of the signal measured on the spectral channels below 400 nm.
The magnification of details without compromising quality is possible due to the combination of high color accuracy and high spatial sampling (0.1 mm) obtained from the image-cubes. Thus, it is possible to visualize, for instance, the pattern of paint cracks (craquelure), which is a detail of great interest in conservation documentation [38].
As a case study, the analysis of the “Polittico dell’Intercessione” (ca. 1425), a panel with five partitions (97 cm × 222 cm), painted by Gentile da Fabriano (c. 1370–1420), in the Church of San Niccolò Oltrarno in Florence, is reported. The HSI measurement was mainly focused on a color evaluation of the paint surface before and after the challenging restoration operations. These five partitions depict Saint Louis of Toulouse, the Resurrection of Lazarus, the scene of the intercession of Christ and the Holy Virgin with God, Saints Cosma and Damian, and also Saints Julian and Bernard [56]. It was considered a cryptic, unreadable work until an important restoration project made it comprehensible. The first documentation on this work, dated 1862, reported that the painting belonged to the Florentine church of San Niccolò Oltrarno. In 1897, the panel was badly damaged by fire, which magnified the alterations induced by previous restoration. Therefore, the polyptych was considered to be irrevocably damaged and, consequently, was stored for decades, until 1995, when it was selected for a new restoration project carried out by the Opificio delle Pietre Dure (OPD) in Florence, Italy [56].
Due to a standardized experimental acquisition protocol, that includes reproducibility of illumination, data-acquisition, repositioning, etc., IFAC-CNR HSI device measurements guarantee comparability between different measurements sessions. In this specific case, it was possible to colorimetrically compare the images collected before and after the restoration procedures on the painting. These data were stored as CIEL*a*b*76 TIF images files; it was possible, therefore, to visualize the three-color parameters (L*, a*, b*) as three separated grey level maps, in which high values for the three parameters corresponded to brighter pixels in the resulting image file. In Figure 2, the reconstruction of a detail of the scene “the Resurrection of Lazarus” is reported. Here, the sRGB image reconstructed from the HSI data at the end of the restoration procedure is reported together with the images of the b* colorimetric values calculated for the CIEL*a*b*1976 color space before and after the restoration, and with their color-difference image. In the analysis of this detail—as the gamut of the colors used by the artist was dramatically reduced after the fire of 1897 and by the previous restoration, it is evident that the color parameters have changed with different extents depending on the areas of the paintings. As an example, three spots were highlighted and their colorimetric values were calculated from the data cube (Table 1). They were chosen in order to show the chromatic effect of the restoration procedure on three colors, which are affected in a different manner by the presence of a yellowish aged varnish. Furthermore, this effect, which can be visually appreciated, has to be documented by using a CIE colorimetric approach. To better visualize the results on the painting after the removal of the aged varnish, CIE b* value can be used; it represents the blue–yellow component, with yellow and blue in the positive and negative directions, respectively. In this case, as expected, the bluish area (point 1) reduced the amount of yellow, while the red and flesh tone areas (points 2 and 3, respectively) increased this value resulting after the restoration more yellow (and red) than before. In addition, a* and L* values increased at different levels once the work was finished.
The visualization of these colorimetric variations through a new elaborated image difference is an important tool provided to conservators and art historians to understand the extent the restoration has changed the work.

5. Application of a New Compact and Portable HSI System

The relatively new Specim IQ HSI camera differs from other HSI devices available on the market since it integrates into a single compact and portable device housing the hyper-spectral sensor, additional color cameras, flash-memory card for data storage. Furthermore, this system works like a reflex camera with an interactive and touch display in the back of the camera that allow controlling the entire set of data acquisition and processing operations (Figure 3). Moreover, it has been designed as a mobile, hand-held and stand-alone HSI camera for applications in different kinds of environments [31,58]. The integrated color camera supports the spectral camera usage by allowing the visualization of the scene and adjustment of the manual focus of the spectral camera.
The camera can be used equally well indoors or outdoors and it works both with natural and artificial light. Like the majority of hyper-spectral cameras, Specim IQ is a push-broom system that makes a line-scan over the target area. The scanning is performed with internal mechanisms. The acquisition process usually takes from a few seconds upwards, so the camera has to be mounted on a tripod.
Specim IQ technical characteristics are the following: 400–1000 nm working range; 7 nm spectral resolution; 3.5 nm spectral sampling; 204 spectral bands; 512 × 512 pixels resulting image; unprocessed and processed data of each are directly saved by the camera and occupied approximately 300 MB. The hyper-spectral data are visualized immediately after the measurement and the user can add metadata to them “Saint Catherine carried by the Angels on Mount Sinai” (49.6 cm × 67 cm, private collection, 19th century, oil painting on canvas) by the Austrian artist Karl von Blaas (1815–1894) is here used as an example of the way in which Specim IQ can be applied to the study of artworks.
The HSI data served to identify the pigments the artist had used and to map them with Spectral Angle Mapping (SAM) algorithm over a particular area. The SAM procedure is a spectral classification tool which is uploaded in the camera’s software. The obtained results revealed, for instance, that the artist had used Thénard’s blue pigment (also known as cobalt blue, an important synthetic pigment used since the 19th century) in the blue and bluish areas. The image that appeared in the camera’s screen can be seen in Figure 3b, in which the graph at the right indicates the pixel extracted from the blue right sleeve of the angel. The identification of the blue pigment as cobalt blue was based on the entire spectral shape and its absorption features in the 500–700 nm range [59]. Subsequently, its areal distribution was mapped with the SAM tolerance thresholding option that was selected from the camera SAM mask tool (Figure 3c). This map was obtained by using a spectrum selected from the blue area of the imaged scene as reference.

6. Conclusions

After almost two decades since HSI was slowly introduced in the CH field, this technique has proven its versatility and utility for investigating different typologies of artworks, providing element to answer a variety of conservative issues. The HSI technique demonstrated enormous potentialities, which are partly still unexploited, in particular, in the examination of paintings and works on paper. This imaging technique offers great potential for future developments. In recent years, new application trends are emerging, including applications to photographic materials and an accurate color analysis as a support for restoration interventions. Moreover, new models of user-friendly HSI devices featuring portability and compactness are spreading the use of HSI techniques to several contexts and different environments. All these can contribute to creating other innovative HSI methods and applications and leading to new directions in the study of CH objects.
Finally, HSI is finding its place among other techniques as a preliminary approach to the study of artworks, specifically because it can analyze the entire surface of any nearly flat object.

Author Contributions

Conceptualization, M.P. and C.C.; methodology, M.P., C.C. and A.C.; software, A.C. and L.S.; validation, M.P., C.C., A.C. and L.S.; formal Analysis, M.P., C.C. and A.C.; investigation, M.P., C.C., A.C. and L.S.; resources, M.P., A.C. and L.S.; data curation, A.C. and L.S.; writing—original draft preparation, M.P. and C.C.; writing—review and editing, M.P. and C.C.; visualization, M.P., C.C., A.C. and L.S.; supervision, M.P.; project administration, M.P.; funding acquisition, M.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by Regione Toscana (Memoria Fotografica project).

Acknowledgments

The authors are grateful to Stefano Scarpelli for having allowed us to acquire data in his conservation studio on the painting by Karl von Blaas. The authors would like to thank the Opificio delle Pietre Dure (OPD), in particular Marco Ciatti and Cecilia Frosinini for their support in the investigation of the Gentile da Fabriano painting. Finally, the authors wish to thank Tuscan Region for having co-founded the Memoria Fotografica Project, and thank the photographer Daniele Dainelli and the colleagues who participated in this project for their invaluable support. A special thanks to Gabriele Simone and Sirpa Salenius for their useful discussions.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Taft, W.S.; Mayer, J.W. The Science of Paintings; Springer: New York, NY, USA, 2000. [Google Scholar]
  2. Pinna, D.; Galeotti, M.; Mazzeo, R. Practical Handbook on Diagnosis of Paintings on Movable Support. European Project ARTECH; Centro Di: Firenze, Italy, 2009. [Google Scholar]
  3. Artioli, G.; Angelini, I. Scientific Methods and Cultural Heritage: An Introduction to the Application of Materials Science to Archaeometry and Conservation Science; Oxford University Press: Oxford, UK, 2010. [Google Scholar]
  4. Mairinger, F. UV-, IR- and X-ray- imaging. In Non-Destructive Microanalysis of Cultural Heritage Materials; Janssens, K.H.A., Grieken, R., Eds.; Wilson & Wilson Elsevier: Antwerp, Belgium, 2004; pp. 15–73. [Google Scholar]
  5. Van Asperen de Boer, J.R.J. Infrared reflectography: A method for the examination of paintings. Appl. Opt. 1968, 7, 1711–1714. [Google Scholar] [CrossRef] [PubMed]
  6. Aldrovandi, A.; Bertani, D.; Cetica, M.; Matteini, M. Multispectral image processing of paintings. Stud. Conserv. 1988, 33, 154–159. [Google Scholar]
  7. Schreiner, M.; Frühmann, B.; Jembrih-Simbürger, D.; Linke, R. X-rays in art and archaeology: An overview. Powder Diffr. 2004, 19, 3–11. [Google Scholar] [CrossRef] [Green Version]
  8. Saunders, D.; Billinge, R.; Cupitt, J.; Atkinson, N.; Liang, H. A new camera for high-resolution infrared imaging of works of art. Stud. Conserv. 2006, 51, 277–290. [Google Scholar] [CrossRef] [Green Version]
  9. Hackney, S.; Townsend, J. Methods of examination and analysis. In Paint and Purpose—A Study of Technique in British Art; Hackney, S., Jones, R., Townsend, J., Eds.; Tate Gallery Publishing: London, UK, 1999; pp. 17–24. [Google Scholar]
  10. Striova, J.; Ruberto, C.; Barucci, M.; Blažek, J.; Kunzelman, D.; Dal Fovo, A.; Fontana, R. Spectral imaging and archival data in analysing Madonna of the Rabbit paintings by Manet and Titian. Angew. Chem. Int. Ed. 2018, 57, 7408–7412. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Liang, H. Advances in multispectral and hyperspectral imaging for archaeology and art conservation. Appl. Phys. A 2012, 106, 309–323. [Google Scholar] [CrossRef] [Green Version]
  12. Cucci, C.; Delaney, J.K.; Picollo, M. Reflectance hyperspectral imaging for investigation of works of art: Old master paintings and illuminated manuscripts. Acc. Chem. Res. 2016, 49, 2070–2079. [Google Scholar] [CrossRef]
  13. Delaney, J.K.; Zeibel, J.G.; Thoury, M.; Littleton, R.; Palmer, M.; Morales, K.M.; de la Rie, E.R.; Hoenigswald, A. Visible and infrared imaging spectroscopy of picasso’s harlequin musician: Mapping and identification of artist materials In Situ. Appl. Spectrosc. 2010, 64, 584–594. [Google Scholar] [CrossRef]
  14. Mounier, A.; Daniel, A. Hyperspectral imaging for the study of two thirteenth-century italian miniatures from the marcadé collection, treasury of the saint-andre cathedral in bordeaux, france. Stud. Conserv. 2015, 60, S200–S209. [Google Scholar] [CrossRef] [Green Version]
  15. Cucci, C.; Webb, E.K.; Casini, A.; Ginanni, M.; Prandi, E.; Stefani, L.; Vitorino, T.; Picollo, M. Short-wave infrared reflectance hyperspectral imaging for painting investigations: A methodological study. J. Am. Inst. Conserv. 2019, 58, 16–36. [Google Scholar] [CrossRef]
  16. Cucci, C.; Bracci, S.; Casini, A.; Innocenti, S.; Picollo, M.; Stefani, L.; Rao, I.G.; Scudieri, M. The illuminated manuscript Corale 43 and its attribution to Beato Angelico: Non-invasive analysis by FORS, XRF and hyperspectral imaging techniques. Microchem. J. 2018, 138, 45–57. [Google Scholar] [CrossRef]
  17. Casini, A.; Bacci, M.; Cucci, C.; Lotti, F.; Porcinai, S.; Picollo, M.; Radicati, B.; Poggesi, M.; Stefani, L. Fiber optic reflectance spectroscopy and hyper-spectral image spectroscopy: Two integrated techniques for the study of the Madonna dei Fusi. In Optical Methods for Arts and Archaeology; Salimbeni, P., Ed.; SPIE: Bellingham, WA, USA, 2005; Volume 5857, p. 58570M-1-8. [Google Scholar]
  18. Janssens, K.; Dik, J.; Cotte, M.; Susini, J. Photon-based techniques for nondestructive subsurface analysis of painted cultural heritage artifacts. Acc. Chem. Res. 2010, 43, 814–825. [Google Scholar] [CrossRef] [PubMed]
  19. Romano, F.P.; Caliri, C.; Nicotra, P.; Di Martino, S.; Pappalardo, L.; Rizzo, F.; Santos, H.C. Real-time elemental imaging of large dimension paintings with a novel mobile macro X-ray fluorescence (MA-XRF) scanning technique. J. Anal. At. Spectrom. 2017, 32, 773–781. [Google Scholar] [CrossRef]
  20. Fukunaga, K.; Picollo, M. Characterisation of works of art. In Terahertz Spectroscopy and Imaging; Peiponen, K.E., Zeitler, A., Kuwata-Gonokami, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; Volume 171, pp. 521–538. [Google Scholar]
  21. Pastorelli, G.; Trafela, T.; Taday, P.F.; Portieri, A.; Lowe, D.; Fukunaga, K.; Strlič, M. Characterisation of historic plastics using terahertz time-domain spectroscopy and pulsed imaging. Analy. Bioanal. Chem. 2012, 403, 1405–1414. [Google Scholar] [CrossRef]
  22. Kruse, F.A.; Lefkoff, A.B.; Boardman, J.B.; Heidebrecht, K.B.; Shapiro, A.T.; Barloon, P.J.; Goetz, A.F.H. The spectral image processing system (SIPS)—Interactive visualization and analysis of imaging spectrometer data. Remote Sens. Environ. 1993, 44, 145–163. [Google Scholar] [CrossRef]
  23. Mardia, K.V.; Kent, J.T.; Bibby, J.M. Multivariate Analysis; Academic Press: London, UK; New York, NY, USA; Toronto, ON, Canada; Sydney, Australia; San Francisco, CA, USA, 1979. [Google Scholar]
  24. Martens, H.; Naes, T. Multivariate Calibration; Wiley and Sons: New York, NY, USA, 1989. [Google Scholar]
  25. McInnes, L.; Healy, J.; Melville, J. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv 2018, arXiv:1802.03426v2. [Google Scholar]
  26. Bai, D.; Messinger, D.W.; Howell, D. A hyperspectral imaging spectral unmixing and classification approach to pigment mapping in the Gough & Selden Maps. J. Am. Inst. Conserv. 2019, 58, 68–89. [Google Scholar]
  27. Deborah, H.; George, S.; Hardeberg, J.Y. Spectral-divergence based pigment discrimination and mapping: A case study on The Scream (1893) by Edvard Munch. J. Am. Inst. Conserv. 2019, 58, 90–107. [Google Scholar] [CrossRef] [Green Version]
  28. Delaney, J.K.; Picollo, M. JAIC special issue on Reflectance hyperspectral imaging to support documentation and conservation of 2D artworks. J. Am. Inst. Conserv. 2019, 58, 1–2. [Google Scholar] [CrossRef] [Green Version]
  29. Liang, H.; Lucian, A.; Lange, R.; Shing Cheung, C.; Su, B. Remote spectral imaging with simultaneous extraction of 3D topography for historical wall paintings. J. Photogramm. Remote Sens. 2014, 95, 13–22. [Google Scholar] [CrossRef] [Green Version]
  30. Cucci, C.; Picollo, M.; Chiarantini, L.; Sereni, B. Hyperspectral remote sensing techniques applied to the non-invasive investigation of mural paintings: A feasibility study carried out on a wall painting by Beato Angelico in Florence. In Optics for Arts, Architecture, and Archaeology V; Pezzati, T., Ed.; SPIE: Bellingham, WA, USA, 2015; Volume 9527, p. 95270P-1-9. [Google Scholar]
  31. Cucci, C.; Casini, A.; Stefani, L.; Picollo, M.; Jussila, J. Bridging research with innovative products: A compact hyperspectral camera for investigating artworks: A feasibility study. In Optics for Arts, Architecture, and Archaeology VI; Pezzati, T., Ed.; SPIE: Bellingham, WA, USA, 2017; Volume 10331, p. 1033106-1-13. [Google Scholar]
  32. Camaiti, M.; Benvenuti, M.; Costagliola, P.; Di Benedetto, F.; Moretti, S. Hyperspectral sensors for the characterization of cultural heritage surfaces. In Sensing the Past, Geotechnologies and the Environment Series; Masini, N., Soldovieri, F., Eds.; Springer International Publishing: Cham, Switzerland, 2017; Volume 16, pp. 289–311. [Google Scholar]
  33. Picollo, M.; Casini, A.; Cherubini, F.; Cucci, C.; Grazzi, F.; Stefani, L. Potenzialità e problematiche dell’uso di tecniche di spettroscopia d’immagine per lo studio di pellicole fotografiche. In Memoria Fotografica. Storia di un Recupero Collettivo; Picollo, M., Cattaneo, B., Vianelli, D., Eds.; CNR Publisher: Firenze, Italy, 2020; pp. 43–46. [Google Scholar]
  34. Burger, W.; Burge, M.J. Principles of Digital Image Processing. Core Algorithms; Springer: London, UK, 2009; pp. 97–124. [Google Scholar]
  35. Berns, R.S. The science of digitizing paintings for color-accurate image archives: A review. J. Imaging Sci. Technol. 2001, 45, 305–325. [Google Scholar]
  36. CIE. 15:2004 Technical Report. Colorimetry, 3rd ed.; Commission International de I’Eclairage (CIE), Central Bureau of the CIE: Vienna, Austria, 2004. [Google Scholar]
  37. Marcus, R.T. The measurement of color. In Color for Science, Art and Technology; Nassau, K., Ed.; Elsevier: Amsterdam, The Netherland, 1998; pp. 31–96. [Google Scholar]
  38. Martinez, K.; Cupitt, J.; Saunders, D.; Pillay, R. Ten years of art imaging research. Proc. IEEE 2002, 90, 28–41. [Google Scholar] [CrossRef]
  39. Li, Q.; He, X.; Wang, Y.; Liu, H.; Xu, D.; Guo, F. Review of spectral imaging technology in biomedical engineering: Achievements and challenges. J. Biomed. Opt. 2013, 18, 100901–100928. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Pinto, P.D.; Linhares, J.M.M.; Nascimento, S.M.C. Correlated color temperature preferred by observers for illumination of artistic paintings. J. Opt. Soc. Am. A 2008, 25, 623–630. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Montagner, C.; Linhares, J.M.M.; Vilarigues, M.; Melo, M.J.; Nascimento, S.M.C. Supporting history of art with colorimetry: The paintings of Amadeo de Souza-Cardoso. Color. Res. Appl. 2018, 43, 304–310. [Google Scholar] [CrossRef]
  42. Amano, K.; Linhares, J.M.M.; Nascimento, S.M.C. Color constancy of color reproductions in artpaintings. J. Opt. Soc. Am. A 2018, 35, B324–B333. [Google Scholar] [CrossRef] [Green Version]
  43. Casini, A.; Lotti, F.; Picollo, M.; Stefani, L.; Aldrovandi, A. Fourier transform interferometric imaging spectrometry: A new tool for the study of reflectance and fluorescence of polychrome surfaces. In Conservation Science Conference, Proceedings of the Conservation Science 2002, 18–21 September 2002, Edinburgh, UK; Townsend, J.H., Eremin, K., Adriaens, A., Eds.; Archetype: London, UK, 2003; pp. 249–253. [Google Scholar]
  44. Perri, A.; de Faria, B.E.N.; Ferreira, D.C.T.; Comelli, D.; Valentini, G.; Polli, D.; Cerullo, G.N.; Manzoni, C. A hyperspectral camera for conservation science, based on a birefringent ultrastable common path interferometer. In Optics for Arts, Architecture, and Archaeology VII; Pezzati, T., Ed.; SPIE: Bellingham, WA, USA, 2019; Volume 11058, p. 110580B. [Google Scholar]
  45. Hagen, N.; Kester, R.T.; Gao, L.; Tkaczyk, T.S. Snapshot advantage: A review of the light collection improvement for parallel high-dimensional measurement systems. Opt. Eng. 2012, 51, 1–7. [Google Scholar] [CrossRef]
  46. Jung, A.; Vohland, M.; Thiele-Bruhn, S. Use of a portable camera for proximal soil sensing with hyperspectral image data. Remote Sens. 2015, 7, 11434–11448. [Google Scholar] [CrossRef] [Green Version]
  47. Cucci, C.; Casini, A.; Picollo, M.; Poggesi, M.; Stefani, L. Open issues in hyperspectral imaging for diagnostics on paintings: When high-spectral and spatial resolution turns into data redundancy. In Optics for Arts, Architecture, and Archaeology III; Pezzati, S., Ed.; SPIE: Bellingham, WA, USA, 2011; Volume 8084, p. 808408. [Google Scholar]
  48. Cucci, C.; Casini, A.; Picollo, M.; Stefani, L. Extending hyperspectral imaging from Vis to NIR spectral regions: A novel scanner for the in-depth analysis of polychrome surfaces. In Optics for Arts, Architecture, and Archaeology IV; Pezzati, T., Ed.; SPIE: Bellingham, WA, USA, 2013; Volume 8790, p. 879009. [Google Scholar]
  49. Brandt, E.S. Analysis for spectral sensitizing dyes in photographic films by enhanced Raman scattering spectroscopy. Anal. Chem. 1989, 61, 391–398. [Google Scholar] [CrossRef]
  50. Digney-Peer, S.; Burnstock, A.; Leamer, T.; Khanjian, H.; Hoogland, F.; Boon, J. The migration of surfactants in acrylic emulsion paint films. Stud. Conserv. 2004, 49, 202–207. [Google Scholar] [CrossRef]
  51. Gaspard, S.; Oujja, M.; Rebollar, E.; Abrusci, C.; Catalina, F.; Castillejo, M. Characterization of cinematographic films by laser induced breakdown spectroscopy. Spectrochim. Acta Part B At. Spectrosc. 2007, 62, 1612–1617. [Google Scholar] [CrossRef]
  52. Gschwind, R.; Zbinden, E.; Trumpy, G.; Delaney, J.K. Color negatives at the demise of silver halides. In Proceedings of the Photographic Materials, ICOM-CC, 18th Triennial Conference, Copenhagen, Denmark, 4–8 September 2017. [Google Scholar]
  53. Belkin, M.; Niyogi, P. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2001; Volume 14, pp. 585–591. [Google Scholar]
  54. Belkin, M.; Niyogi, P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 2003, 15, 1373–1396. [Google Scholar] [CrossRef] [Green Version]
  55. Trumpy, G.; Flueckiger, B. Light source criteria for digitizing color films. In Proceedings of the Colour and Visual Computing Symposium (CVCS), Gjovik University College, Gjovik, Norway, 25–26 August 2015. [Google Scholar]
  56. Ciatti, M.; Frosinini, C. (Eds.) Il Gentile Risorto. Il “Polittico dell’Intercessione” di Gentile da Fabriano. Studi e Restauro; Edifir Edizioni Firenze: Florence, Italy, 2006. [Google Scholar]
  57. Sharma, G.; Wu, W.; Dalal, E.N. The ciede2000 colour-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Res. Appl. 2005, 30, 21–30. [Google Scholar] [CrossRef]
  58. Behmann, J.; Acebron, K.; Emin, D.; Bennertz, S.; Matsubara, S.; Thomas, S.; Bohnenkamp, D.; Kuska, M.T.; Jussila, J.; Salo, H.; et al. Specim IQ: Evaluation of a new, miniaturized handheld hyperspectral camera and its application for plant phenotyping and disease detection. Sensors 2018, 18, 441. [Google Scholar] [CrossRef] [Green Version]
  59. Bacci, M.; Picollo, M. Non-destructive spectroscopic detection of Cobalt (II) in paintings and glasses. Stud. Conserv. 1996, 41, 136–144. [Google Scholar]
  60. Picollo, M.; Casini, A.; Cucci, C.; Jussila, J.; Poggesi, M.; Stefani, L. A New Compact VNIR Hyperspectral Imaging System for Non-Invasive Analysis in the Fineart and Architecture Fields. In Proceedings of the Electronic Imaging & the Visual Arts EVA2018, Florence, Italy, 9–10 May 2018; Cappellini, Ed.; Firenze University Press: Florence, Italy, 2018; pp. 69–74. [Google Scholar]
Figure 1. (a) sRGB image of the investigated frame; (b) UMAP image with the selected five sub-sets of fairly distinct pixels (highlighted in red, green, blue, cyan, and yellow); (c) spectra extracted from the data-cube representative of the UMAP five classes reported in (b); (d) displacement of the UMAP five classes on the HSI reconstructed visible image.
Figure 1. (a) sRGB image of the investigated frame; (b) UMAP image with the selected five sub-sets of fairly distinct pixels (highlighted in red, green, blue, cyan, and yellow); (c) spectra extracted from the data-cube representative of the UMAP five classes reported in (b); (d) displacement of the UMAP five classes on the HSI reconstructed visible image.
Sensors 20 02843 g001
Figure 2. From left to right: sRGB of the Lazarus detail reconstructed from the HSI data at the end of the restoration procedure with the spots whose colorimetric values are reported in Table 1; images in grey scale of the b* colorimetric values before restoration; after being restored; image difference: b* colorimetric values (after–before).
Figure 2. From left to right: sRGB of the Lazarus detail reconstructed from the HSI data at the end of the restoration procedure with the spots whose colorimetric values are reported in Table 1; images in grey scale of the b* colorimetric values before restoration; after being restored; image difference: b* colorimetric values (after–before).
Sensors 20 02843 g002
Figure 3. (a) Specim IQ hyper-spectral camera; (b) image displayed in the camera’s screen with the pixel’s reflectance spectrum; (c) cobalt blue distribution map provided by the SAM algorithm in the camera software (in white) on the painting’s surface (figure modified from [60]).
Figure 3. (a) Specim IQ hyper-spectral camera; (b) image displayed in the camera’s screen with the pixel’s reflectance spectrum; (c) cobalt blue distribution map provided by the SAM algorithm in the camera software (in white) on the painting’s surface (figure modified from [60]).
Sensors 20 02843 g003
Table 1. Colorimetric values on three selected spots (average of 5 × 5 pixel, spectral acquisition sampled by 2) calculated from the spectra extracted by the cube-files acquired before (L*b, a*b, b*b) and after (L*a, a*a, b*a) the restoration process. CIE standard 2° observer and D65 illuminant, CIEDE2000 color-difference formula [57].
Table 1. Colorimetric values on three selected spots (average of 5 × 5 pixel, spectral acquisition sampled by 2) calculated from the spectra extracted by the cube-files acquired before (L*b, a*b, b*b) and after (L*a, a*a, b*a) the restoration process. CIE standard 2° observer and D65 illuminant, CIEDE2000 color-difference formula [57].
SpotL*bL*a∆L*a*ba*a∆a*b*bb*a∆b*∆E00
127.2628.050.792.914.171.266.141.99−4.154.24
232.6235.062.4427.8234.977.1520.6629.208.544.64
323.6542.9419.298.8914.335.449.5631.6522.0919.58
Coordinates of the spots in the image: (1) x = 250, y = 239; (2) x = 280, y = 310; (3) x = 340, y = 415.

Share and Cite

MDPI and ACS Style

Picollo, M.; Cucci, C.; Casini, A.; Stefani, L. Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios. Sensors 2020, 20, 2843. https://doi.org/10.3390/s20102843

AMA Style

Picollo M, Cucci C, Casini A, Stefani L. Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios. Sensors. 2020; 20(10):2843. https://doi.org/10.3390/s20102843

Chicago/Turabian Style

Picollo, Marcello, Costanza Cucci, Andrea Casini, and Lorenzo Stefani. 2020. "Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios" Sensors 20, no. 10: 2843. https://doi.org/10.3390/s20102843

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop