Digital 3D Technologies for Humanities Research and Education: An Overview

Abstract

Digital 3D modelling and visualization technologies have been widely applied to support research in the humanities since the 1980s. Since technological backgrounds, project opportunities, and methodological considerations for application are widely discussed in the literature, one of the next tasks is to validate these techniques within a wider scientific community and establish them in the culture of academic disciplines. This article resulted from a postdoctoral thesis and is intended to provide a comprehensive overview on the use of digital 3D technologies in the humanities with regards to (1) scenarios, user communities, and epistemic challenges; (2) technologies, UX design, and workflows; and (3) framework conditions as legislation, infrastructures, and teaching programs. Although the results are of relevance for 3D modelling in all humanities disciplines, the focus of our studies is on modelling of past architectural and cultural landscape objects via interpretative 3D reconstruction methods.

1. Introduction

For more than 30 years, digital 3D modelling and visualization technologies have been widely used to support research and education in the humanities, especially but not exclusively for historical architecture. Despite the immense efforts spent on the establishment of information technologies and in particular 3D technologies for digital 3D modelling and visualization as daily use tools for researchers in humanities, the current situation is still ambiguous. On the one hand, humanities researchers frequently use a wide scope of digital tools for information search, communication, publication, and research support (e.g., reference management or personal organization) [1]. There is also a huge number of projects investigating and utilizing those technologies in various settings.

On the other hand, the use of digital tools for research work differs widely between the individual sub-disciplines of humanities, and the development of that field is driven by language and textually related disciplines like linguistics or edition studies. With regards to visual oriented disciplines like art, architectural history, archaeology, and cultural heritage—as the main target groups of 3D modelling—the affinity and competence regarding digital research methods is—except for archaeology—underdeveloped [2]. Similarly to digital humanities in general [3], the knowledge and skills in practice (e.g., curators) concerning 3D technologies poses a major challenge [4].

1.1. The Purpose of This Article

While technological backgrounds, project opportunities, and methodologies are widely discussed in the literature (e.g., [5,6,7,8,9,10]), key issues are to determine the benefits and limitations of using 3D technologies; to develop and disseminate tasks, user appropriate tools, and processes; and to widely establish competencies for their use. Against this background, the interest guiding this article is to draw the “big picture” and to provide a comprehensive overview on the field of 3D modelling in the humanities and its state of the art. Input came from 15 research and development projects at regional, national, and EU levels carried out between 2010 and 2021 by me or my research group, (single studies are listed in Appendix A and [11]) and an extensive literature review carried out as part of my post-doc thesis. Those investigations were guided by the following questions:

• What marks a scholarly culture concerning 3D modelling and visualization in the humanities?
• What are the scenarios and best practices?
• What are the conceptual, technical, design implications and workflows?
• What are framework conditions as legislation, infrastructures, and teaching programs?

The article is proposed to provide an overview and introduction in the field of 3D modelling for learners and scholars in digital humanities and heritage studies. Since there are already various studies published to provide a comprehensive view on 3D modelling, the added value of this article is to include a domain-overarching perspective. Although the focus is on modelling of past architectural and cultural landscape objects via interpretative 3D reconstruction methods, the described findings are of relevance also for scenarios employing 3D technologies in the humanities. As this article focuses on the scientific use of 3D models, it only sparingly covers the use of 3D models of cultural heritage for other purposes like entertainment. There are numerous investigations about the use of 3D modelling of cultural heritage in films [12] and digital games [13,14,15]. The latter context in particular poses various specific challenges to 3D modelling, such as the need to create a playable environment known as ludoforming [16,17,18] and motivational and instructional design [19,20,21].

1.2. The Structure of This Article

Main dedication of this article is to structure and review topic areas related to digital 3D technologies in the humanities and introduce related key concepts, overarching technological and methodological settings, and recent developments. It therefore connects and continues several research and networking activities on those topics—that related work is discussed in Section 2.

The reviewed areas (Figure 1) following from Section 3 include (1) definitions and key concepts of relevance, (2) the epistemic function and methodological setting of 3D modelling, (3) scientific communities, (4) workflows in 3D modelling. We investigated (5) technologies to enhance the creation of 3D models and (6) 3D visualization technologies and the perception of 3D models. (7) Documentation of 3D models and processes has a focus on standards. (8) Research on framework conditions primarily focuses on legal requirements and the education of 3D modelling competencies and recommendations for repositories. The article is concluded by Section 11 with implications and prospects.

2. Related Work

2.1. Background

For a long time, funding priorities in the EU and Germany focused primarily on technological aspects such as cost minimization, ease of use of software tools for creating digital 3D models [6], or application (e.g., funded by the Creative Europe Programme [22]). Closely related is the focus on digital research environments for digital humanities: An overview of infrastructure projects in cultural heritage is provided by [23,24] and a general overview on European digital preservation activities until 2006 in [25]. While research infrastructures as DARIAH and CLARIN (recently merged into CLARIAH) develop and operate virtual research environments for humanities scholars, especially for text-related research approaches, ARIADNE, CARARE, and adjacent projects are dedicated to supporting archaeological information management on a European level [26,27]. The EUROPEANA virtual library and its sub-projects [28] are dedicated repositories for digital cultural heritage assets in Europe; they collect and aggregate institutions like museums, libraries, and archives [29] (a critical appraisal of the EU attempts at digital libraries: [30]). The complementary E-RIHS infrastructure initiative focuses on infrastructure for heritage sciences [31].

Former funding programs have not fully recognized the fact that digital 3D technologies are applied in complex socio-technical arrangements. A paradigm shifted in funding politics since 2010. Since then, besides development of technical infrastructures like research environments or digital repositories, human resources, and transnational knowledge exchange and cooperation, social and economic impacts, and its valorization and dissemination have become increasingly important objects of funding [32]. An evaluation of the FP5–7 research funding programs stated that the aim to “foster the dissemination, transfer and take-up of program results” was underrepresented in these programs [6]. Related to development in the field of cultural heritage, the Horizon 2020 work program and its succeeding Horizon Europe program aim for “an understanding of Europe’s intellectual basis” and the usage of “new technologies […] as they enable new and richer interpretations of our common European culture while contributing to sustainable economic growth” ([33], p. 5) as well as to develop innovative research infrastructures to foster research, education, and publication of “knowledge-based resources such as collections [or] archives” to a European audience ([34], p. 4, see also [35,36]). Recently, 3D modelling of cultural heritage came in focus of various attempts as the digital turn of the cultural heritage sector after the COVID-19 pandemic [37] as well as the strengthening of the digital market by the Digital Europe Data Space [38].

2.2. Current State of Empirical Research on 3D Technologies in the Humanities

2.2.1. Research on Structures and Topics

Various attempts have been made during the last decades to evaluate, quantify, and qualify the usage of digital 3D modelling for particular fields of humanities by using social empirical methods. Most of these approaches focus on a qualitative analysis, e.g., by expert boards or surveys. The EPOCH network of excellence (2004–2008) employed focus group discussions and perspectives on digital 3D techniques in cultural heritage studies [5]. While qualitative approaches are appropriate to identify and explain [39] phenomena in terms of evolutions, current states, and perspectives, they show only limited usefulness for quantifying uncovered phenomena or investigating scientific structures. The VIA project organized a series of workshops and questionnaire-based surveys to investigate a situation specifically on visualization in archaeology in the UK [40]. From 2012, the Enumerate project has performed bi-annual monitoring of digitization activities of cultural heritage institutions within the EU area—primarily focusing on museums and archives [41,42]. Vice versa, the DARIAH DIMPO workgroup is periodically monitoring the digital humanities community [43]. Recently, several monitoring actions were conducted by the European Commission (c.f. [44,45]), e.g., to investigate a state of digital competency in cultural institutions [46]. Against the background of the COVID pandemic, several associations surveyed the consequences for cultural institutions and their digital transition (c.f. e.g., [47], an overview: [48]). Research regarding scholarly behavior often relies on analyzing the publication record. With regards to a scholarly area of visual digital humanities and its adjacent fields like digital heritage, Hicks et al. [49] stated that publication and research habits are widely spread between single disciplines in the (digital) humanities. Similarly, Leydesdorff et al. [50] examined the disciplinary canon in humanities and digital humanities by employing bibliometric methods. With regards to a scholarly community within the digital humanities, Terras [51] reported that until 2006, especially US-, Canada-, and UK-based researchers contributed to academic discourse. Similarly, Grandjean performed a social network analysis of Twitter to map the digital humanities community [52]. Specifically for digital heritage, Scollar [53] investigated the Conference on Computer Application in Archaeologies (CAA) from 1973 until 1996. Secondly, information habits of visual digital humanities scholars are the focus of various studies. Since older investigations found large differences in information behavior between scholars in different disciplines [54], nowadays, many scholars in art history and architecture rely heavily on digital information and perform visual search strategies [55,56].

2.2.2. Research on Epistemics

Investigations on epistemic settings in visual digital humanities are widely driven by researchers originating in humanities and mostly focus on exemplification and problematization within a certain disciplinary context. With regards to the use of digital methods in art history, Drucker [57] sketches a historical evolution and current state of application of digital methods in the humanities. Kohle defined fields of supplement by digital tools and practices in art history [10] and Heusinger for a general visual humanities research process [58]. Similarly, many texts describe a comprehensive state of the art and methodologies for digital archaeology (e.g., [15,59]). Furthermore, there are many standards, guidelines, and rules for dealing with historical content [60,61,62,63,64]. An adjacent issue is general workflow modelling of archaeological reasoning. Against this background, Barcelo discusses various approaches for computable reasoning and artificial intelligence to support archaeological reasoning [65]. Some meta-reviews on similar aspects in museology exist (e.g., [66,67,68]) and methodological overviews are available for adjacent disciplines, such as game research [69], editorial studies [70,71,72], or graphic design [73]. In contrast to philosophical approaches, there is relatively little empirical research on practices and users of digital 3D modelling in the humanities (c.f. [67]). As an example, Huvila investigated user roles and practices in archaeology [74,75] as well as certain practices within the ARKDIS project [76]. Another empirical perspective is the research on usability and requirements for software design for humanities researchers, which were investigated within the VERA project [77,78].

2.2.3. Research on Modelling Technologies

The research on technologies for 3D modelling and especially 3D digitization has been important for decades. In recent decades, various large-scale projects on the EU level on 3D modelling in cultural heritage (examples in the Horizon 2020 program are: INCEPTION; FP7: 3D-COFORM; 3D-ICONS; FP6: EPOCH, Aim@Shape) primarily focused on 3D digitization from contemporary survey data. A major objective during the FP7 to Horizon 2020 framework programs was to ease use, reduce costs, and increase quality. With the following transition to the Horizon Europe framework, valorization and wide use became the focus. As recent projects on the EU level, the VIGIE study [79] examines use cases for 3D/4D digitization of tangible heritage and the EU DT–20 competence center [80] investigates and develops support structures for 3D digitization from contemporary survey data. Various overview articles were published to provide an overview on particular technologies of relevance for 3D modelling and visualization, recently for instance on laser scanning [81], photogrammetry [82], machine learning [83], and extended reality technologies [84].

3. Definitions

In the following sections (c.f. Figure 2), the key concepts of 3D modelling in the humanities are introduced (other articles on terminology: [85,86]). (Visual) digital humanities—besides digital heritage and digital disciplines in the humanities such as digital art history or digital archaeology—marks the disciplinary space in which 3D modelling and 3D reconstruction in the humanities are discussed and methodologically anchored. Objects of 3D modelling are cultural heritage objects. The key focus of this article is on knowledge and knowledge-related effects—e.g., knowledge generation and transformation (research), transfer (teaching), and preservation (documentation).

3.1. Visual Digital Humanities as an Investigated Field

3.1.1. Visual Approaches in Digital Humanities

Despite various attempts [87,88,89,90,91], the definition of digital humanities is still blurred and heterogeneous [90,92]. From a historical perspective, the digital humanities have evolved since the mid-2000s through the development of an independent epistemic culture from the historical computer science and “humanities computing” [93,94,95,96,97]. There is also a broad consensus that digital humanities deal with “the application of technology to humanities work” [92]. The data foci of digital humanities are texts, audio-visual content, images, and objects. While the use of digital methods in the text-oriented disciplines is currently widely established and standardized ([98], p. 10), the scope of digital methods related to images and other visual objects based on vision rather than close reading remains—despite various attempts [5,7,99,100,101]—essentially uncharted. Possible reasons may be the “diverse nature of the methods used” in disciplines focusing on these types of artifacts ([102], but also the heterogeneous level of establishment of digital research methods in those disciplines [49]. Although all disciplines in the humanities are dealing with vision and visualization, some disciplines are particularly engaged here:

• Art and architectural history studies investigate mainly works of art and architecture from the late antiquity to the modern age (e.g., [103]) to provide insights into their origin and meaning [104]. Methods for investigating genetic and morphologic connections are covered by analyzing style (c.f. [105]) and structure (c.f. [106]). Another important range of methods is concerned with the meaning of the works of art (iconography) and systems of meaning (iconology) (c.f. [105]).
• Museology focuses on the presentation of cultural heritage, collection and educating of the public (c.f. [107]). Digital technologies are used to enhance museum visits, e.g., as visitor information systems and didactically enhanced applications. Other scenarios are virtually accessible collections and virtual museums, which means virtual collections and exhibitions with no counterpart in the real world (c.f. [108]).
• Archaeology investigates tangible remains and evidence of human culture ([109], p. 11) to generate a representation of what exists now and closely approximate what may have once been [110]. Often, the physical preservation of the archaeological site is not possible. Therefore, thorough documentation and data collection is highly relevant. Surveying techniques [111,112,113,114,115] and traditional photos and plans are used to document excavations.
• Heritage studies comprises a variety of approaches dealing with human culture and behaviour related to heritage [116,117]. Relevant strands are derived from humanities, social sciences, design and engineering, most frequently anthropology, history studies and architecture (c.f. [118]).

3.1.2. A definition of Visual Digital Humanities

A common bond in disciplines dealing with vision in the digital humanities may be their grounding in visual literacy as “the abilities to understand (read), and use (write) images (and spatial objects), as well as to think and learn in terms of images (and spatial objects)” ([119], p. 26). Against this background, “visual digital humanities” cover research approaches in the digital humanities dependent on both consuming and producing pictorial and spatial, rather than textual, information to answer research questions ([120]; other definitions: [121,122]). Visual digital humanities encompass the computational supported research on complex visual information to treat research questions and interests from the humanities, concerning aspects of:

• data collection, e.g., through digitization;
• data retrieval from database records with the transfer of knowledge;
• examining visual humanities questions, e.g., a composition of complex figurative paintings;
• reconstructing, simulating, and producing objects;
• administering and organizing people and objects ([9,58]).

For visual digital humanities a range includes the analysis of complex visual information, their collection and semantic enrichment, and the creation of imagery in the context of:

• image analysis (e.g., pattern analysis of large-scale image collections);
• perception-based techniques (e.g., visuospatial analysis of architectural objects);
• spatial modelling (e.g., 3D reconstruction of historical architecture);
• visualization (e.g., sketching for visuospatial reasoning).

Objects are cultural heritage artifacts and images, and scholars in visual digital humanities use technologies to “understand (read), and use (write) images [and spatial objects], as well as to think and learn in terms of images [and spatial objects]” [123].

3.2. Cultural Heritage as a Research Base

3.2.1. Types of Cultural Heritage

Cultural heritage can be understood as traces and expressions from the past as attribute values and is used in contemporary society (c.f. [124]). Cultural heritage could be regarded as the only legacy that cannot be inherited; instead, it must constantly be acquired [125]. Since cultural heritage traditionally focuses on tangible objects, a broader understanding adds intangible heritage (e.g., dances, customs, workflows) and natural heritage. Although there are some great examples for 3D modelling of intangible heritage, for instance regarding customs and daily life [126,127] in past cultures (a recent overview: [128]), the focus of 3D modelling in this article is on tangible heritage. Another concept of relevance is digital (cultural) heritage, which includes materials like texts, images, etc., created digitally or digitized, as well as digital resources of human knowledge or expression (e.g., cultural, educational, or scientific) [129]. This latter facet also comprises various digital technologies to study cultural heritage. Due to this scope digital heritage is seen as an “agora” [101]. Around those topics, various scholarly communities have formed during the last few decades. In our previous research, we attempted to determine the boundaries of digital heritage studies as a scholarly field—with adjacent scholarly fields such as digital humanities, digital archaeology, or digital history studies, or concerning related scholarly communities [44]. Digital heritage comprises technologies to preserve, research, and communicate cultural heritage (c.f. [130]).

3.2.2. Sources

The concept of sources is predominant for any study of history in the humanities [131]. Sources are not a specific class of cultural heritage items, but all heritage items provide information about past events, phenomena, or objects. Consequently, historical research into those entities is based on sources. Sources are always biased, by the creators’ intention or limited view of history, and understanding by sources is therefore never a neutral act, but requires critical reflection [132], as formalized by source criticism (c.f. Chapter 4). There are classifications of sources: for instance by the type of textual, image, or audiovisual media, or object sources such as physical remains of buildings [133,134]. Another main distinction is between primary and secondary sources. Primary sources date from the time period being studied or were created by participants to describe the object being studied. They are subjective and reflect the individual view of the author [132]. A secondary source interprets or analyses a historical object, event, or phenomenon (e.g., a scientific text about an architectural object). Occasionally, tertiary sources are mentioned as another concept—including material based on secondary (or primary) sources, e.g., compilations, collections or (digital) repositories of sources [135,136]. How extant the heritage objects are is of interest in 3D modelling. As discussed next in Section 3.3, a main distinction should be drawn between (a) the 3D digitization where an extant cultural heritage object is a source, and (b) 3D reconstructions, where the modelled object can only be envisioned through other sources describing it (e.g., planning documents describing a never realized architectural building, destroyed or altered objects) [137].

3.3. 3D Modelling Technologies as a Research Object

3.3.1. Modelling

Model theory: Modelling means creating “the (simplified) replica of an original system”, which must be “sufficiently similar to the original system with respect to the purpose of its realization” ([138], pp. 131; see also [139], footnote 16; [140]). In the 3D/4D modelling in humanities, this model is created “post factum”—after the original ([141], p. 335). Unlike architectural drafts, these models are created subsequently to the original. An established explanatory scheme of models is the general model theory. In this context, a model represents a simplified or reduced version of an original. An element of pragmatism denominates the subjective and purpose-oriented nature of a model [138]:

• Feature of reproduction: models represent the originals that come from the world of imagination, from expressions, as well as from physical objects and symbols.
• Feature of reduction: models usually do not include all features of the original but only those considered relevant by the creator or user.
• Feature of pragmatism: models function as a surrogate of the original subjects within a certain time span, for a certain purpose (transactions).

Computer-based 3D modelling: Computer-based, i.e., digital, 3D technologies have become increasingly important for sustaining conservation, research, and broad accessibility of cultural heritage as knowledge carriers, research tools, learning materials, and means of representation over the last three decades (c.f. [142,143,144]). Theoretical foundations and epistemological recommendations of 3D modelling of cultural heritage have been studied for a long time, e.g., within various EU projects [6,145]; on a national level in Germany by the task group for 3D reconstruction of the DHd association [146], the Digital Art History workgroup [147], and the DFG Network for 3D reconstruction of architectural history [148], and by a large number of publications (most recently: [149,150,151]). An overarching consensus is that 3D modelling represents or translates either a material cultural object or an intangible cultural phenomenon into a spatial, temporal, and semantic virtual model. Main differences refer to the kind of object of assessment in terms of material and immaterial objects (e.g., usages or digital data). Furthermore, there is an essential difference between the reconstruction of objects that are no longer existent or that have never been realized (e.g., the current status of unrealized plans) and the digitization of objects that are still existent [152].

3.3.2. Digital vs. Virtual

There is a basic consensus that digital 3D models are created using computers and thus differ from physical reconstructions of artifacts or even paper-based reconstructions such as architectural drawings (c.f. [153]). In this context, the terms “digital” and “virtual” reconstruction are largely used interchangeably, although the underlying concepts of the digital as “data in the form of especially binary digits” [154] are quite different from the virtual as “existing or occurring on computers or on the Internet” [155]. While the digital thus describes a materiality, the concept of the virtual is based on a reference to reality in terms of content. Empirically, in the German-speaking world, the term virtual reconstruction is used more frequently, with 61.000 results compared to 13.000 results for “digital reconstruction” at Google search [156], while “digital reconstruction” predominates in English with 484.000 vs. 181.000 results. In accordance with the practice described, both terms will also be used synonymously in this article (c.f. [157]).

3.3.3. Reconstruction vs. Digitalization

3D digitization or reality-based modelling: 3D Digitization or reality-based modelling [84,158] stands for “the process of converting something to digital form”. [159]. Digitization describes the technological transfer of a real object to a digital asset. For this purpose, various data acquisition technologies are used (c.f. [157,160,161]). For tangible objects, a main distinction is between light-dependent and light-independent methods [162]. Light-dependent systems emit light to retrieve information about the 3D surface. This comprises active approaches where coded light is projected onto a surface (e.g., white-light scanners using structured light to determine the surface shape and laser scanning sending laser beams at a varying angle to determine 3D surface points using the time-of-flight principle [81,163]) and passive methods using imagery as videos or photos [164] without specifically coded light [162,165]. The outcome is a dense 3D point cloud, which is then processed into a meshed surface [166]. Methods not using visible light comprise a large variety of tomographic methods such as CT scanning to model the internal structures or volume of the object [163]. A taxonomy of the quality of 3D digitization of tangible heritage was recently developed within the EU VIGIE study [79]. According to this study, the main attributes describing a digitized object are geometry, composition (as material information), and production (as the model acquisition process) (c.f. [167]).

3D reconstruction or virtual-based modelling: Whereas digitization refers to the technological conversion of an object into a digital representation, a digital reconstruction or virtual-based modelling process [84] requires the human interpretation of data to create a hypothesis of a past object (c.f. [152,168,169,170]). The model is then mostly created on the computer using manually controlled computer graphic modelling software originating from construction and engineering in the case of computer-aided design (CAD), or from design and creative industries by computer-generated imaging and graphics software (CGI). Since those processes are highly labor-intensive, approaches to reduce the required work include generative or parametric modelling (predefining objects by rulesets with changeable parameters) or semiautomated modelling (e.g., from historical imagery) [152,169].

3D meshing and texturing: Both, 3D digitization and reconstruction lead to 3D models representing surfaces and/or volumes of a tangible heritage object. Since modelling approaches of volumetric models vary significantly depending on the methods used [171], the 3D surface representation can be based on points (point clouds), triangular meshes (vertices, edges, faces), curves and shapes (generative approaches as NURBS), geometric solids (constructive solid geometry, CSG), and boundary representations (B-reps) (cited according to [163]).

Besides the geometry features of a 3D model, its radiometric parameters and materiality representation are relevant [85]. For surfaces, a main distinction is between synthetically generated procedural textures and reality-based textures. Related to the latter category, another distinction is between the acquisition of material properties [172] and digital visualization [173].

Simulation: Simulation refers to a “procedure for reproducing a system with its dynamic processes in a model that can be experimented with in order to arrive at findings that can be transferred to reality” (translated from [174], p. 1; a definition from humanities: [175], see also [176]). Law and Kelton [177] distinguish three types of simulation depending on the models used:

• Static vs. dynamic: A static simulation provides a replica of a system at a specific point in time. A dynamic simulation represents a system whose states, attributes, processes, etc. are time-dependently variable [178].
• Discrete (countable) vs. continuously (measurable) values.
• Deterministic (always the same output for a specific set of values) vs. stochastic (random output at a certain level) values.

In the context of cultural heritage, the term “simulations” is used in various ways:

• Simulation as visualization (e.g., [179]): Simulation means the computed combination of various features as material appearance, lighting, and geometrical behavior, either of static scenes (as images) or of time variate or dynamic (as films or interactive games) to a visual output. This refers to rendering as computation of a virtual model with the purpose to create a visualization [180].
• Simulation as an analytical instrument. Examples include:
  • Object behavior, often in disaster situations, as fluid simulation (CFD) to analyze flooding [181], simulations of structural behavior, e.g., in case of monuments in earthquakes [182] or fire simulation [183], but also to prove requirements for materials in construction [184].
  • Simulation of climate effects, e.g., lighting simulation to assess conditions in historic buildings [185,186] or degradation by climate features [187], or simulations of ancient ventilation systems [188].
  • Simulation of cultural effects as crowd simulation [189], mechanical processes [190], or daily cultural life [126,191].

3.4. Knowledge as a Theoretical Approach

As mentioned before, the creation and reception of digital 3D reconstructions and the visualization of visual humanities objects are based on complex sociotechnical interaction processes. According to Barcelo, we “do not understand past social actions by enumerating [all possibilities]” ([65], p. 414), but need a linkage between digital tools and human interpretation. Against this background, our research focuses on aspects of knowledge transformation and transfer, within and between humans as well as between humans and—as data—computers. This is closely related to concepts of intrapersonal knowledge such as reasoning or memorization, as well as for groups of people, their communication, and joint mental modelling [192]. Moreover, concepts like “visual reasoning” or “embodied knowledge” [193,194] focus on an object that contains and represents knowledge, e.g., architecture. A well-established and hierarchical classification of information and knowledge is the distinction between signs, data, information, and knowledge (

Table 1. Knowledge pyramid (c.f. [195,196]).

Sign
Data	Perceptible signs
Information	Data which are useable for an individual
Knowledge	Information that is relevant and mentally connectable for an individual

). In this definition, knowledge does not only include a perception and cognition of signs, but also aspects of their relevance and mental connection ability for a recipient [195].

Why should we focus on knowledge? At a glance, the main intention of both research and education is to gain and transfer knowledge. According to Müller [197], knowledge in the context of visual media relates to (1) the production of visual media, (2) the visual medium as an object, and (3) the reception of visual media. Digital 3D reconstruction models, as well as visualizations, act as (1) boundary objects—in terms of cross-culturally understandable media (c.f. [198])—for research and communication in the context of visual humanities and (2) embody substantial knowledge—in terms of psychological-physiological cognition, i.e., on proportions or dimensions of objects—when creating models and visualizations [199]. From a theoretical perspective, both aspects are closely related to semiotics and model theory. According to semiotics, all visual entities represent a certain meaning [200]. Specific geometric shapes are recognized by humans as letters and words with a certain meaning. Graphical shapes like arrows direct human vision. These effects do not only rely on individual cultural or professional backgrounds—i.e., an archaeologist focuses on other aspects when seeing a depiction of an ancient shrine than an architectural engineer does [201,202]. In addition, they are influenced by the contexts or “frames”, as dependencies of content and context of visual communication media (definition: [203]; current research perspectives: [204]). The visual asset is presented in as well as by knowledge embedded or “embodied” in visual media. Lastly, visual perception and reasoning are highly influenced by the properties or “Gestalt” of visual assets [205]—e.g., its color or shape—and related level of abstraction [206]. While semiotics focus on function and Gestalt on signs, model theory—as mentioned in the previous paragraph—focuses on the relation between an original and a model as its “abstraction” [207].

4. Epistemics and Methodology

4.1. Theoretical Background and Methodology

Science and technology studies provide several approaches focusing on historical, philosophical, and sociological aspects [208,209], and empirical methods for investigating researchers and academic fields. A prominent method characterizes fields of research by specific epistemic cultures in terms of “architectures of empirical approaches, specific constructions of the referent, particular ontologies of instruments, and different social machines” ([210], p. 3), techniques to gain insights, vocabularies, publication bodies, and habits [211]. According to this approach, scholarly fields “(a) have a particular object of research […], (b) have a body of accumulated specialist knowledge […], (c) have theories and concepts […], (d) use specific terminologies […], (e) have developed specific research methods […], and (f) must have some institutional manifestation in the form of subjects taught at universities or colleges” ([209], p. 9). Shared narratives are important facilitators of a disciplinary culture, which has been investigated in digital humanities (c.f. [97]). On a more operational level, the community of practice approach introduced by Lave and Wenger [212] defines these communities as marked by mutual engagement, a joint enterprise, and a shared repertoire of knowledge and culture [213]. Modes and modalities of the genesis and transformation of scientific insights and “paradigms” [214] are a core topic of the philosophy of science [215,216] and especially in digital humanities numerous articles has investigated their epistemic modes (e.g., [217,218,219]). To investigate an epistemic culture and modes of research for 3D technologies in the humanities, we employed literature reviews as a low standardized method to gain an overview on a specific topic [220,221] in combination with expert opinions gained via group discussions. This chapter highlights the purposes and epistemic challenges for 3D modelling in the humanities (c.f. Figure 3).

4.2. A Brief Historiography of 3D Reconstruction and Modelling

3D modelling involves established methods that were utilized in history studies long before the advent of computer-aided visualization techniques [222]. As early as the Renaissance, scholars studied the appearance of the architecture of the past, analyzing it by means of images, and using it in their creative processes as a model for constructing their own contemporary buildings [153]. As architectural history became established as an academic discipline, reconstruction gained new importance, especially with regard to architecture that had been lost; studies were made of the appearance of the Late Antique Basilica of St Peter in Rome, which had been demolished in 1514 [223,224,225], the early construction phases of the Cathedral of Santiago de Compostela [226,227] or, as a prominent present-day example, the Berlin City Palace [228,229]. Such traditional reconstructions are prompted by questions as to their original appearance, often posed as issues in the field of buildings archaeology, which cannot be verified through in-situ observation. They may also serve—as in the case of the Berlin City Palace—as the basis for an actual architectural reconstruction. The advent of (visualization oriented) 3D modelling techniques was initiated primarily by 3D reconstruction in the early 1980s [230,231], while 3D modelling via the digitization of heritage objects became mainstream in the early 2000s [232].

4.3. Purpose of 3D Modelling

A main distinction of 3D modelling by purpose is between preservation, research, and education of cultural heritage (c.f. [130]), although these concepts are often closely interlinked.

4.3.1. Education

In education, 3D models of cultural heritage are used in various settings:

• The predominant purpose still is to illustrate and teach history and heritage in informal settings like museums, games, or television broadcasts [12,233]. Especially, the role and effects of media like visualizations in museum education is the focus of scientific discourse (e.g., [78,234,235,236,237,238]), as are various settings employ 3D models (e.g., [239,240,241,242]). As one example, interactive applications for 4D city exploration [243,244,245] allow virtual visits and remote spatial learning [246], support guiding visitors through the city [245,247,248], provide access to additional information, and enable users to gather a virtual view of temporal change and historic spaces, buildings, and monuments or covered parts [248,249,250,251,252,253,254].
• Teaching heritage in formal education is another highly important scenario. A multitude of projects employ 3D models in higher education facilities (e.g., [255,256]) or evaluate the use of 3D models of cultural heritage in school education [257,258].
• Teaching digital competencies via heritage: In the third usage scenario, the primary subjects of teaching are digital literacy and competencies as modelling techniques or VR technologies, while the historical object is used “only” as a training example. This closely relates to the concept of data literacy comprising data collection, exploration, management, analysis, and visualization skills [259]. Data literacy has been defined by various recommendations and standards [260,261].

4.3.2. Research

Concerning research functions of 3D models, these may be distinguished by the typology shown in

Table 2. Research contexts of digital 3D models.

Research Contexts	Source	Object	System
Data quality assessment (e.g., consistency of sources)	X
Visualization (e.g., investigation of shape, or appearance)		X
The creative process (e.g., planning or construction)		X
Conceptualization and contextualization (e.g., typologies, functional segments, archetypical elements, provenance)	X	X	X
Numerical analysis (e.g., structural analysis, lighting)		X
Hypothetic simulation (e.g., of hypothetic objects deriving from an architectural system)			X

, which distinguishes between research objects and objectives [262,263].

Research object: Use of 3D digital reconstruction to research a certain historical object or its parts is common. Three-dimensional reconstruction is also employed to investigate and assess sources. Sometimes not a specific object, but an investigation of schemes and systems is the focus of research, for example, to evaluate the Vitruvian scheme of architectural orders. Against this background, 3D reconstruction methods are often employed to derive archetypes or specific features [264]. The question concerning the “original” being reconstructed is closely related. The “original” can be a certain intention (e.g., of a builder), a specific source, or a historical object. Research objectives include:

• Data quality assessment: This is closely related to the contextualization and assessment of the consistency of sources. For example, digital reconstruction of content depicted in drawings or paintings can be used to test perspective features or consistency (c.f. [265]).
• Visualization: The most common way to visualize is to formulate a hypothesis of the shape, properties, and appearance of a certain historical object. Concerning this aspect, digital reconstruction allows the non-invasive application and testing of alterations or restoration (e.g., for destroyed statues [266,267,268]).
• Process investigation: This includes research into historical preparation processes (e.g., planning or construction processes employed by artisans).
• Conceptualization: A major question for underlying concepts and intentions, e.g., structuring concepts (c.f. [269]), refers to functions of certain parts of an object (e.g., rooms, figuration, or proportions) (e.g., [270,271]).
• Contextualization: The contextualization of objects (e.g., geo-location, relationship to other objects) and identification of archetypal characteristics may refer to artisan specifications and typologies, as well as comparison of iconographical concepts. Contextualization may lead to interest in sources, specific objects, or systems (c.f. [10]).
• Numerical analysis and simulation: This is a frequent use case, especially for 3D digitization models (c.f. e.g., [272]). Occasionally, 3D reconstructions are employed to simulate and analyze no longer extant heritage (c.f. Section 3.3) and structural analysis is one area of application (c.f. [273]).
• Hypothetical simulations: Different usages are possible without making reference to concrete historical objects, e.g., the exploration of hypothetically possible objects, which derive from a certain architectural order and the (hypothetical) limits and boundaries of this system [264,274] or the analysis of perspective drawings like MC Escher’s [275] impossible objects (c.f. Figure 4).

4.3.3. Preservation of a Specific State

3D Digitization techniques are frequently used in preservation (c.f. [276]) to:

• Preserve object properties in case of damage: Recent examples are the recording of surface and material properties prior to an unplanned disaster (currently prominent example: Notre Dame in Paris [277]), in case of planned destruction, but also post factum from extant images (e.g., the destroyed sites in Palmyra and Bamyan [278,279]). In digital 3D reconstruction, the objectives of a virtual model are primarily to assort, store, and compile spatial-related knowledge (c.f. [139]). For example, the 3D model of the Domus Severiana provided a spatial map and thus a possibility to geo-reference sources [280].
• Create replicas: Especially in museums, artifacts are frequently replaced by replicas created in fully digital workflows [281].
• Identify art forgery by creating a 3D digital footprint of the original [282,283].
• Collect information difficult to obtain from the real object as geometric or radiometric measures of artwork due to size, the risk for the artwork, etc.
• Monitor cultural heritage, e.g., for condition assessment [284] and degradation analysis [285,286,287].

4.3.4. Knowledge Organization

3D models are used in various disciplines to structure knowledge and information. (a) Digital inventories as in the spatial book (Raumbuch) approach [280,288] in archaeology and similar approaches for monument documentation [289] focus on spatial organization of digital data—in Raumbuch, related to archaeological excavation. (b) Digital information spaces like Digital Twin in manufacturing [290] extend the inventory by fully digital simulation workflows but focus on contemporary data. (c) 4D models, e.g., city models, add a temporal layer to organize data in a 4D inventory (e.g., [291]). Finally, (d) the Mirrorworld approach describes a universal data integration principle, but is still hypothetical [292]. Those concepts relate to numerous overarching standards and protocols for organizing 3D cultural heritage objects and inventories, e.g., BIM [293]/HBIM (amongst many: [294,295,296,297,298]) for architectural models and 3D-GIS [299] and CityGML [300] for geo and city-scale models.

4.4. Epistemic Challenges of 3D Modelling in Humanities

What epistemic challenges are related to 3D modelling? The analogue research process in history studies traditionally comprises the investigation of a historical object either directly or via historical sources, which researchers inspect visually to answer a specific research question (cf. Figure 5). The virtual 3D techniques (Figure 6) add the transformation layers of digital modelling and computer graphical visualization—potentially losing, altering, or adding information, e.g., by interpretation or selection. This increases the complexity of the research process and potentially causes additional bias in history research due to non-transparency and fuzziness. The 3D reconstruction of no longer extant objects adds another main issue. Digital 3D modelling approaches like CAD strive to show a holistic building and “make it hard to be vague” ([301], cited according to [139,302], also [139,302]) by requiring exact measurements and 3D shape information of all architectural parts. Consequently, 3D reconstructions force their creators to complete missing information that available sources—potentially biased, incomplete, and low-quality—cannot provide, e.g., about parts of buildings not shown in images or impossible to read. This approach contrasts with the problem-centric approach in history studies, the leading paradigm for over 50 years, where the attempt to “show how it actually was” [303] has usually given way to consequently basing research on available historical sources (e.g., [304,305]). A consequent issue is how to overcome this clash between sparse, questionable, partial, or missing historical information and the demands for holistic information to achieve digital 3D reconstruction of past architecture.

4.5. Visual Research Processes and 3D Modelling

Despite several visual approaches to art and architectural history research, such as style criticism as an examination of genetic and morphological connections ([105], p. 20); iconography and iconology. Exploration of the content or symbols behind the visible forms [105,306]; or structural analysis ([105], practices are highly researcher-specific and experience-based. They have not been connected in an overarching methodology [56,307]. Generally, research about the use of images is nothing new and has been utilized in various contexts like engineering, design, architecture, or science. Visual media greatly support the process of reasoning or idea forming within research processes [194] and enable a deep learning effect [308]. Such aspects are theorized in several approaches, such as visual decision making [309] or visual learning theories [310,311,312]. Regarding the quality of images as visual signs, there are many possible dimensions, such as similarities to a depicted object, visual styles, or creation processes [313]. The perception and visual reasoning of art historians [307,314,315] and archaeologists [201] have been subject to many empirical studies. However, most empirical investigations on architectural perception do not distinguish between laypersons and experts, and if they do, the expert group are architects (e.g., [316]) rather than architectural historians. Images as sources and their relevance for 3D reconstruction are a prominent topic in academic literature [317,318], and images are the most prominent type of sources for reconstruction projects [137]. Other than in text-related disciplines, knowledge is mainly gained by the creation of a virtual model and its digital (in most cases, visual) demonstration in the case of digital reconstruction. Such media include the contributions of different authors and a multiplicity of intuitive decisions and expertise [24]. So far, neither an academic culture nor mechanisms have been established for making digital models and generated images scientifically linkable and discussable. This includes the capacity to quote parts or areas in models and images, and the modification of such media by others. In addition to a number of technical requirements, approaches are being developed to document processes and their results, as is the capacity of making a model logic transparent [263,319].

4.6. Scientificity and Scientific Values

Values in science have long been discussed from a philosophical, sociological, and historical perspective. Within this, both the “value of freedom from values” (c.f. [320]) and the absence of value(s) (c.f. [321]) have often been levelled from both an epistemological and an empirical perspective, e.g., in the positivism controversy [322]. In addition to the well-known “technical principles” of objectivity, reliability and validity of scientific work (e.g., [323], p. 22), scientific knowledge must fulfil the “ideals” of “truth”, “explanation and understanding”, “justification”, and comprehensibility ([324], p. 17). A similar long-running discourse on architecture and art history—especially digital images—has considered specific scientific access and value, the “tendency towards fragmentation and small form as well as the prevalence of the quantitative over the qualitative” [325]. Scientific values implied in 3D models in humanities include:

• Authenticity (relationship to the object)
• Hypotheses or modelling (relationship to the research thesis)
• Plausibility (relationship to the cognitive process)
• Recognizability (relationship of the above values to form/design)
• Flawlessness (problems of above-mentioned values in relation to form/design)
• Immersiveness (relationship of object to viewer by means of visualization)
• Source fidelity (problems of object to viewer due to visualization)
• Correspondence with textual explanations (problem awareness through image-text direction)

5. Scholarly Community

5.1. Theoretical Background and Methodology

An investigation of scientific structures is proposed concerning cohorts of authors and their linkage in a scientific community (c.f. Figure 7). One aim is to identify and classify topics of interest, as well as their temporal transitions. Thus, an important object of study is author cohorts of publications. This includes various research approaches calculating key numbers, such as an average count of authors per publication or a rate of publications authored by single individuals. As one example, the cutting-edge analyses of De Solla Price [326] in the early 1960s employed key numbers to investigate the transformation processes of scientific production. A second couple of methodological approaches applied here are based on a clustering authors, e.g., by nationality, to study preferences for international joint authorships [327]. Scientific communities as “group[s] of scientists […] agreed on accepting one paradigm” [328] are another research object. The study of co-authorship networks focuses on detecting structures of scientific cooperation, employing graph analysis methods [329]. Although such approaches are limited to a structure representation [330] and include a number of potential sources of error and limitations, computer-based analysis and evaluation of co-authorships foster several new insights into scientific co-operation [331,332]. For example, a comprehensive investigation of publications in the fields of medicine, science, and computing [333,334] reveals that the “small-world phenomenon” [335], or connection between any two authors using a chain of an average of five to six parties, could be identified for these communities. In addition, a number of smaller studies deal with co-authorship within individual disciplines [336], countries, or regions [337,338]. Besides a description of scientific networks, another issue is identifying important protagonists in scientific communities [339,340]. This latter aspect is of interest for 3D modelling in humanities. Several approaches focus on the modelling of topics described in researched articles. To investigate, we employed a combination of surveys and textmining, the latter comprising statistical and machine learning-based approaches (c.f. [44]).

5.2. Main Authors and Academic Structures

Since a basic idea from sociology is that in most cases, ties between individuals, like common authorship, will foster an exchange of ideas and information [341], we investigated co-authorship patterns with the help of a sample of publications from major conferences in the field of digital cultural heritage (

Table 3. Sample.

Publication	Volumes	No. of Articles
3DArch Conf. (bi-annual conf.) (Contributions published as special issues of ISPRS Archives: [344])	2005–2017	389
CAA Conf. (annual conf.) [345])	1973–1992, 1994–2001, 2004–2009, 2011–2015	1637
VAST Conf. (annual conf.) (Contributions published by Eurographics: [346])	2003–2006, 2008–2012	202
Digital Heritage (bi-annual conf.) (Proceedings available via IEEE: [347,348])	2013, 2015	401
Euromed Conf. (bi-annual conf.) (Proceedings published by Springer: [349])	2006–2016	607
CIPA Conf. (bi-annual conf.) (Contributions published as special issues of ISPRS Archives and ISPRS Annals: [344])	1999–2001, 2005–2017	1248

). Relevant podia dealing with 3D modelling in the humanities were initially identified by three expert interviews and ex-post investigated via an open-ended online questionnaire amongst the article authors, with 988 people participating and 602 completing the survey. Various of the investigated conferences published their proceedings as journals or issues. In addition, the highest rated conference series were more than twice as often named as most relevant by the survey participants as the highest ranked journal. Consequently, we assume that the difference between journals and proceedings seems blurry and conferences are more relevant for scholarly communication than in other research fields. The investigated publications were written by 4894 people and contain nearly 14,500 links between authors of collaborative articles. A particular interest was in examining cross-nationality. As shown in Figure 8, most were written by authors belonging to institutes in the same country (light blue). Facilitators, such as people in the top ten in the categories of number of connections to other authors (degree), relevance as a connecting factor between author groups (betweenness centrality), or the number of publications (count), were highlighted in the graph (c.f. [342]). The giant component cluster (orange rectangle) contains 1664 out of 2951 authors connected to each other. This indicates that 56% of co-authors published in cooperation with just one other person. In comparison, the values for authors of the top-ranked papers in the field of chemistry are over 90% [343].

5.3. Topic Areas

To investigate this topic, we conducted an online survey amongst all authors of publications from major conferences in the field of digital cultural heritage [350]. Out of the queried 3148 persons, 988 individuals participated, and 602 completed the survey. Concerning the individual topics (Figure 9), data management was most frequently named, ranging from GIS and BIM to metadata schemes and data architecture. It was followed by data acquisition, photogrammetry, laser scanning, and other surveying technologies. As an additional note on the survey, many answers did not fit into the predefined categories and were subsumed in “Others”—in most cases, specific methods or objects of research.

Similar to the findings from the survey, a discourse in the above-mentioned sample of conference publications is primarily driven by technologies, and the most common keywords refer to the technologies used. Most research is around data in terms of acquisition and management, visualization, or analysis. Moreover, the observed scientific discourse closely relates to practical projects relating to specific cultural objects, technologies, or practices [351]. Both indications lead to the assumption that the observed scientific community is foremost a community of practice (c.f. [212]).

5.4. Most Cited Publications

To investigate citations, we developed a set of C# and VBS tools to extract single references from full texts into CSV and to separate authors, publication dates, and title information. Within 3391 articles from our corpus, 51,789 references were found, amongst them 11 with more than 20 citations (Figure 10). While most of the top-cited papers describe algorithms, others, like Wheatley’s book on GIS for archaeology [352], provide an overview about specific fields of application.

Current Challenges for 3D Modelling

To identify prospects and demands, we ran five workshops involving around 60 researchers, as well as an online survey with 700 participants to verify findings from these workshops. Lack of funds and awareness were named as the currently most pressing issues (c.f. Figure 11). The primary need identified was to develop human resources. Further requests included sustainable and practicable approaches to access wider scientific communities, widely interoperable documentation and classification strategies, overarching cataloguing of projects and objects, as well as strategies and technologies for exchange between different technological domains [350].

5.5. A Scholarly Culture

Historically seen, digital heritage concentrates on tangible and intangible cultural heritage objects and their preservation, education, and research (e.g., [344]). Despite the wide variety of approaches and topics, digital cultural heritage has evolved to a specific academic field with conferences, journals, and various frequently contributing researchers and institutions [351]. The community is driven by researchers from European countries, especially Italy, with a background in humanities. Moreover, conference series are most relevant for scientific discourse, and EU projects set the pace as the most important research endeavors. A key aspect of digital cultural heritage is cross-disciplinary cooperation. With regards to De Solla Price, digital cultural heritage could be seen as a mode 2 science [326] with an emphasis on cross-disciplinary teamwork, the use of machines, and joint intellectual property. Consequently, a disciplinary culture of digital cultural heritage is common to science and engineering but less so to humanities [120].

5.6. User-Generated 3D Models

In addition to the creation of 3D models in scientific environments, there is a large amount of user-generated 3D heritage content. Sketchfab, currently the largest repository for 3D content, hosted 100,000 3D cultural heritage models in 2019, representing 30% of all 3D models on this platform [353]. The creation of user-generated 3D content is strongly supported by the availability of ready-to-use photogrammetric applications (see Section 7.2) and open-source 3D modeling tools. In terms of level of participation, most citizen science projects use crowdsourcing as the involvement of “non-scientists to help to analyze or collect data as part of a researcher-led project” ([354], p. 259). Examples include collecting and processing images as a prerequisite for 3D photogrammetry [355], or crowd-based creation of 3D models [256,356]. Co-design “involves citizens into the research process from its beginnings, or the stimulus for the research project originates from the citizens” ([357], p. 4). Although more prominent with text- or image-based research in the humanities [358], co-design is frequently used for 3D content and experience design for museums [359,360] or (serious) history games (e.g., [361]). Besides the challenges of participatory processes such as user activation and management, task definition or quality control [362], citizen science tasks in the humanities has to handle complex, non-standardized, and knowledge-intensive tasks and therefore are hard to assess with regards to scientific quality of processes and outcomes [363,364].

6. Workflows

This chapter provides an overview of the workflows involved in creating 3D models (c.f. Figure 12), which differ greatly in 3D digitization and 3D reconstruction. As highlighted in the previous chapter, interdisciplinary collaboration is a key feature of many 3D modeling projects in the humanities and the following sections highlight some key features and strategies for cooperation.

6.1. The Process of 3D Digitization

For 3D digitization, process and workflow schemes [365,366,367,368] differ greatly by application scenario parameters, e.g., retrieval technology and heritage object parameters. Workflows distinguish between documentation (data acquisition, e.g., of image or range data, and registration) and 3D modelling (computing into a 3D model, including point cloud generation, structuring and modelling, and texture mapping) [276,369].

6.1.1. Data Acquisition

The data acquisition quality is influenced by several attributes (cited from [274,362]):

• Resolution in terms of size and granularity of the samples
• Accuracy of the measure in terms of variation from the original
• Repeatability
• Environmental sensitivity, such as robustness of acquisition method under different climate conditions (e.g., light, wind, temperature).

Framework conditions are set by acquisition time, portability of the equipment, flexibility (e.g., to retrieve at different conditions), and price [369].

6.1.2. Data Processing

Data processing comprises a wide range of mostly algorithmic approaches to creating a 3D model out of the acquired data (technical processes are discussed in Section 4.4). The aim of related workflows is to set up a multi-stage data processing pipeline, which varies by data type, quality, expected results and object (recent examples for museum artifacts [370], monuments [371]). Pipeline design is influenced by objectives including levels of output quality, velocity, reproducibility, flexibility, robustness, and automation (c.f. [372,373]). This contradicts with the huge variety of cultural heritage qualities [82,374]. Despite recent achievements, e.g., with regards to data fusion (e.g., [372]), most pipelines are still highly purpose built and require manual processing to reach satisfying model quality.

6.2. The Process of 3D Reconstruction

The process of digital 3D reconstruction encompasses the creation of a virtual model by means of software tools, which is mostly done by specialized modelers, followed by visualization, through which the model is rendered into a presentation format. This process is usually closely accompanied by historical research through which a sound understanding of the object to be modelled is developed on the base of sources that provide information from the past [170,356,369]. In view of the resulting division of labor, it is essential to consider the cooperation between those involved and the associated aspects of communication and quality management. The entire working process of virtual 3D reconstruction can roughly be divided into sources, modelling, and visualization (c.f.

Table 4. Classification of the digital 3D reconstruction process as regards sources, modelling, and visualization.

Sources	Modelling	Visualization
Historical sources such as images: veduta, panoramas; plans; textual sources: construction news, invoices for building work etc. Contemporary sources such as images: esp. photographs; plans; data: sensory analysis and surveys, topographic reliefs, street maps, etc.; texts: scientific papers, esp. studies and architectural history Logical sources such as: architectural systems; analogies/typologies; model logics	Semiautomatic model generation Procedural generators Manual modelling using digital tools	Static images or renderings Animations Interactive visualization (e.g., VR applications or interactive tours) Data output (e.g., for production or data-based analyses)

), which may be made up of numerous steps and tasks and take on different forms.

6.3. Interdisciplinarity of 3D Reconstruction

While cross-disciplinarity means the adoption of methods from one discipline by another, interdisciplinary collaboration refers to a “collaboration of several disciplines on a [joint] topic or issue” ([375], pp. 7). The latter necessitates the development of a multi-disciplinary terminology and common methodologies [376,377]. Institutionalization of interdisciplinary collaboration ranges from temporary to the creation of new “hybrid” research disciplines [378], such as the digital humanities.

In digital reconstructions, information technologies serve to produce virtual historical models. In addition to computer science as the tool provider, content-related perspectives acquired from archaeology and the history of culture, art and architecture, architectural research, and museum studies are involved. Owing to the highly specialized nature of the tools, the model is usually not created by the persons responsible for the content, but, in interdisciplinary projects, by modelers who come from computer science, architecture, geosciences, engineering, and design.

As one of the most common explanation models for cross-disciplinary understanding, the common grounds approach originates in cognitive psychology and explains factors and recommendations for successful communication [379]. For two individuals to transfer knowledge successfully, their knowledge, beliefs, and assumptions must be synchronized [380]. One key issue for cross-disciplinary communication is varying meanings of the same terms across disciplines, or the reverse, usage of different terms for the same content [381]. These differences are rarely known in advance, but usually show up only in the course of an interdisciplinary cooperation. While the common grounds theory focuses on communication between individuals, the trading zones approach focuses on modes of information exchange and collaboration in interdisciplinary teams [382]. In these contexts, visualization techniques are important to gather immediate visual feedback to sharpen and approve design visions.

6.4. Research Spotlight: Phenomena and Strategies for Cooperation

Regarding research scenarios and practices in architectural history research, investigation of digital research and information tasks is currently often based on the “mechanistic” idea of research action as a sequence of steps (e.g., research primitives as a sequence of research operations like literature search, writing, comparing [120,383] or eScience [384]). While this paradigm enables granular systematization, even for visual research (e.g., [385,386]), it rarely reflects the specifics of humanities research and their highly individual and tacit procedures. A contrasting paradigm of research action in the humanities is to handle it as a black box, which is based on experience and implicit knowledge [387] and studied in terms of information and research behavior [56,388], science and technology studies (STS), epistemic cultures [211], and philosophy of science [215,216]. This approach is highly specific to the research question or sub-community. To link both paradigms, we employed a case study approach (c.f. [389]) to investigate four 3D reconstruction projects [168]. As a research paradigm we used a grounded theory approach [390]. The guiding principle is the development of subject-related (“grounded”) hypotheses with the goal to identify and model influencing factors and their dependencies [390]. For data acquisition, we employed a combination of guideline-based expert interviews [391,392], observations (c.f. [393]), and process artifacts for gaining information about cross-disciplinary projects on digital 3D reconstruction.

What are key findings from this research? As proven empirically [394], many challenges for 3D reconstruction projects are connected to a lack of interdisciplinary understanding. To learn about these topics, we performed four case studies of 3D reconstruction projects. Data was collected via expert interviews (c.f. [391]) and observations (c.f. [393]); the research paradigm combined heuristic frameworks and grounded theory research [395]. What are the findings? Due to the high complexity and team-based workflows, aspects, and usage practices for communication, cooperation, and quality management are highly relevant within 3D reconstruction projects. Intensive support by images during a reconstruction process could foster interdisciplinary communication, and could be used as “creoles” [396] to exchange and share mental models. For that, it is necessary to synchronize terminologies or to employ common grounds like symbols, colors, or tags [168]. Especially if people with different disciplinary backgrounds are involved, visual media fosters communication and quality negotiations, for example, by comparing source images and renderings of the created virtual reconstruction. Furthermore, several projects successfully adopted highly standardized conventions from architectural drawings for interdisciplinary exchange. Such decisions and tasks should be started at an early project stage and should be controlled and adapted throughout the entire process. Ideally, such visual coding schemes would be a mental model shared by all members of the project team and would be documented and based on either extant coding schemes, e.g., from engineering, or would use natural coding like physical analogies or concrete depictions [397] to make these issues recognizable at later times or even accessible for later works. However, in all cases, images would only support communication and, especially for complex tasks and interdisciplinary exchange, personal contact would be more useful than communicating information over long distances. Resulting challenges include access to and evaluation of models and images to make authorship transparent, as well as references between reconstruction and (explainable) fundamental knowledge such as sources. A specific challenge is presented by the division of the labor that is usually involved in the project. It is evident from published project reports that interpretative 3D reconstruction projects are almost always interdisciplinary in nature, with the teams mostly only coming together temporarily, unlike the situation in companies or institutions [398]. The tasks are usually divided between historical research and the creation of the model. Consequently, aspects such as the organization of work, the distribution of tasks, cooperation, and communication, are therefore correspondingly important [137].

7. 3D Modelling Technologies

This chapter presents the main technologies and approaches for creating 3D models through digitization and reconstruction (c.f. Figure 13). With the advent of machine learning, several hybrid approaches have emerged that combine automated processing with generative design technologies.

A large variety of approaches and technologies are used to create 3D models (Figure 14). According to Francesco et al., approaches to 3D modelling include [152]:

• Geometrical model: acquisition of all dimensions using sensing technologies;
• Manual model: revision of plan/drawing in 2D to obtain 3D without direct measurements;
• Hybrid model: combination of plan in 2D with 3D through measurements by means of an instrument;
• Reconstructed model: reconstruction entirely without measurements.

7.1. 3D Geometry Representation

A representation of 3D geometry information can be based on points, mathematical shapes, or polygonal information. In the case of the point cloud, an object surface is described by a list of points (or voxels) in space. Mathematical representations as Non-Uniform Rational B-Splines (NURBS) use equation sets to describe shapes in a discrete way [399]. Polygonal meshes describe surfaces by a list of vertices and tri- or quadrangular faces. There is a wide range of approaches and much to convert between those formats, e.g., to create polygonal meshes from (digitized) point clouds (e.g., [400,401]) or describe mathematical shape (e.g., [402]).

7.2. 3D Digitization

7.2.1. Surface

Various large-scale projects on 3D modelling in cultural heritage primarily focused on 3D digitization from contemporary survey data (c.f. [374,403,404]). As mentioned in Section 3.3.3, a major distinction is between active systems such as laser scanning and passive systems like photogrammetry from photographs. Today, both approaches are frequently combined to obtain ever more accurate and multiresolution digitization [84,318]. Most approaches lead to a 3D point cloud, which has to be calculated during the meshing step into a polygonal mesh, corresponding normals (as information about the orientation) and—in the case of photography-based sensing—radiometric information.

Photo-based approaches: Passive approaches to surface modelling are primarily based on the use of images. Due to progress in both the quality and availability of camera hardware and software for image analysis [403], most digitized models are now acquired via photogrammetry. A well-established method is 3D modelling of architectural heritage from current, native digital photographs or video content via photogrammetric algorithms. The development of different feature matching methods [405,406] and the integration of those into the structure-from-motion workflow has led to extensive use (e.g., [164,407,408,409,410,411]). Today, a large variety of image spatialization algorithms are available [82,412,413], and photogrammetry is used in various settings such as 3D underwater archaeology [414], landscape surveying [415], and monument recording [416]. While currently extant approaches achieve remarkable quality for natively digital imagery, they often fail with historical imagery due to the demand for high quality and numbers of input images [409,417,418]. Consequently, current approaches often reach low quality or fail to process historical, non-native digital imagery due to sparse samples, missing metadata, incompleteness, and radiometric quality issues [406]. These are not only technical but fundamental barriers, and despite much research, current photogrammetry dealing with historical images [419,420], e.g., re-photo approaches [421,422], still requires much manual processing. Initial attempts to advance photogrammetry from historical imagery use video [423] or—invented by my group—architectural feature-based photogrammetry [424], and features derived by neural networks [425] seem promising avenues for further automation, but still require a great deal of previous optimization and large samples.

Range-based approaches: Similar to photogrammetry, range-based approaches such as airborne, terrestrial, or close-range laser scanners [81,426,427] are most frequently used to acquire tangible heritage. All systems use laser beams sent to the surface to retrieve 3D information, e.g., via triangulation, pulse (time-of-flight; ToF), or phase-comparison [367]. Robust and fast scanning systems have been available since the mid-2000s and enable time efficient and high-resolution 3D scans [428]. Laser scanning was the primary acquisition method until 2010, but its relevance declined with advances in photo-based techniques [429]. Despite this, laser-based approaches are still of high relevance, especially to document large-scale objects in a short time and for specific material properties such as high reflecting surfaces [359,427,428].

7.2.2. Volume

Numerous surveying techniques focus on the acquisition of volume information, revealing the inner structure and materiality of a tangible heritage object. For example, computer tomography (CT) is a radiation-based method. Concerning cultural heritage, it is frequently used to investigate the inner structure of an object, e.g., with regards to material composition [430,431], to gain information about hidden layers in images and texts [432], or reveal inner structures [433].

7.2.3. Others

There are various other 3D modelling approaches, distinguishable as non-destructive and destructive techniques. As examples for non-destructive techniques for monuments and artworks, multispectral analysis supports mainly material analysis of multi-layered surfaces, e.g., of paintings [172,434], but also for globes [435] or sculptures [171]. IR thermography detects emissions in the infrared thermal range of the electromagnetic spectrum to measure heat emission [436]. This is used to non-destructively detect anomalies in heritage sculptures [171]. Especially in landscape archaeology, a large number of geophysical surveying techniques are used for anomaly detection below the surface and vegetation (c.f. [437]). Examples are ground penetrating radar (GPR), resistivity, and magnetometry techniques [438,439,440,441]. As an example for destructive techniques, the Drilling Resistance Measurements System (DRMS) quantifies the hardness of stone based on its resistance to drilling [436,442].

7.3. 3D Reconstruction

In a digital reconstruction, models are created by a human-driven use of computer software. In contrast to digitization processes, where the development of technical workflows and tools is a focus in the academic discourse, human-driven workflows for digital reconstruction mostly rely on application of standardized software [394]. This software touches a variety of technological domains like geo information systems (GIS), computer graphics (CGI), computer-aided design (CAD), or building information management (BIM), each with their own standards. All approaches recommend specific tools and workflows and offer specific benefits, whether information on object volumes (BIM), a highly realistic appearance of surfaces (CGI), accurate size measurements (CAD), or large-scale geo-referenced information (GIS).

A 3D reconstruction geometry can be created as static modelling. Static modelling requires the individual definition of every 3D geometric surface feature and is therefore a highly flexible but labor intense process. The main approaches for static modelling are boundary representation (b-rep) and polygonal modelling [443]. Boundary representation modelling is the primary surface representation method in CAD and describes surfaces piecewise via connected polynomial representations [444]. In contrast, polygonal modelling, as used in CGI and for meshed dense point clouds created via 3D digitization, describes geometrical surfaces by triangles. Each triangle or polygon has clear information about its inner and outer side and a normal as a definition of orientation. Specific workflows for model creation differ greatly for each software program—b-rep modelling workflows tend to be shape-oriented (e.g., curves, geometric primitives) since polygonal modelling primarily focuses on points, edges, and faces (polygons).

Generative modelling: Generative modelling creates a 3D geometry via pre-programmed rules [443]. Based on static modelling techniques, it reduces effort and simplifies the solid modelling process. Generative modelling still requires manual work, but eases the modelling process by predefining the structural elements of buildings and altering them on a parameter level (e.g., defining the properties and number of windows of a façade). Numerous projects use generative modelling primarily to create large-scale heritage structures such as 4D city models [190,445,446] and to enhance modelling quality, e.g., by a parametrization of 3D models from scanned data [447]. Another generative modelling approach tries to structure the manual modelling process by identifying common steps in the reconstruction work and translating them into digital ontology-based manuals [448]. The combination of generative modelling with recognized footprints from historical cadastre data is currently under investigation in various projects [449,450]—e.g., by using machine learning to predict building heights from map data [450].

7.4. Hybrid Methods

Traditional algorithmic-only approaches, as in photogrammetry, employ algorithms to complete a specific operation, e.g., to detect, describe, and match geometric features in images [405]. In contrast, machine learning employs algorithmic approaches to train a statistical model in a supervised or unsupervised way via training data—e.g., to detect features [451]. Current evolvements in computer vision are closely coupled to the massive renaissance in machine learning [452] with the use of convolutional neural networks (CNNs) (c.f. [453]) since 2012. Machine learning approaches are currently heavily researched and used for image and 3D point cloud analytics in cultural heritage (as recent overview [83]), but also increasingly for 3D modelling tasks.

3D/4D model generation via machine learning: In 3D/4D reconstruction of cultural heritage, machine learning-based technologies are primarily used for specific tasks within the modelling process: to preselect imagery [454,455], in semantic segmentation to classify parts of images [456,457,458], and to recognize specific objects (e.g., [458,459,460,461]). Another strand—bypassing the modelling stage—is generating visualizations directly from imagery [459,462,463]. Machine learning-based technologies currently require large-scale training data (e.g., [83,458,459,460]) only capable of recognizing well-documented and visually distinctive landmark buildings [464]. Machine learning fails to deal with less distinctive architecture, such as houses of similar style. Regarding transparency, most machine learning approaches are applied within black box settings in a non-standardized way (c.f. [83,465]). Specifically generative adversarial networks (GAN) as a combination of proposal and assessment components of machine learning are frequently employed in 3D modelling. Application scenarios are single photo digitization [466], completion of incomplete 3D digitized models [467,468], or photo-based reconstructions [469].

8. Visualization and Design

The created virtual 3D models are most frequently presented primarily as visual output, which could be static images, animations, or even interactive visualizations like computer games. According to Ware, visualization can support research and understanding in five ways (cited according to [15], p. 5):

• It may facilitate the cognition of large amounts of data.
• It can promote the perception of unanticipated emergent properties.
• It sometimes highlights problems in data quality.
• It clarifies the relationships between large- and small-scale features.
• It helps in the formulation of hypotheses.

3D models are an increasingly important basis for further analytical steps, such as finite elements methods (FEM), light or fluid simulation (CFD) (c.f. Section 2.2). Since 3D modelling is still primarily connoted to visual sensing, 3D spatial modelling is relevant for other senses including sound [470,471], scent [472], and multi-sensual representation [473]. In this chapter, relevant media formats and design-related challenges and developments are highlighted (c.f. Figure 15).

8.1. Media

Media comprise approaches such as extended realities (XR) or films, which require devices on the user side to be perceived. Recent strands are holographic visualizations, including volumetric displays of heritage objects [474]. Deviceless approaches include rapid prototyping of manufactured models or printed images, which can be observed by viewers without specific devices.

8.1.1. Images and Films

Visual media and perception are the main sources of information, communication and research on cultural heritage (c.f. [475,476,477]) and images are still the most frequent output of 3D modelling projects [262,263]. Those results are used in films and animations [12] or as still images. Formats and purposes differ greatly between the main target groups: the general public and domain experts [478]. With regard to the represented knowledge, visualizations equally serve to represent relations, processes, and a constructive structure [479].

8.1.2. Extended Reality

In contrast to asynchronous media like images and films, XR provides a real-time, high level of engagement visualization, which is not constrained to a single point of view and allows variable levels of interaction between the users and the virtual content [480]. A wide scope of interactive technologies [481] range on the continuum between real and virtual (c.f. [482]). This ranges from real environments, augmented realities (AR) and augmented virtuality (AV)—together also denominated as mixed reality (MR)—to virtual reality (VR), or fully computer-generated visual representations. According to Russo [84], MR can be additive by adding information that does not exist, or subtractive by hiding or deleting parts of the real world. Layouts for user interaction are highly dynamic, perspective-dependent, and require a high degree of temporal coherence [483,484]. So far, the representation of time-independent data has been investigated most, and developed into interaction patterns [485,486]. Overviews about XR applications in cultural heritage have been presented in various publications with regards to museums [237,242,252] and virtual tourism (recently [250]).

8.1.3. Rapid Prototyping

A wide range of rapid prototyping techniques is used to create physical representations of tangible heritage [487]. This comprises additive manufacturing techniques that add layers of material to a 3D structure and subtractive techniques shaping a 3D structure by removing material [281,488,489,490]. A major influencing factor is the scale of reproduction, ranging from 1:1 reproduction at various scale to miniaturizations (e.g., city models) or maximizations of very small objects. Highly relevant technologies for additive manufacturing are (1) stereolithography, which builds objects layerwise by tracing a laser beam on the surface of a vat of liquid photopolymer and hardening by UV or thermal processing, and (2) laser sintering is based on powders. Those are applied layerwise and selectively fused or melted at the surface by laser bonding to the layer below. After removing unprocessed powder, a 3D structure remains. (3) Fused filament fabrication is the layerwise application of thermoplastic materials, which are heated and deposited layer by layer (c.f. [491]). Subtractive approaches comprise computer-controlled milling tools (CNC machinery), which mill 3D structures out of solid blocks of material [492] or laser engravings, which uses a cross laser beam to selectively overheat spots within solid glass and cause micro cracks with differing refraction properties (c.f. [491]).

8.2. Visual Design and Perception

The quality of perception is generally influenced by visual parameters, such as shading and size (e.g., Gestalt theory [493], cognition psychology [494,495]), cultural settings incl. visual semiotics [496]), engagement [497], and research purposes such as visuospatial reasoning [193,206,309] or visual research [385,386,496]). Expertise of the researcher with a specific content is another parameter, including, visual competency/literacy [119,498,499], comprising the terms “visuality” [500], and, more precisely, “visual competence” [501] or “visual literacy” (e.g., [502,503]). Architectural psychology has identified two key influencing factors regarding the perception of architecture: form structure and form content [504]. Structural elements include basic aesthetic ones such as color, form, spatial arrangement, movement, and depth [505]. Content-related dimensions include cultural meanings, habits, and astonishment or novelty for the viewer [506].

Besides this, various specific findings determine visual perception: cognitive processes such as orienteering and wayfinding heavily rely on known structures as landmarks [507]. The combination of media sources as text and image strongly influences the viewer’s selective attention and frame an overall interpretation [315,508,509,510,511,512,513]. Increased familiarity with specific content increases the ability to focus on certain aspects and visually recognize patterns [201,202,396] and enables the ability to understand more abstract visual content [396,397]. According to the concept of professional vision [201,396,504], visual elements are mostly perceived as structures by non-experts, while experts classify and contextualize perceived contents during viewing [514]. The material/visible/formal and immaterial/functional parts are perceived asymmetrically. As a consequence, aesthetic evaluations by experts such as architects are practically incoherent with those of laypersons [515]. The readability and usability of visualizations are highly dependent on the respective visual experiences in dealing with the type of visualization and the architecture depicted (c.f. [504]), but decisively determined by which groups of recipients were targeted with the visualizations and which visual experiences were assumed in each case.

Digital 3D/4D reconstructions [516,517] have joined a long tradition of architectural reconstruction and depiction [302,518]. Despite much research, a main criticism of the studies of viewer effects is that they primarily take descriptive rather than quantitative approaches [519,520,521]. What are the visual properties of 3D/4D visualizations?

• The level of detail (LoD) needs to be appropriate to the research task [522]. While detailed visualizations of historical reconstructions are advantageous for imaginability [206], they are suspected of distracting scholars from their research questions [523] or leading to cognitive overload [524].
• Visual style is often discussed in terms of fit to scholarly recommendations and distraction of viewers (e.g., [144]). Since early 3D/4D reconstructions strived for more immersive and realistic visualization [516], a current scope comprises a large variety of photorealistic and non-photorealistic styles [525,526,527].
• Much research about the visualization of different degrees of certainty [528,529,530] has led to a multitude of visual strategies for heritage content [531]. Current approaches can be roughly categorized into enrichment of representations by explanatory elements [532] and adaptation of representation quality, e.g., LoD or visual styling [522,530,533,534,535,536,537].
• Scaling has been frequently assessed as an important parameter for perceiving architecture [522,538].
• Visual acuity is the ability to distinguish details: in contrast to scale, the main influencing factor is the distance to a virtual or physical object [519,539,540,541].
• Perspective depiction and perception of architecture includes the viewer effect of different fields of view [541,542,543]; the top-down view is the predominant mode for investigating landscapes [544].
• Lighting is well known to be relevant for investigating historical architecture [545]. Specific workflows and approaches (based on visual comparisons) are proposed for virtual visualization [546,547,548,549,550,551].
• Color is highly relevant for a perception of historical objects and ranging from realistic coloring to color-scales coding parameters or supporting visual distinction of model parts [552,553]. Accurate reproduction of colors is challenging with regards to both modelling [554,555] and reproduction by digital devices [556].

8.3. Interaction and Motivational Design

The visual appearance user presentation and interaction with 3D content is marked by the level of interactivity and poses various design variables. Specifically for educational settings, motivational design is of relevance for engaging with users and enable learning.

8.3.1. Interaction Design

A classification of interaction is done by the engagement taxonomy developed by Grissom et al., which differentiates between six degrees of interactivity for visual output [497]. According to Tullis and Albert, user experience (UX) “refers to all aspects of someone’s interaction with a product, application, or system” ([557], p. 15).

Digital Interface Design: For the design of digital interfaces, many interaction parameters are technically determined—e.g., by capabilities of virtual reality glasses. Especially, XR applications go well beyond passive information presentation and enable multiple freedom degrees for interaction design, as well as requirements, e.g., to avoid motion-sickness of users due to continuous movement (e.g., [558,559]). With regards to acceptance, the perceived usefulness of applications plays a major role, which in turn depends on optimal stimuli of representation and interaction [560]. Layouts for user interaction are highly dynamic and perspective-dependent and require a high degree of temporal coherence [483,484]. Currently, mainly the representation of time-independent data has been investigated and developed into interaction patterns [485,486], since validated strategies for time-dependent 4D data presentation are still missing.

Linked physical and digital spaces: Especially in cultural institutions and at sites, digital 3D applications are embedded in physical spaces. There are various recommendations for the design of linked virtual and physical spaces, e.g., in museums [561,562] or at heritage sites via location-based 3D applications connect [563,564]. Some recent evolvements comprised multi-user story-based virtual experiences in physical museums, enabling to enrich museum visits by augmented experiences [565].

8.3.2. Motivational Design

Gamified and playful approaches: According to Deterding et al., gamification implicates the use of “elements of games that do not give rise to entire games” ([566], p. 2). It therefore differentiates from playful design, that—in contrast to gamification—contains no rules or specific goals, and serious games, defined as full-fledged games for non-entertainment purposes [566]. All three types are intensively used with 3D contexts in heritage, particularly in educational settings. Examples are:

• Playful design: Examples are Minecraft-like creation games [567] or massive open online environments (MUVEs) as 3D social interaction spaces [568].
• Gamification: Examples for 3D heritage are applications using gamified elements and strategies to enhance interaction with heritage content [569].
• Serious games: Various games were either developed for didactic purposes or derived from entertainment games [236,570].

Storytelling: Storytelling stands for the use of narratives and fictional or non-fiction stories to present a subject [571]. It is well investigated by psychological [572,573,574] and educational studies (e.g., [575]) that narratives can have both, a positive motivational effect by making a subject more alive and engaging to the audience, and a positive learning effect by reducing cognitive load. Storytelling is widely used to digitally present heritage content [576], and particularly for digital 3D models in humanities in various educational settings [577,578,579,580]. Amongst the multitude of freedom degrees for design (e.g., by fictionality, media, storytelling modes, polyvocationality, etc. [576]), general types of scenarios from an educational perspective are:

• Expositional: Story elements are used in a pre-scripted way to explain specific 3D entities or link groups of entities to each other. Examples are for instance the presentation of underwater archaeology by stories told by an avatar [577].
• Explorative: Open-world-games or open-ended platforms encourage dialogic encounters where the user has various choices to interact with a dynamic supply of narrative contents. Examples are for instance the explorer modes in Assassins Creed [13] or dynamically scripted heritage experiences [581].
• Constructive or connective approaches: Users can co-create stories and contribute story content and share with others. An example is the Jena4D project where users contribute location-based personal stories and photographs in a 4D environment [582].

8.3.3. Research Spotlight: How Are Virtually Represented Structures Perceived?

Our interest in this case is how (1) digital environments and applications and (2) products of digital reconstructions and visualization processes in the context of visual humanities have to be designed to meet user needs (On the distinction between user and usage centered design: [575]). From a process perspective, a main distinction is for the conception of UX and its empirical testing. In the context of this study, we focus on usability engineering. In contrast to UX, the goal of usability is “making a system fit the bodies and minds of its users in context” ([583], p. 7). Both approaches are widely described in the literature [583,584,585,586,587] and comprise techniques to support the development of interaction concepts out of empirically grounded or hypothetical information on objects and processes. In a strict view, use cases are not part of usability conception techniques (c.f. [585]), but a vital instrument to describe functionality and intended interaction workflow for developing (prototype) products primarily as text or graphics [588]. While UX design and realization are widely addressed in digital reconstruction and visualization discourse, empirical testing methods are rarely described in the research literature on digital reconstruction in visual humanities (e.g., [589,590]). For proving hypotheses on design employed both, standardized questionnaires [591,592] and the observation of test users to observe how individuals cope with a specific task on a product and identify potential weaknesses in user-product interaction. During four studies, we researched how virtually represented architecture is perceived [533]. The studies involved in total ~90 persons and used methods from usability testing. The research questions were:

• How much detail is needed to recognize virtual architecture?
• How well can building properties be estimated in the virtual in comparison to real world settings?
• How is aesthetics perceived in the virtual with regards to presentation forms?
• How well can mistakes be found in a visualization?

What are the findings? A current academic discourse focuses on the level of detail of modelled buildings [593]. Our findings concerning property estimation and recognizability, as well as studies from other fields of visualization [594], lead to the assumption that the perception of objects is highly influenced by the surroundings shown and visual framing. Therefore, a suggestion for improving estimation and recognizability may be to focus on modelling surrounding objects, rather than on higher level of detail. There is also a long discourse about the visual representation of different degrees of hypothesis, from enrichment of representations by explanatory elements to adaptation of representation quality [535]. In the context of architectural visualization, however, viewer effect has rarely been empirically tested. Abstraction, e.g., of colors, has a direct influence on assessment of perceived aesthetic qualities. This possible bias seems especially relevant for visual assessment and analysis. Current 3D modelling projects tend to model objects as accurately as possible. Against this background, it is our finding that observers rarely notice subtle changes. Another finding is that geometric and radiometric errors in the digital visualizations were poorly recognized by both experts and laypersons. There were also major differences between the groups in naming false-positive errors. Both findings led to the hypothesis that there is no common perceptual strategy even for architectural historians—this will be tested in a future study.

9. Documentation

Investigating documentation principles is a core topic of library and information studies (LIS) [595]. Philosophy of documentation is related to information conceptualization as models and theories [596,597], e.g., classification [598], information creation [599], and information behavior [600]. As a methodological approach, documentation behavior is primarily investigated using empirical methods described above [595]. Specifically for the documentation of scientific processes, scholarly publication has had established principles for a long time. With regards to the documentation of digital processes and via digital means, there are numerous guidelines of overarching relevance (e.g., [601,602]). The technical classification of research results is usually practiced via metadata and paradata. The process and principles for developing metadata schemes are formalized [603] via the request for comments approach widely used for technical standardization [604]. This chapter provides a brief overview about strategies and types of documentation and data formats (c.f. Figure 16).

9.1. Documentation Strategies

Documentation strategies address the preservation of knowledge, and are thus linked to transparency, reproducibility, and portability. These objectives entail clarification of “included sources, decisions, workflows, possible misinterpretations and methodology” [605]. Internally, this helps project partners with safeguarding and communicating their contribution. Externally, such records are intended for evaluation and discussion of the project’s rationale and result. While this distinction may appear fuzzy to the end user, distinction between process and documenting outputs is a more appropriate approach. A high-level guide to digitization is provided by the German research foundation [601].

9.2. Documentation of 3D Models

9.2.1. Documentation of Results

The documentation of 3D modelling results through metadata is well established nowadays. Numerous initiatives are advancing the development of domain-specific thesauruses for art and architectural history content—e.g., ICONCLASS [606] and the Getty Art & Architecture Thesaurus (AAT) [607]. Ontologies such as Wikibase [608] or CIDOC-CRM also define relations between data. In particular, CIDOC-CRM became an overarching standard for heritage documentation [609] and is fixed as an ISO standard. Several sectoral standards like IFC for BIM [293] and GML [300] for geo and city-scale models are of relevance. The quality of implementation in application ontologies is heterogeneous [610,611]. Derived categories for classification can be the employed reference ontology, as well as the adopted application ontology [593]. Besides the description of results, the demand for standardization of 3D models—comprising digitization and reconstruction—is still high [612].

9.2.2. Process Documentation

In contrast to result documentation strategies, current approaches to documentation of the creation process are still theoretical [613] or highly prototypic [605]. For 3D digitization processes and workflows differ greatly by application scenario (cf. Section 4.3). For 3D reconstruction, it is evident that in a majority of projects process documentation occurs by personal notes, communication artifacts, or versioning of states [24]. While these artifacts “document” a workflow and communication history, another question concerns the employed software, algorithms, and documentation of computational processing. For 3D reconstruction from historical sources, guidelines are provided by the Charters of London [60] and Seville [61], but these do not present a clearly applicable methodology and therefore are rarely applied in practice [351], so they are a long way from providing a transparent workflow. Despite much research (c.f. [8]) and numerous methods and tools [593,614,615,616,617,618] (e.g., reverse design to validate reconstructions according to their fit to historical images or acquired data [619,620]); documentation methods to justify modelling steps and decisions taken during the 3D/4D reconstruction [617,621,622]; with assessment of possible scenarios by numerically calculating an index of reliability [317,613,623]) there is still no consensus about methodology to document the creation process of 3D reconstructions in a transparent way.

9.3. 3D Data

A wide range of 3D file types are currently available. A main distinction is between proprietary 3D model formats (C4D for Maxon Cinema 4D or MAX for Autodesk 3D Studio Max), which are specific to the software in which they were created, and overarching formats like OBJ, DAE, STL, FBX, X3D, and gITF, which can be opened and created by many software tools [624]. Surface representations as in the formats mentioned above are distinct from volume information as in DICOM. Several approaches are using specific standards integrating both 3D and metadata, such as IFC for BIM [625] or Shapefiles for GIS [626]. Since formats such as OBJ and PLY are proprietary structured, DAE and X3D follow an XML-based data organization. Formats like X3D and gITF are specifically designed for browser-based viewing [163]. Another issue is storing 3D information (c.f. Section 7). Generative approaches do not rely on storing resultant 3D geometries, but on parameters, and generate a 3D object in real time [627,628]. In contrast, discrete approaches store all 3D information: (A) point clouds as a set of points in a defined coordinate system; (B) wireframe/polygon models as vertices connected with edges and polygons; and (C) voxels as volume pixels. While geometric data can largely be sorted to one of these archetypes, there is a heterogeneous scope of formats for radiometric or dynamic information.

10. Framework Conditions

3D modelling in the humanities is shaped by various framework conditions (c.f. Figure 17). Infrastructures, legal frameworks and teaching programs form the basis for the creation, exchange and use of 3D models in the humanities.

10.1. Infrastructures

Currently, a large variety of infrastructures are targeting 3D models. Recently, several overview reports on extant platforms and repositories [163,629,630,631] and 3D visualization frameworks and formats [624] were compiled. In the current situation, Sketchfab is the most widely used platform worldwide; its heritage and history section contained around 74,000 3D models as of early 2019 [632].

10.1.1. Virtual Research Environments

Other as repositories as collections of models, virtual research environments (VREs) are web-based information systems that provide a working environment for researchers, e.g., by including various tools for research (e.g., for analysis, comparison etc.) [633]. Specifically for cultural heritage research, a large number of VREs is available—often for specific communities like archaeology [634] or architectural history [635]. Depending on the group of users, there are a number of partially contradictory requirements: history research requires the comparability and contextualization of sources [636,637,638,639]. Furthermore, the transparent relationship between source and representation is essential [144,623]. Finally, visualization should ideally allow the identification of abstract characteristics such as ideas and systems, breaks, or deviations [640,641]. Concerning the use and development of information, two essential modes are discernible: browsing as a self-directed search of historical sources and information (c.f. [56]), and location- or context-related information shared in the course of heritage presentation (c.f. [642]). Last but not least, both research and communication approaches are usually individually adapted to a specific context. Digital tools are either tailored to specific individual scholarly communities or focus on laypersons and do not meet the requirements of researchers [643].

10.1.2. 3D Information Systems

In extension to classically 2D information systems, 3D user interfaces allow users to interact with computer-generated 3D environments [644]. Especially in German-speaking countries, such systems usually focus on 2D spatial time-related mapping of historical artifacts with related relation and aggregation information. This is reflected not only in a large number of projects [645], but also in the Europeana 4D Interface developed as an infrastructure for this purpose, which forms the basis of the DARIAH Geobrowser [646] and, despite its name, aims primarily at 2D mapping. As described using the example of the photo browser, perspective representations of 3D data open up a number of possibilities, especially with a view to linking and illustrating complex historical information as opposed to mapping. In addition, virtual city and landscape models can be enriched with a variety of other site-related information [299,647]. The linking of historical sources of different genres, their digitized data, digital research artifacts, and research results as well as associated meta-, para-, and context data has long been the focus of a large number of projects [8]. Specifically for history content, purposes range from humanities research and information issues to education and tourist applications [648].

10.1.3. Challenges for the Design of 3D Information Systems

The mainly technology-driven interaction forms and industry standards that have become established for 2D user interfaces [649,650,651] are not yet available for 3D. Research results on interaction solutions have so far gained little acceptance in design practice of 3D applications for cultural heritage [644]. Since for example the 3D-GUIde project is underway to gain grounded implications for a design of 3D interfaces [486], these design principles need to be adapted specifically for researchers in architectural history. Simple approaches based on 2D maps (e.g., Google Maps [652] or OpenStreetMap [653]) are available, also for historical images. (e.g., Historypin [654]). In this spatial context, information like the distribution of images becomes visible. Deeper information becomes available in a 3D perspective: the orientation of the image and the situation of the photographer within the context of the surrounding buildings becomes clearer [655]. The user can take up the position and orientation of the camera and blend between the image and the 3D city model. Combined with various historic states of a 3D model into a 4D city model, this allows comparisons between buildings.

10.1.4. Research Spotlight: Recommendations for Digital Libraries

Our research focused on information behavior in the humanities, particularly scholarly use of 3D repositories [24,624,645,656]. Consequently, we developed several general implications for user-centered design:

• Cross-domain data compilation: Integration is a main issue in an era where primary digitization improves interoperability of existing assets. In the 3D reconstruction context, several domains (e.g., GIS, CGI, CAD, BIM) all contribute relevant information. The postulation includes more than the model’s properties and embodies traditionally or automatically generated para- and metadata, even incomplete sets.
• Modular middleware to replace single platforms: Enhanced flexibility in terms of input and output along with use of established portals and existing technical solutions can probably boost a fresh population of repositories and consequently reach critical user attendance, crucial in all cost/benefit considerations. A conservative approach focusing on existing structures favors user-friendly middleware (as a broker) and stable APIs. Commercial platforms with high impact on the public may frequently outperform academic counterparts with very limited usage.
• Persistent and multiple data access: Data lifelines and accessibility should be extended as far as possible, whereas redundancies have to be avoided to prevent use of outdated versions. Access to a specific asset can directly reach the right partner with a clearly defined authorship, responsibility, and identifiers. Existing commercial and academic platforms may provide information about projects (metadata), including links to the provider. Asset handling will largely profit from established systems like DOI.
• Data authority of the authors: Access to digital assets should be fully controlled by the author. Citation of digital objects, a correct reference to authorship, might be handled similarly to how DOIs work in connection to publications.
• Generating additional value: Data suppliers benefit from secure archiving and structuring. Data integration into a broader context substantially adds value. Other advantages could well be a commercial marketing of selected data. Gamification and crowdsourcing are further elements from which research can greatly benefit.
• Multiple ways to access information: A digital information repository should reflect the heterogeneity of purposes, disciplinary approaches, and data qualities by offering several modes to access and individual prerequisites. Aspects of user experience, accessibility, and prominence play important roles in proposing digital platforms.
• In-depth 3D data classification: Since 3D content is currently classified primarily on an object scale, segmentation and classification of parts of 3D models from point or polygon cloud data enables more detailed knowledge integration and comparison.
• Knowledge integration: A major issue is to link models and knowledge bases, such as sources and information used for a 3D reconstruction. In this context, Linked Open Data (a recent review: [657]) is a future-oriented technology for knowledge organization and formalization in structured data models.

10.2. Legislation

Legal aspects in 3D modelling concern several dimensions—the most relevant are copyright and design laws [163,658]:

• Since older cultural heritage objects are now public domain [659], younger objects may be subject to copyright. Especially for museum objects, other legislation on property rights may effectively prevent 3D digitization without an exhibitor’s permission [660].
• Reproduction media such as photographs potentially become an original object and adds IP for both copyright and design laws. The legal situation is uneven, not least with regards to the level of originality necessary (especially in digitization of analogue heritage objects, e.g., plans) to gain IP for media depicting or digitizing a cultural heritage object [661,662].
• The 3D model may be subject to copyright [663]. It is currently being debated whether 3D digitization fulfils requirements for being original or is simply copying and therefore “creates only non-original works” [664]. Although the situation for 3D reconstructions seems more clear, as substantial originality is inherent, not all national legislation sees reconstructions as original work [665].
• 3D printing is subject to specific legislation. Specific overviews about copyright in the EU for 3D printing date from 2010 [666], 2014 [667], and 2019 [668].
• Recent developments at EU level [45], the Open Data Directive [669] and the Digital Single Market Directive [670], provide political and legal framework conditions for digital humanities and digital heritage research. Despite this EU-wide attempt, the current situation on member state level is very uneven [45].

10.3. Teaching

Although the following paragraph focus on teaching 3D modelling in higher education, there are various studies about the education of 3D modelling skills in school education [671,672], as well as in vocational training. For the latter case, required competencies for cultural heritage professionals were investigated during the European Year of Cultural Heritage in 2018 [673] with an analysis of member states [46].

What is the current state of education in digital cultural heritage and digital humanities in higher education? Digital literacy and technology education has gained much relevance in humanities and heritage-related disciplines during recent decades. Against this background, we examined the current state of educational programs in digital cultural heritage (DCH) and related disciplines (primarily in Europe with supplemental information from the US), highlighted core topics, challenges, and demands, and showed innovative formats and prospects [674]. Within Europe, there are several opportunities to study digital heritage. For this article, options at Bachelor’s, Master’s, and the doctoral level were examined using study portals as three European platforms within a partnership with institutions like the EC, DAAD, British Council, and UNESCO Institute for Lifelong Learning that provide prospective students from all over the world a curated review of study courses [675,676,677]. For DCH, we do not claim that these results are complete, but sketch an overview of possibilities for studying DCH in Europe. The search term at all three portals was “digital heritage” and led to 267 mentions (for comparison: “Cultural heritage” attained 555 mentions for bachelor’s studies and 585 mentions for master’s studies in Europe). More than half of the DCH education options in Europe comprise a Master’s degree (a total of BA: 119, MA: 136, PhD: 12). One possible reason for this could be the strong orientation toward research in heritage fields combined with digitally based infrastructures. Digital methods are primarily used in research settings [678], which is more relevant at the Master’s level than in Bachelor’s programs (c.f. Figure 18).

The countries in Europe where DCH or related subjects are studied are Austria, France, Greece, Hungary, Ireland, Italy, the Netherlands, Poland, Russia, Slovakia, Spain, Sweden, Switzerland, Turkey, and the UK. The UK offers the largest number of DCH programs and is unique in that the majority are at the Bachelor’s rather than the Master’s level (c.f. Figure 19). In European countries, Bachelor’s programs are minimal or unavailable (e.g., Sweden or The Netherlands).

At all levels, DCH is mostly connected to humanities disciplines (total 141); arts, design, and architecture (total 83); and social sciences (total 79). In addition, applied sciences, business management, computer science and IT, hospitality, leisure and sports, as well as journalism and media are relevant sub-disciplines where DHC is assigned (c.f. Figure 20).

10.4. Research Spotlight: Education

For more than a decade, it has been argued that students in higher education need to attain skills necessary for employment, in addition to acquiring specific expertise and knowledge in their discipline [679]. The shift towards an approach that will focus more on the development of practical skills and not exclusively on academic competencies indicates that employers prefer graduates with communication skills and teamwork experience [680]. Based on the assumption that skills for interdisciplinary project work are best acquired within a practical project [681], a guide for such projects might encourage more teachers to use them as a practice-related educational strategy. A common approach to acquiring skills dealing with complex and multi-perspective problems is project-based learning (PBL) or “learning by doing”, originally introduced by Dewey [682]. By definition, PBL is an individual or group activity for an extended period with a specific result like a product or presentation, which drives the planning, production, and evaluation process [680]. It is an instructional, “student-centered strategy in which students determine what they need to learn” in order to solve nontrivial, real-world problems and where “teachers act as facilitators” to foster both problem solving and learning progress [683]. Usually, more than one approach or answer will solve the problem and students are encouraged to seek out multiple sources of information and evaluate different methods. PBL promotes skills like cooperative ability, critical reasoning, creative thinking, responsibility, and communication [684,685]. In comparison to other learning designs, PBL promises a greater satisfaction, student engagement, and long-term retention of acquired competencies [681]. We adopted and evaluated team project-based learning approaches to support student education in digital 3D modelling. As observed in two courses so far, strategies for cooperation within student project teams for creating virtual representations evolve slowly, and mostly as a reaction to upcoming problems and demands. Related competencies are highly based on implicit knowledge and experience. Consequently, teaching best practices prior to project work is less effective than coaching during the project work [686].

11. Implications

The described research activities are proposed to contribute to a comprehensive discourse on the unique epistemology of digital 3D modelling in humanities. This will deliver practically tested implications for software design, education, and organizational settings. Some overarching findings are highlighted in this chapter.

11.1. Organizational Perspectives on 3D Modelling

From an organizational perspective on digital 3D technologies, both an academic culture and institutionalization are currently lacking. Other than in text-related disciplines, knowledge is mainly gained by the creation of a virtual model and its digital, in most cases visual, demonstration in the case of digital reconstruction. Contributions of different authors and a multiplicity of intuitive decisions are included in such media, which are based on know-how [24]. So far, neither academic nor institutional mechanisms make digital 3D models scientifically linkable and discussable. An adjacent discourse is currently about the definition of “scientific 3D reconstructions” and 3D scholarly editions [687] to define requirements for 3D models in scholarly contexts.

11.1.1. Infrastructures

Access to 3D models is needed to evaluate and assess the quality of the 3D data. In case of 3D reconstructions, this also includes highlighting the knowledge and sources basing the modelling process. In addition to a number of technical requirements described in the following section, this implies a need to develop documentation of processes and their results and to increase capacity of making a model logic transparent [263,319]. A recent overview on 3D information systems and user demands revealed especially the high demand for open and accessible 3D content [688]. Beside various infrastructure projects on European level [689,690], in Germany three projects of the German Research Council (DFG) are developing infrastructures for 3D reconstruction—one of them dedicated to enhancing the DFG Viewer by 3D viewing capabilities [691], the others focus on the documentation of the modelling process [692] and a virtual research infrastructure to connect the 3D model with further knowledge bases [693].

11.1.2. Cross Media Linked and Authority Data

Since metadata for digital objects is well established, 3D content segmentation and indexing are still unsolved these days, although this is the biggest community demand for standardization [45]. Metadata provide a widely established concept for documenting processes and their results and have the capacity of making a model logic transparent [263,319]. A main distinction is between comprehensive reference ontologies optimized for domain-independent compatibility and custom-designed domain ontologies as modifications of reference ontologies to serve specific purposes [694,695,696]. An important prerequisite in this respect is to identify and link elements across media (as overview: [657]). Against this background, authority data is important to avoid data silos and to link enclosed projects across different media [697]. An overarching challenge is to classify 3D content information. Simple structures [698], but also complex objects such as buildings [699,700] can be automatically segmented and assigned from datasets created by imaging processes [456,457]. Inferences can also be made on which parts of the image reference which parts of the 3D object geometries [701,702]. Machine learning is playing an increasingly important role in image segmentation and object recognition [703,704] as well as classification of unstructured 3D data [416,705,706,707,708]. A current extension is to investigate the multimodal 3D retrieval and cross-validation comprising 3D data, images, and texts [709].

11.1.3. Disciplinary Anchoring

While single references to topics of digital reconstruction such as museology and archaeology have been taken up by panels and workgroups anchored in these fields, in the German-speaking area, there are—in comparison to textual digital humanities—still few academic chairs particularly dedicated to digital 3D modelling. From a disciplinary perspective, it is also still an open question how 3D modelling in the humanities will be anchored—as part of the digital humanities, of digital heritage studies as heritage sciences, or “repatriated” into the original disciplines? In 2021 the trend seems to be towards parallel multiple options, but this is a still ongoing process.

11.1.4. Internationalization

Highly specialized and small sub-communities dealing with 3D modelling on specific topics like urban history geomatics or technologies (e.g., BIM) do not involve many people (c.f. [120]). Thus, cross-national exchange is essential, which may be why digital humanists consider themselves as internationally well linked. This high level of linkage includes external institutions such as museums, archives, and galleries. Despite recent policies and initiatives, the EU has sought to develop a holistic and complementary approach to create a strong system for cultural heritage-led innovations across Europe. Legal initiatives and directives are still fragmented and not fully synchronized between the member countries, compounded by a lack of mechanisms to cope with 3D content [45,710]. This still poses various challenges to cross-national innovation. Policies influence research topics; this is currently under investigation in a set of studies conducted by my group [44]. Cross-national collaboration in research as intended by the European Commission has led to cooperation patterns in research publications that differs from other continents: fertilizers are no longer the proximity or shared language of the co-authors, but their location is within the EU. It will be interesting to see whether current policy interventions—e.g., the strong link to impact and valorization in EU Horizon Europe or the COVID-19 Recovery Funds—will shape research topics and cooperation patterns similarly to the cross-national cooperation required for projects in many EU funding programs.

11.2. 3D Modelling in Education

Despite various attempts and projects to employ digital 3D modelling in education, teaching digital 3D technologies still plays a minor role in academic contexts. With regards to functions, in the context of higher or museum education, these methods are primarily used to illustrate and teach history. Another increasingly important scenario is to encourage students to research and explore historical objects by reproducing them virtually [256]. In a third usage scenario, the primary subjects of teaching are digital 3D reconstruction techniques, while the historical object is used “only” as a training example to acquire technical or practical skills (See also [711]).

Moreover, digital 3D reconstruction techniques were employed in various educational settings focusing on the process of reconstruction of cultural heritage objects (e.g., [4,712]). Despite these activities, there is still no wide consensus on establishing a specific digital visual humanities education paradigm, and larger studies on the education of digital methods in visual humanities are still needed [712].

11.2.1. Core Curricula

Numerous professional disciplines are engaged with cultural heritage and its digitization. Consequently, widely differing skills and competencies are needed, which has drastically changed in the digital age. A spectrum of DCH competencies comprises digital data curation, communication, and tools [713]. Although digital competencies are essential for heritage professionals at all career stages [351], training in digital heritage is still mostly provided through academic courses and curricula. How is the situation in related fields? Competencies in cultural heritage education are described in several publications, both recently [714] and previously [715,716]. Existing programs encompass topics about certain cultural artifacts, implementation in organizations and institutions, and technical research, use, and publishing requirements (e.g., [717]). The latter include producing digital knowledge, analyzing and visualizing, and management (e.g., [718]). Concerning digital archaeology, the International Campus of the Spanish Society of Virtual Archaeology, the Virtual Archaeology International Network, INNOVA, and 18 research groups from 14 universities [719] have created a core curriculum for Spanish universities. A similar curriculum has been developed for digital curation (c.f. [720]) and investigated within the DigCurV Curriculum Framework [721,722]. Although these frameworks target similar sets of competencies, they do not focus directly on DCH.

11.2.2. Competency Centers

Competency centers are hubs for servicing cultural heritage, offering technical support, infrastructure, and training. Competency centers have been well established for DH (c.f. [723]) and DCH on national levels, as in Sweden [724] for more than a decade. Since 2020 the concept of competency centers has been under development to cover the European level with regards to heritage conservation [725].

11.2.3. Lifelong Learning

Competencies in both digital and content-specific areas have been named the most important challenge for the cultural heritage sector [351]. Several EU research frameworks [726,727,728] acknowledge that learning, working, and living are closely connected. Due to digitization, life is changing rapidly, and it is essential to evolve teaching concepts in response to this phenomenon. The European Heritage Strategy for the 21st Century [729] mentions the need to incorporate heritage education into school curricula. In addition to subject-specific and general key qualifications, some superior frameworks record overarching digital competencies, e.g., for lifelong learning [730]. For example, the Italian national plan for the digitization of cultural heritage [731] aims to provide the multiple offices operating in cultural heritage with a model for managing digitization processes, to accelerate their development and facilitate access to the services offered (c.f. [674]).

11.2.4. Citizen Science and Education

Finally, citizen science approaches have found their way—as mentioned in Section 5.6—into the research and teaching of various university disciplines in recent years [732], including digital heritage (e.g., [733]). In this context, knowledge transfer to citizen scientists creates particular didactic challenges [734]. This is not least because the target or interested group for cultural heritage projects are often not digital natives and thus have to be addressed via other channels [735]. Knowledge transfer via digital citizen science activities nevertheless appears to be a promising approach, as public involvement in generating knowledge increases willingness to engage with cultural heritage.

11.3. Design and Technological Implications

11.3.1. Fit-for-Purpose 3D Visualizations

Regarding design implications for digital tools, research environments, and products, the main specific requirements of digital reconstructions are space- and time-related classification, as well as identification and visualization of created digital models, related (source) materials (e.g., using world-wide valid unified resource identifiers), and their relationships. Digital 3D techniques have been developed by using a multiplicity of different technologies from domains such as GIS, CGI, CAD, and BIM or CityEngines, which are not very compatible [24] and not convertible without loss. Adjacent tasks are assessing, developing, and spreading technologies and strategies on the interoperability of data. Therefore, there are special requirements concerning the interactivity and simulation quality of a created visual experience. A specific challenge is to visualize various levels of accuracy of knowledge extending from authentic finds, which may be contradictory [536,736,737,738] to scientific hypotheses. This comprises shapes, colors, and lighting situations [550]—i.e., naked flames as the most common historical lighting of interiors—in terms of historical life and behavior [126,739,740] and representation involving multiple senses [473].

11.3.2. Validated Design Patterns for 3D/4D Heritage Presentation

Visually performed research and learning tasks are manifold with regards to 3D modelling in the humanities—the analysis of gaze relations in urban planning ensembles, the visual evaluation of differences in historical architectural photographs, or the search for building phases and their sequences in architecture. Since pattern libraries for 2D (e.g., [741,742]) and contemporary 3D visualizations and interfaces (e.g., [485,743]) are already well investigated, a typology of validated strategies for the visualization and interaction with 3D and 4D data in cultural heritage is still missing. Despite the intense research shown in Section 8.2, it has rarely been empirically examined what are requirements to the design of 3D/4D visualizations and interfaces to serve visual research and learning and what effect visualizations of historical objects has on the user. Concerning the linking between physical and virtual worlds, a next phase may be enabling cyber-physical spaces [744] also for cultural institutions in which humans interact with virtual systems via sensors, actuators, or smart devices and receiving personalized experiences [745].

11.3.3. 3D Modelling from Sparse Sources

Despite long-term technological evolution, some fundamental challenges still hinder 3D modelling and particularly 3D reconstruction on a large scale. Especially machine learning-based approaches are promising to overcome the issue of labor-intense manual model generation in case of no more extant objects. Typically, AI-based technologies require large-scale training data. Currently, these technologies can only recognize well documented and visually distinctive landmark buildings [464,746], but fail to deal with less distinctive architecture, e.g., houses of similar style. To overcome the problem of very small training datasets, single or even zero shot learning have been tested for cultural heritage [747,748] to enhance the quality of models trained on sparse data. In another approach, single photo 3D reconstruction [466] has been successfully applied to retrieve 3D information from single images.

11.3.4. Digitizing Everything

Let me close with a possible utopia and/or dystopia of “digitizing everything”. This universal Mirrorworld approach [292] (describing the data utopia of a ubiquitous data space) and the closely related Metaverse [749] (a social utopia of an “always on” virtual environment or social space) was recently pushed by pandemic de-socializing, but also by the trend towards ubiquitous platform models. Virtual visits to tourist destinations [750] and 3D modelling from crowdsourced imagery [751] are only two small—heritage specific—facets of how people contribute to or utilize “digitizing everywhere”. The future will reveal the implications of this development [752]. This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.