Elsevier

Applied Energy

Volume 327, 1 December 2022, 120049
Applied Energy

How close are urban scale building simulations to measured data? Examining bias derived from building metadata in urban building energy modeling

https://doi.org/10.1016/j.apenergy.2022.120049Get rights and content

Highlights

  • Measured building energy use was compared to building energy simulations for more than 50 thousand buildings.

  • Urban scale models building energy simulations effectively capture the distributions of aggregated buildings.

  • Individual building variation is not well captured by uncalibrated urban scale models.

  • Current urban scale data sources are a primary reason for poor individual building characterization.

Abstract

Residential and commercial buildings in the United States accounted for 40% of total energy in 2020. Building energy modeling (BEM) is a useful tool that allows individuals, researchers, companies, or utilities to save energy by optimizing buildings through estimation of building technology savings and performance projection of building energy under various environmental conditions. Urban building energy modeling (UBEM) expands the scope beyond individual buildings to the buildings in a neighborhood, city, utility and more. Yet there is a knowledge gap in the literature as to how these models compare to measured data on an individual and aggregated basis. As UBEM data and methods continue to develop, it is important to consider the accuracy, bias, and limitations of the models. Nation-scale data and UBEM software suite named Automatic Building Energy Modeling (AutoBEM) was used to model 50,843 buildings in Chattanooga, Tennessee. The uncalibrated simulation results were compared to aggregated 15-minute electricity data for the year 2019 with visualizations highlighting sources of bias in building data and the AutoBEM framework while considering how they relate to other UBEM methods. Estimation of building type and year of constructions are found to be the major sources of bias. Accounting for the amount of conditioned area per building significantly improves the overall fit of the simulated energy use intensity. it was found that inherent variation in building energy use contributes to R2 values between 0.008 and 0.095 across building types but slope values near 1 for the total number of buildings. This indicates the need for building aggregation for representative building energy modeling with data sources available at an urban scale while illustrating the need for additional individual building data and model improvement beyond the originally produced UBEM models for individual building analysis.

Introduction

With residential and commercial buildings in the United States contributing 39% of carbon emissions, 73% of electricity use, and 80% of electrical demand in 2020, buildings are the most significant opportunity for sustainability. Urban building energy modeling (UBEM) is a growing field within building energy modeling (BEM) for which thousands to millions of buildings are modeled to maximize modeling impact. Creating representative building energy models allows one to understand how different building types, systems, and technologies will perform under various conditions throughout a country. Modeling individual buildings has the potential to enable actionable decisions at the building or portfolio level, while stimulating private sector activity to refine such models toward investment grade status required for financing building retrofits. Modeling individual buildings at city- to nation-scale has the potential to impact energy, demand, emissions, and costs while informing coupling of buildings with the electric grid for resilience. Such building-specific models can be mapped to feeders, substations, or other critical infrastructure and constitute valuable tools for a utility to assess load throughout their service area and facilitate load shaping to cleaner or cheaper generation assets.

There is significant complexity when developing a representative building energy model as no two buildings perform exactly the same. Some of the bias that is attributable to the unique energy performance of each building can be eliminated through simulation adjustment or calibration to measured data, but performance data is rarely available for each building. For this reason, it is valuable to understand and quantify the sources of inherent bias for uncalibrated UBEM. Comparing measured and simulated (predicted) annual energy use intensity (EUI) is useful as it allows buildings of all sizes to be compared and better understand sources of bias. While previous UBEM has validated individual buildings for hundreds to two thousand buildings against monthly or annual data, this paper validates 50,843 building energy models against sub-hourly (15-min.) measured data and analyzes systemic bias.

This analysis investigates bias and the sources of bias in Automatic Building Energy Modeling (AutoBEM) [1], a UBEM framework that has been used to model hundreds of millions of buildings since it was developed including the creation and sharing of a model of 122.9 million buildings (97.8% of the United States) [2]. In one analysis, AutoBEM was used to model nearly 180,000 buildings of the Electric Power Board of Chattanooga, Tennessee (EPB). In close partnership with this utility, related research includes validation against measured data [3], [4], estimates of building retrofit technology savings [5], [6], and forecasting of building energy use under various climate scenarios [7].

AutoBEM is a bottom-up, physics-based UBEM framework. These modeling frameworks produce a physics-based building energy model for each building in the area of interest. This method has many advantages including individual building granularity and the ability to modify the building energy model to evaluate different building technologies in an effort to save energy, demand, cost, and emissions. As examples This allows the user to determine which buildings are good candidates for a certain technology or understand how grid resilience will be impacted by a heatwave. If individual building resolution is not necessary, top-down UBEM approaches can be used. These approaches are typically data-focused and are at a municipality scale, often incorporating economic, transportation, and other pertinent urban data into their predictions.

The measured data from the utility is used to examine the sources of bias in the AutoBEM models and input data. AutoBEM used a variety of data sources to generate a building energy model for every building in EPB’s service area, simulate with weather data from the same year as measured data (2019), and compare simulation outputs to measured data. Annual building EUI is used to compare buildings in this area with the models generated using AutoBEM as well as the typical U.S. building stock.

While this analysis covers a single UBEM framework, many sources of bias are applicable to common UBEM methods. Reviews such as [8], [9] contain many state of the art UBEM methods and a fairly comprehensive summary of UBEM efforts is contained in [10]. Many of the methods utilize similar physical building data, building energy modeling simulation engines, and related data that likely leads to similar sources of bias. The authors of this work anticipate and encourage other studies to reproduce and compare against the findings of this paper as a step toward establishment of best-practices and standards for UBEM.

UBEM biases are introduced into the simulated data from a variety of sources. Simulation results are likely to be heavily influenced by occupant behavior not just at the individual building level but at the urban level [11]. The influence of occupant behaviors on UBEM simulation results is primarily from the challenge of accurately and fully incorporating diversity in equipment schedules and other energy-related occupant activities into building models’ EUI or timing of energy use. This is relevant in residential districts where occupancy can significantly affect peak demand and energy use [12]. Some of the occupant behaviors vary from passive exchange of heat with space; opening and closing doors and windows; adjustment of thermostat settings, light setting, blinds and shades, or clothing levels; use of personal heating and cooling devices; and consumption of warm or cold drinks’ [13]. Many studies assert that the factors which motivate occupant activities in buildings are many and include occupants’ personal choice, environmental behaviors, social interactions and the general cultural context [14], [15], [16].

Another source of bias in UBEM simulation data is using typical weather environment (or representative city) for an entire district or city without considering the true meteorological conditions at that place and time—including microclimate variation between the building and closest weather data (usu. airports) [11]. The urban climate and microclimate, and how they interact with building models, can have a significant impact on building energy use [17]. These conditions can affect the air temperature and humidity of the local environment, movement of air, solar radiation as well as specular and diffuse reflections [18]. There is also a reverse action of buildings on the boundary conditions of urban canopy modes in the following ways: (1) the building geometries and layout have thermal effects on heat and airflow [19]; (2) heat exchange between the buildings’ exterior surface and the urban environment through simulation of sensible heat convection and radiation; and (3) the release of heat and moisture by buildings to the urban environment through cooling towers, condensing units and exhaust air [11]. The bidirectional impacts of buildings and the urban environment on one another needs to be factored properly into the assessment of UBEM. To demonstrate how the use of more weather stations can reduce temperature bias, [20] used all available weather stations within the IECC climate zone over the western U.S. in a UBEM and this led to a reduction of the average absolute summertime temperature bias from  4.0 °C to  1.5 °C.

The distribution of internal loads in buildings can vary widely from one building to the other, even when they are of the same type and vintage. This can lead to significant discrepancy between simulated energy use and what is actually consumed [21]. While energy use is in buildings are more complex, modeling uncertainties often lead to a simplification of mechanical systems and their controls which fail to fully capture the dynamic behavior or part-load use of the mechanical systems including the response from their control systems [22]. Parameters related to specifications for HVAC use, systems and plants scheduling, and casual gains can all lead to an incomplete or inaccurate modeling of buildings energy systems [23]. [24] detected performance gap in thermal modeling and linked it an underestimation of the values used for specifying fan powers in simulation compared to actual use. Also, [25] identified a similar underestimation of lighting loads as a significant source of source bias in simulated building energy use.

Several other building energy researchers have explored the relationship between measured and predicted EUI. A review of UBEM techniques in 2016 aggregated error rates for UBEM applications of the time and reported average error ranges well above 10% [26]. The UBEM applications in this study also were relatively small compared to today’s standard, with the majority of studies focusing on less than 100 buildings. Another more recent review of 78 UBEM analyses reported error ranges from 1% to 1000% for single buildings [27]. A recent study of 1.2 million London residential buildings compared measured EUI to the UK’s Asset Rating tool value [28] with a weak correlation indicated between the measured data and the building efficiency rating values. An older work compared measured and predicted daily EUI for the city of Houston, TX. It was found that the daily predictions were typically within 10% of existing building types. Because of their daily scale, a focus on hourly patterns was evaluated with general prototype building schedules specified as a primary source of error in the predictions. A study of 172 residential buildings in Kuwait emphasized the need for stochastic occupant modeling in UBEM, showing that deterministic models ignore uncertainties up to 30% when considering single buildings [29]. A 2016 review evaluated the measured vs simulated EUI of building energy modeling tools for various building types all over the world. While not a UBEM review, this work highlights several sources of bias in individual building energy modeling that naturally relate to UBEM [29].

While there are many comparisons of simulated to measured building energy use in the literature, few of these exercises have been done for multiple buildings, let alone at an urban scale. This analysis compares tens of thousands of simulations to measured building energy data to better understand the strengths and limitations of UBEM frameworks as well as typical data that must be used at the urban scale.

Section snippets

Materials and methods

To assess differences in simulated and measured data, individual building simulations were compared to measured meter electricity data for a sample of real buildings. In partnership with EPB, measured electricity data for the year 2019 was shared from smart meters in Chattanooga, Tennessee for 50,843 buildings used in this study. The data was shared as 15-minute electricity data but was aggregated and scaled by the area to be analyzed as annual electricity use intensity.

AutoBEM utilizes

Results

Measured data from 50,843 selected buildings in EPB’s service territory is compared to individual simulations with visualizations for illustrating how well the models represent the measured data. For the following visualizations, the EUI values consist of only electricity. The models were converted to all electric HVAC and water heating for a better comparison to the all electric data measured data. The lack of natural gas data is considered in a bias adjustment further in the document.

The data

Discussion

There are a few main sources of bias that are evident from the results. While there may be some bias in AutoBEM (or UBEM frameworks generally), the majority of the bias is derived from the input data, which has a huge impact on individual simulation performance compared to measured data. There are two main sources of bias in the input data: missing or limited data describing each building and mis-labeled data.

The first source of bias related to input data is missing or limited data. This data

Limitations

There are several major limitations of the UBEM study. First, reliance on (only) electricity data limits the broader applicability of the work. Moreover, having other energy types would allow for a more reliable comparison between models and measured data. Second, use of general building characteristic data sources is limited, difficult, and intractable for many organizations. While thousands of inputs are defined for each model, only a few are directly measured, and most are interpreted or

Conclusions

This article compares measured and simulated building energy data for more than 50 thousand buildings in Chattanooga, TN. There are several primary findings from this work: (1) An uncalibrated UBEM framework (AutoBEM) is compared to measured data to evaluate its performance, both on individual buildings and the aggregated building distributions. Individual results were compared using scatter plots and trendlines while the distributions were compared using box-plots. It was found that the

CRediT authorship contribution statement

Brett Bass: Conceptualization, Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Visualization. Joshua New: Conceptualization, Writing – review & editing, Supervision, Project administration, Funding acquisition. Nicholas Clinton: Data curation, Writing – review & editing. Mark Adams: Software, Development. Bill Copeland: Data curation, Writing – review & editing. Charles Amoo: Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was funded by field work proposal CEBT105 under US Department of Energy Building Technologies Office Activity Number BT0305000, as well as Office of Electricity Activity Number TE1103000. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive,

References (50)

  • OraiopoulosA. et al.

    On the accuracy of urban building energy modelling

    Renew Sustain Energy Rev

    (2022)
  • SzcześniakJ.T. et al.

    A method for using street view imagery to auto-extract window-to-wall ratios and its relevance for urban-level daylighting and energy simulations

    Build Environ

    (2022)
  • KangJ. et al.

    Building instance classification using street view images

    ISPRS J Photogramm Remote Sens

    (2018)
  • SalimF.D. et al.

    Modelling urban-scale occupant behaviour, mobility, and energy in buildings: A survey

    Build Environ

    (2020)
  • Mosteiro-RomeroM. et al.

    A novel population-based occupancy modeling approach for district-scale simulations compared to standard-based methods

    Build Environ

    (2020)
  • GongP. et al.

    Annual maps of global artificial impervious area (GAIA) between 1985 and 2018

    Remote Sens Environ

    (2020)
  • NutkiewiczA. et al.

    Data-driven urban energy simulation (DUE-S): A framework for integrating engineering simulation and machine learning methods in a multi-scale urban energy modeling workflow

    Appl Energy

    (2018)
  • NewJ. et al.

    Scaling beyond tax assessor data

    (2020)
  • NewJ. et al.

    Model America - data and models of every U.S. building

    (2021)
  • GarrisonE. et al.

    Accuracy of a crude approach to urban multi-scale building energy models compared to 15-min electricity use

    (2019)
  • GarrisonE. et al.

    Quality control methods for advanced metering infrastructure data

    (2021)
  • BassB. et al.

    Potential demand reduction from buildings in a simulated utility

    (2019)
  • NewJ. et al.

    Distribution of potential savings from urban-scale energy modeling of a utility

    (2021)
  • BassB. et al.

    Future meteorological year weather data from IPCC scenarios

    (2020)
  • ReynaJ.

    U.S. building stock characterization study: A national typology for decarbonizing U.S. buildingsNREL/TP-5500-81186

    (2021)
  • Cited by (9)

    • Estimating cooling loads of Arizona State University buildings using microclimate data and machine learning

      2023, Journal of Building Engineering
      Citation Excerpt :

      They found that adding 5% mature trees reduced cooling loads by 2.7% in July. Results from these studies may be highly influenced by potential bias that exists within BES models that might not be evident [6]. For example, a modeled building could be validated with actual data using statistical indices, but the model might inaccurately simulate wall/roof insulation or air ventilation.

    View all citing articles on Scopus
    View full text