Sage Journals: Discover world-class research

Abstract

Increasing the uptake of retrofit heat pumps in the UK is essential for the decarbonisation of heat in the residential sector, which will require greater certainty in the ability of heat pumps to deliver cost effective comfort conditions and carbon emissions reductions. Heat pump performance is linked to the quality of design and installation practices, and users need assurances that predicted performance can be achieved. This paper presents a sensitivity analysis of energy prediction for domestic properties against the variation of five key input variables: heat loss coefficient, set point temperature, system coefficient of performance, casual gains and fabric thermal mass. The energy prediction model used was a degree-day based simplified predictor model that explicitly models the impacts of the control regime (fixed time start, night set back or optimum start). This is a feature not typically available for widely adopted steady state energy models. Modelling forecast errors are shown to be highly significant for heat loss coefficient, set point temperature and coefficient of performance, leading to the need for greater accuracy in the accuracy of these input variables.

Practical application

Selection and operational control of heat pumps differ from traditional heating systems such as gas or oil fired boilers. Correct sizing of heat pumps is much more critical to their operational performance than other systems, and design methods require more certainty in their forecasting of economy and carbon emissions, especially for retrofit applications. This paper indicates the range of input errors that can occur, and the impacts of these on performance forecasts.

Keywords

Heat pumps energy performance sensitivity analysis prediction errors degree-days

Introduction

Decarbonising of heating energy is a key challenge in the buildings sector, with heat pumps being a prime technology of choice in many mature economies with significant heating demands. In the UK this has meant an ambition to install 600,000 domestic heat pumps per year by 2028.¹ While a proportion of this total will be for new homes, a significant number will be via retrofits to existing properties, for which government grants are available. Essential to building confidence for heat pump retrofits is the reliable estimation of expected running costs and carbon savings, together with assurances that thermal comfort conditions can be adequately maintained. Sizing of heat pumps is a critical consideration as over-sizing can lead to system inefficiencies, while under-sizing can compromise thermal comfort. Heat pump efficiency (the coefficient of performance or COP) varies over the heating season, and it is important to match optimum COP to the most frequently occurring conditions. For new builds heat pump sizing can be relatively assured, whereas the diversity of age, style and condition of existing domestic buildings make retrofits more challenging. This requires decisions around the need for fabric improvements, heat emitter upgrades, system size and control strategies, all of which have cost and efficiency implications.

The margins of error are much less forgiving for design of heat pumps systems than they are for fossil fuelled boilers. This paper examines how errors and uncertainties in the assumptions about the thermal characteristics of a building will impact on the modelling predictions of retrofit heat pump energy performance. Errors in these assumptions will directly lead to modelling errors in forecasting and may also lead to unpredictable operational performance due to incorrect system sizing. Simple heating energy models – as opposed to dynamic thermal simulation models – require fewer inputs and are much easier to manipulate for sensitivity analysis against key input variables. These are generally steady state, and do not incorporate the dynamics of different control regimes. A simplified energy prediction model that has been calibrated against a number of different domestic properties is used in this study to show how heat pump performance prediction can be radically in error depending on the magnitude of the inputs, pointing to the need for reliable and accurate thermal characterisation of a property. The model incorporates quasi-dynamic principles that can account for thermal mass effects and differing control strategies. It is not designed as a design heat loss calculator, but can be used to augment design tools in assessing energy performance.

Context and literature review

Determining the thermal characteristics

Designing a retrofit heat pump system necessarily starts with an assessment of the thermal characteristics of a building including rate of heat transfer (conduction and convection), and ability to store heat. There are two distinct methods for determining these characteristics: calculation and measurement, with the former being the most widely practised to date. Calculations are generally based on a site survey and will usually require a number of assumptions including: amount and type of cavity wall and floor insulation (invasive inspections are rarely carried out), the rate of air infiltration and its variability, and the effective thermal storage capacity of the building (if it is considered at all). An experienced surveyor can make reasonable judgements about insulation levels, but air tightness and infiltration estimates can be much more subjective. Air tightness tests are rarely conducted for domestic retrofits, and lower cost alternatives such as tracer gas techniques (for example using CO₂ concentrations) can be unreliable. The influence of wind speed and direction are impossible to calculate with any certainty, and their impacts will depend on orientation and build quality. Therefore, the difficulty in determining the accuracy of air infiltration adds very significant uncertainty to heat loss calculations.

The most widely used calculation methodology in the UK is the Reduced Data Standard Assessment Procedure (RdSAP)² which is the official method used to generate Energy Performance Certificates for existing dwellings. RdSAP is underpinned by the calculation methodology SAP 10.2,³ which is due to be replaced in 2025 by the Home Energy Model (HEM).⁴ Other calculation tools exist, for example the Passive House Planning Package (PHPP)⁵; some heat pump and boiler manufacturers and energy assessors provide their own calculation tools. The underlying physics of these models is the same, but input interfaces, default values and processing algorithms differ, which can affect the results arising from different models. Time frame resolution is an example of algorithmic difference between models. For example, the HEM has a half hourly resolution (with steady state calculations), whereas SAP 10.2 uses monthly average values, and most dynamic thermal models have resolutions in the order of minutes.

Validation work on HEM compared the model to two other steady-state models, PHPP and SAP 10.2, and one dynamic simulation tool ESP-r.⁶ The variation observed in the results between models depended upon house typology, changes to default values and input specifications. These variations ranged from a few percent to over 50%. The causes of the variations between models, and how much is attributable to the heat loss coefficient, are not transparent. It is not possible to conclude which model, and which input configuration, is the most accurate (even though dynamic models are often assumed to be superior, which may not necessarily be the case).

HEM was also compared against measured energy use.^7,8 For one set of tests the difference between HEM and measured energy was in the region of −17% to +20%. Other tests showed differences of around +/− 10%. Note that for these tests HEM used calculated heat loss coefficients. Some of the stated uncertainties in this work related to infiltration rates, although how much of the error is attributable to these could not be quantified.

Measuring heat loss coefficient (HLC)

For the reasons outlined above there has been a move to measurement of heat loss coefficients for domestic properties, particularly for heat pump retrofits where sizing is important. There are two distinct types of measurement:

• with a dwelling unoccupied and a well-defined set of tests, measurements and calculations, usually over a limited time frame;

• with the dwelling occupied and a variety of measurement and analytical methods.

The most long-standing measurement method is the co-heating test where a suitably prepared (unoccupied) property is heated to a fixed temperature with electrical resistance heaters for a period of time (a few days or more) to establish quasi-steady state heat transfer. Regression analysis is used to determine the HLC¹, with additional measurements and suitable corrections to account for solar and internal gains. More detailed expositions of co-heating tests can be found in.^9–11 The test does not reveal details about building thermal capacity, and is relatively expensive and highly disruptive to conduct. One study¹² found that the uncertainty in co-heating test estimates could be within the range of 8-10% where sufficient rigour is applied to the test phase.

An alternative method is the QUB (quick measurements of energy efficiency of buildings) test^13,10,11 which is shorter (one day) with analysis that includes thermal capacity effects. Under controlled conditions the QUB test has shown good agreement with the co-heating test, although the former may be less reliable in poorer thermal performance buildings.¹⁰ Both tests require supplementary air leakage tests to disaggregate air infiltration and fabric component heat losses.

The SMETER (Smart Meter Enabled Thermal Efficiency Ratings) programme evaluated eight SMETER technologies¹⁰ that provide methods for measuring the HLC of domestic dwellings. These technologies ranged in complexity from smart meter data only through to multiple sensors and additional controls, and were tested in 30 occupied houses. All houses were also subjected to co-heating tests, with some subjected to fan pressure tests. QUB tests were also conducted in some instances. All properties had gas fired boilers, with no heat metering. HLCs were presumably calculated taking account of the surveyed boiler efficiencies, but no detail on this issue is provided, apart from a table of SAP values of boiler efficiencies. The issue of assumed system efficiency (rather than measured) becomes critical for heat pumps as discussed in the next section.

The SMETER tests in¹⁰ provide a large set of results with varying degrees of agreement between the different SMETERs and the co-heating tests. Most have overlapping confidence intervals with the co-heating tests, but the confidence intervals are large (>40% in some cases), meaning a large spread in HLCs for many of the SMETERS. Mean estimates can vary from 4.1% to 27%. The SMETER results also showed variation with RdSAP calculated values, with some up to 26% difference, and most with confidence intervals greater than 20%. Given that this study was rigorously conducted and highly controlled it highlights the difficulty of obtaining values of HLCs that can be used with confidence.

Co-heating tests are not appropriate to conduct at the scale of retrofit work required to decarbonise heating in UK housing. Other longer term non-invasive HLC measurement methods have been developed^14,15 with reported uncertainties of between 12-30%. It is therefore important to understand the impact that uncertainty in HLC can have in the estimated forecasts of energy consumption of a building, particularly one that has been retrofitted with a heat pump. This is further compounded by the estimated change in HLC that will occur due to fabric upgrades of a building (see for example¹¹), which will give additional uncertainties.

The authors have reported their own methodology for HLC evaluation¹⁶ that employs a variable base temperature degree-day regression analysis with high resolution energy data. (The method is based on the same principles as the predictor model described in the following section). That study found the differences between calculated and measured HLCs as shown in Table 1 for 10 houses. These tests are based on smart meter and heat meter data in older (pre-1980s) properties, with additional sensors in the houses. The monitoring was conducted over several months, and in some cases two heating seasons. It is striking that all but two of the HLCs are lower than the RdSAP calculated survey values, and in some cases greatly so. This is at odds with the finding of Jack et al¹² who suggest that modelling under-estimates actual energy consumption, but this may be related to the types of property studied. It is important to note that if surveyed HLCs (from Table 1) are used to model the buildings, this greatly over-estimates energy when compared to the actual. There was no significant evidence of under-heating in these properties, and the measurement and analysis inherently accounts for internal temperature variation.

Table 1.

Differences between calculated and measured HLCs for 10 houses.¹⁶

Property identifier	Heating system	Calculated HLC (W/K)	Measured HLC (W/K)	% Difference
IC260	ASHP	359	371	3%
IC268	Gas combi	180	150	−17%
ICB6	ASHP	504	183	−64%
ICB24_1	ASHP	300	198	−34%
ICH63	Gas combi	319	329	3%
IK4	Gas combi	351	267	−24%
IL26	Gas combi	399	174	−56%
IS	Gas combi	531	147	−72%
ICB4	Gas combi	527	279	−47%
ICB24_2	Gas combi	372	248	−33%

The method is not fully developed, but it has the following advantages:

• It is based on long term metered data and can provide good characterisation of the heating system demand;

• Time frames can be changed (daily, weekly, monthly) to iron out short term thermal capacity and operational effects;

• Casual gains can be measured or estimated, with rapid assessment of their veracity;

• Outliers can indicate extreme weather effects and the impacts on HLC;

• Occupant behaviours may be identified.

Where the method fails this is usually an indicator of highly non-standard heating system operation, and further investigation is required to understand building, system, or behavioural anomalies.

The reported range of uncertainties in HLCs flows through to uncertainties in the estimation of future energy use, cost and carbon, which are major factors driving consumer take-up of retrofit. These are coupled with uncertainties in casual (mainly solar) gains, system efficiencies (typically not measured), thermal capacity effects and control settings and strategies. Unreliable forecasting of heat pump costs and carbon emissions will greatly undermine confidence in the technology. This paper examines how uncertainties in energy modelling outputs may be quantified due to variations to these five key input variables. Through systematic study it is hoped to provide an exposition of the potential errors that can accumulate in energy models.

The predictor model

The predictor model used in this study is based on methods set out in CIBSE Technical Memorandum TM41¹⁷ that uses mean values (in this case daily values) of internal temperature, casual heat gains and system efficiency to calculate the energy consumption as shown in equation (1).

E = \frac{24 H L C \sum ({\bar{θ}}_{i} - \frac{{\bar{Q}}_{G}}{H L C} - {\bar{θ}}_{o}) ∆ t}{η}

(1)

Where HLC is the heat loss coefficient in kW/K, Q_G is the casual heat gain in kW, η is the system efficiency (or COP) and θ_i and θ_o are the internal and outdoor temperatures respectively. Where the time period, Δt, is one day the summation is the number of degree-days, and multiplying by 24 converts the overall result to kWh. (For the best results, where possible, hourly outdoor temperatures are used to calculate degree hours, and summed over each 24 h period).

An important feature of this model is the calculation of the 24 h mean internal temperature, which is necessary for capturing the difference in operation and performance of heat pumps when compared with gas boilers, and for comparing operational control strategies. This temperature is dependent upon the heat loss coefficient, the building thermal capacitance (C) in kJ/K, the set point temperature and the outdoor temperature, as well as the rate and timings of heat input into the space from both the heating system (Q_H) and casual gains (Q_G). This level of complexity is what typically steers us towards dynamic simulation tools. The TM41 approach assumes that using average values at a suitable level of granularity can give similar results, while being easier to interrogate and simpler to validate for domestic building energy.

[Note: in our previous work¹⁶ the mean internal temperature was a directly measured value, which meant that thermal capacitance and heating system operation times did not need to be known explicitly when determining the HLC, although some assumptions had to be applied around the solar gain factor for a building].

Equation (2) shows the basis of the method where the rate of change of internal temperature over a defined period can be determined using the parameters set out above.

C \frac{d θ_{i}}{d t} = Q_{i n} - H L C (θ_{i} - θ_{o})

(2)

Where Q_in = Q_H + Q_G. This can be solved for various circumstances, for example when the heat input is zero (overnight cooling), or for a defined heating-up period in the morning when Q_H is a maximum. These solutions can be used to calculate mean values of θ_i over 24 h periods. TM41 provides one such solution for the case of optimum start control (with a defined set point temperature and a required time to meet this). In this work the model has been extended with solutions for fixed time start (where the set point may be reached at different times on different days dependent on outdoor conditions), and night set back (NSB) which is typically used for heat pump control. For NSB overnight cooling is limited to a fixed indoor temperature, typically 3 K below the set point temperature, so that the heating system output can return the building to the required set point in reasonable time. Figure 1 shows an example 24 hour indoor temperature profile generated by the model.

Figure 1.

Modelled 24 h indoor temperature profile for night set back control.

Equation (3) shows the calculation of the mean internal temperature for the overnight cooling and reheating of the building with night set back, where θ_SB is the set back temperature and the time constant τ = C/(HLC x 3600)

{\bar{θ}}_{o v e r n i g h t} = \frac{θ_{o} [(t_{S B} - t_{1}) + (t_{3} - t_{2})] + (θ_{s p} - θ_{o}) [e^{(\frac{t_{3} - t_{2}}{τ})} - e^{(\frac{t_{S B} - t_{1}}{τ})}] + \frac{τ Q_{i n}}{H L C} [1 + \frac{t_{3} - t_{2}}{τ} - e^{\frac{t_{3} - t_{2}}{τ}}] + [(t_{2} - t_{1}) - (t_{S B} - t_{1})] θ_{S B}}{(t_{2} - t_{1}) + (t_{3} - t_{2})}

(3)

Equation (4) shows the 24 h mean internal temperature of the building.

{\bar{θ}}_{i} = \frac{{\bar{θ}}_{o v e r n i g h t} [(t_{3} - t_{2}) + (t_{2} - t_{1})] + θ_{s p} [24 - (t_{3} - t_{2}) - (t_{2} - t_{1})]}{24}

(4)

We then define a base temperature as

θ_{b} = {\bar{θ}}_{i} - \frac{{\bar{Q}}_{G}}{H L C}

(5)

and daily degree-days as

D_{d} = \frac{\sum_{h = 1}^{24} (θ_{b} - θ_{o, h})}{24}

(6)

Where θ_o,h is the hourly outdoor temperature. The degree-day approach allows the model to incorporate the quasi-dynamics of the indoor temperature variation over 24 h (which depends on outdoor temperature, HLC and thermal capacitance), and the casual gains to the space. It therefore handles intermittent operation better than other steady-state models. Note that equation (6) can use hourly, daily mean or monthly mean outdoor temperatures to generate degree-days, but hourly values are the most precise. Hourly temperatures for a specific location can be obtained, usually at a cost, and require some computational effort. To keep the model simple, we adopt the approach of generating monthly degree-days (D_m) using

{\bar{θ}}_{i}

and

{\bar{Q}}_{G}

for an average monthly outdoor temperature,

{\bar{θ}}_{o, m}

using Hitchin’s formula¹⁷

D_{m} = \frac{N (θ_{b} - {\bar{θ}}_{o, m})}{1 - e^{- k (θ_{b} - {\bar{θ}}_{o, m})}}

(7)

Where N is the number of days in the month, and k is a constant, which is typically 0.71 for the UK.

Monthly degree-days are calculated for each month of the year, with monthly delivered space heating energy requirement (SH), in kWh, calculated by

S H = 24 H L C D_{m}

(8)

The heat pump electrical energy is calculated from

E = \frac{S H}{C O P}

(9)

Where COP is the heat pump coefficient of performance. Day et al¹⁶ have shown a relationship between measured COP and daily degree-days, which can be closely represented by a 4^th order polynomial of the form

C O P = α^{‴} {D_{d}}^{4} + α^{″} {D_{d}}^{3} + α^{'} {D_{d}}^{2} + α D_{d} + β

(10)

In theory the coefficients for equation (10) need to be established for different heat pump types and specific applications. However, not enough data is yet available to provide definitive values, other than for a few measured examples. An example can be found in,¹⁶ but more data is required to provide a comprehensive set of characteristic COP profiles. Other studies have identified the importance of hourly COP modelling,^18,19 but these apply second order polynomial models and are used in whole sector modelling studies. Other work uses outdoor temperature related COP characteristics²⁰ or presents COP models related to input and output heat pump temperatures.²¹ However, these present only partial solutions to presenting in situ weather related COPs, and it is not possible to synthesise comprehensive curves for COP variation across the heating season range of temperatures. Heat pumps often run with weather compensation, so both source and demand side temperatures vary in ways that are not easily modelled under laboratory conditions. These studies also do not take account of parasitic losses in the heat pump (e.g. compressor oil warming) which become a higher proportion of input energy in warmer, part-load conditions. Abid et al²¹ do present some monthly estimates of COP variation for different control regimes, but these do not show the degradation of COP seen in our study for warmer conditions.

The Electrification of Heat Demonstration Project²² run by the Energy Systems Catapult has generated a large data set of retrofit heat pump performance, with some analysis of Seasonal Performance Factors (effectively COP over 12 months), and cold day COPs.^23,24 While this does not present COP results in a way compatible with equation (10), the data sets are publicly available, and present an opportunity for some more detailed analysis of weather related COP that could be adapted for improved energy performance predictions.

The use of Seasonal Performance Factors (or seasonal COPs) means that cold weather months may over-estimate COP when demand is highest, leading to under-estimating monthly and annual energy demand. Heat pump energy consumption is far more sensitive to inter-seasonal variation than are gas boilers, which have a smaller variation range throughout the heating season.

Methodology

The purpose of this paper is to show how variation of building thermal properties, heat gains and control regimes impact on predicted energy demand. This helps to understand how errors in input assumptions (e.g. from the quality of a site survey or applying the wrong set point temperature) will affect the model outputs, which in turn informs the need for appropriate levels of data collection to provide confidence for a retrofit design solution.

Six of the monitored properties were selected for the quantity and quality of the data sets (see Table 2), and the prior work that had been carried out to validate the HLC measurements. More details were also available for these properties in terms of built form, occupancy and system and controls, and operational patterns.

Table 2.

Description of selected properties.

Property	Type	Heating system
IC260	4 bedroom detached house in a rural location	11 kW heat pump installed in March 2023
ICB2	5 bedroom modern townhouse in urban location, with triple glazing and solar PV	11 kW heat pump
ICB6	6 bedroom 1990s detached cottage in rural location	14 kW heat pump
IK04	4 bed 1970s detached house with modern extension in suburban location	24 kW gas fired combi-boiler
IL26	3 bedroom 1950s Semi detached house with 2000s extension and electric underfloor heating	24 kW gas fired combi-boiler
ICH63	3 bedroom detached house in a rural location	24 kW gas fired combi-boiler

For each property the data included the following:

− Hourly or daily heating energy consumption, disaggregated by space heating and DHW

− Heat metered delivered energy for space heating and DHW

− Daily general electricity consumption (excluding heat pump or electric vehicle)

− Hourly temperatures for four or five rooms

− Hourly (in some cases 5 min) local weather data that includes dry and wet bulb temperatures, global horizontal solar irradiation, wind speed and direction, and precipitation.

These data have been aggregated into suitable daily, weekly and monthly summations or averages. The HLCs have been measured and determined as described in¹⁶ and Morris et al,²⁵ with estimates made of the effective thermal capacitance and solar gain factors. These, along with the occupancy patterns and known control regimes, and measured system efficiencies, were put into the predictor model described above to give monthly and annual predictions of space heating delivered and consumed energy, carbon emissions and cost. These predicted values were compared with the actual energy demand and usage for each property. This provided an initial indication of the model’s ability to predict actual energy consumption.

The key input variables of system efficiency, set point temperature, heat loss coefficient, thermal capacity and casual heat gains were systematically varied to see how these changed the model outputs relative to the initial predictions. The aim of this was to establish which variables constitute the greatest uncertainty to a prediction, and therefore require the greatest attention at the data collection and preparation stage.

Morris et al²⁵ also showed that the HLC is not constant over the heating season, and that there is a co-dependency of this with thermal capacitance. This is not entirely predictable, but the upshot is that the use of a single HLC throughout the heating season may not give consistent results. A second set of tests were devised to look at what variation in month-by-month HLCs would be required to give results with no resulting error. Once this revised baseline was established it was used to see the impact of using different control strategies on energy consumption: fixed time start (FST), night set back (NSB) or optimum start (OS).

Table 3 shows the tests that were carried out to observe the changes in predicted energy consumed.

Table 3.

Set of tests carried out for each property.

Test 1	Change of set point temperature ±2°C (10 steps)
Test 2	Change of average annual boiler efficiency ±1%, ±2%, ±5%, ±5%, ±10%
Test 3	Change of fixed annual HLC (U’) value ±1%, ±2%, ±5%, ±5%, ±10%
Test 4	Change of thermal capacity ±1%, ±2%, ±5%, ±5%, ±10%
Test 5	Change of monthly casual gains ±1%, ±2%, ±5%, ±5%, ±10%
Test 6	Monthly change of HLC (U’) value to achieve close to unity slope value
Test 7	Using optimum monthly modifiers, compare fixed time start off time: 7.5 h, 8 h, 9 h
Test 8	Using optimum monthly modifiers, change mode of operation to optimum start
Test 9	Night set back mode, compare set back temperatures: 17°C, 16°C and 15°C

Results

The tests were conducted on six properties where there was good understanding of the data quality and knowledge of the condition of the buildings. A heat pump example is shown below, followed by the overall results of the six properties.

Heat pump example IC260

IC260 is a four bedroom detached house in a rural location with an 11 kW heat pump installed in March 2023. Table 4 shows the key input parameters for the initial model run. The initial HLC was determined from the best regression fits of heat energy delivered against degree-days. If the calculated fabric heat loss coefficient is to be taken as reliable, the measured HLC can be used to infer the average rate of air infiltration losses. The active thermal capacitance is also inferred from the measured data as the value that provides the best match to the cool down and heat up rates. This is generally found to be significantly higher than the theoretically calculated value. The solar gain factor was also determined from the regression analysis as the value that returned the most reliable regressions statistics (see¹⁶ for a discussion on this).

Table 4.

Key input variables for property IC260.

Input parameter	Value used	Notes
HLC (fabric only)kW/K	0.203	Calculated from fabric survey
Air changes per hour	1.33	Balancing parameter to achieve measured HLC
HLC kW/K	0.371	Regression analysis (Surveyed value = 0.359 kW/K)
C value kJ/K	46,127	Applying Newtonian cooling algorithm to free cooling events. Theoretical value = 13.98 MJ/K, with factor of 3.3 applied for the model
Solar gain factor	0.5	Assumed value based on regression analysis (balancing parameter). Glazed area 22 m²
Set point temperatures:
Day	20°C	From known set point values, corroborated via measured values
Overnight	17°C
COP	Bespoke COP curve	Best fit of measured values: $C O P = - 0.0002 D_{d}^{4} + 0.0113 D_{d}^{3} - 0.1979 D_{d}^{2} + 1.2963 D_{d} + 1.6846$

Figure 2 shows the initial plot of modelled versus actual monthly space heating consumed energy for property IC260. It shows the regression line values and R², together with the 95% confidence intervals. For a perfect match we want to see a slope of 1 and an intercept of 0, with an R² of 1. The R² is high, showing a consistent performance across all months, but with a general under-prediction for all months with the exception of September and November. This suggests a systematic error in the model to generally under-predict.

Figure 2.

Modelled versus actual space heating energy consumption base case for IC260, showing regression best fit and 95% confidence intervals.

Table 5 shows the values and % differences. The annual under-prediction is in the region of 6%, which might be considered to be acceptable (around £5 per month on a £1000 per annum energy bill) for practical applications. However, we want to know how reliable the input assumptions are, and also try to explain the systematic error. As this error varies from month to month it might be reasonable to assume that variation in HLC (as discussed above) might be the dominant cause. September and June show the highest discrepancies, most likely because the number of heating days is partial for these months, even though the degree-day approach should account for heating period durations. These low heating months are typically where the degree-day method is shown to be less accurate.

Table 5.

Monthly actual and predicted energy consumption and % difference for IC260.

	Sep	Oct	Nov	Dec	Jan	Feb	Mar	Apr	May	Jun	Total
Actual SH energy consumption kWh	65	333	709	835	1146	662	608	541	275	59.4	5233
Predicted SH energy consumption kWh	108	323	737	769	1030	590	564	462	267	76	4924
% difference	65.6%	−2.9%	3.9%	−7.9%	−10.2%	−10.8%	−7.3%	−14.7%	−3.1%	27.9%	−5.9%

Systematic variations were carried out for the key variables as shown in Table 3 (tests 1 to 5), and the changes in predicted energy consumption recorded. The results are shown in Figure 3 for four variables, while the results for set point variation are shown in Figure 4 (the scale is not directly comparable).

Figure 3.

Percentage variation in predicted space heating energy consumption for variations in thermal capacity, HLC, casual gains and COP for IC260

Figure 4.

Percentage change in predicted space heating energy consumption for changes in set point temperature for IC260.

From Figure 3 it is clear that the most influential parameter is the HLC with a ∼±12% change in energy for a ±10% change in input. This is followed by COP at −8.6%/+10.5%. These two together could lead to very significant additive errors. Note that in this example the COP used was a set of measured values and is probably as good as is likely to be achieved². It is interesting the note that casual gains have a smaller effect again, while thermal capacity variation has negligible impact. This indicates the importance of using measured HLC values where possible, particularly as the air infiltration component cannot be accurately guessed at by a site survey alone. The fabric component is likely to be more robust but will still contain uncertainties if the actual construction is not known – e.g. what cavity fill might exist or the nature of the floor construction.

By far the largest influence on modelled energy use is the assumed set point temperature, although the effect cannot be directly compared against the other parameters. It is important to note that predictions made for a given set point can be severely undermined by operation at a different temperature (a user behaviour factor).

The results of test six are shown in Figures 5 and 6 and Table 6. For each month a multiplier was applied to the HLC to modify the value in such a way as to offset the model error to give an (almost) ideal relationship (Figure 5). The modifiers are presented in Table 6 and shown graphically in Figure 6. This indicates the extent to which the HLC might theoretically be varying throughout the heating season. Reasons for such a variation are unclear, but this was routinely seen across all analysed properties. Possible explanations include: changes in prevailing wind direction and strength (wind has been seen as a factor in changing HLCs¹⁶)³; changes in outdoor temperature affecting the thermal dynamics and interactions with the thermal mass of the building. Both of these are considered plausible, and there may be a combination of these, together with other factors such as user control adjustments.

Figure 5.

Modelled versus actual monthly space heating energy consumption with monthly HLCs modified to remove offset for IC260.

Figure 6.

Variations in monthly HLC values to eliminate model offset from actual for IC260.

Table 6.

Monthly actual and predicted energy consumption and % difference with monthly HLC modifiers for IC260.

	Sep	Oct	Nov	Dec	Jan	Feb	Mar	Apr	May	Jun	Total
HLC modifier	0.75	1.02	0.97	1.08	1.10	1.10	1.06	1.12	1.02	0.90
Effective HLC kW/K	0.278	0.379	0.360	0.401	0.408	0.408	0.393	0.416	0.379	0.334	0.376 ave
Actual SH energy consumption kWh	65	333	709	835	1146	662	608	541	275	59.4	5233
Predicted SH energy consumption kWh	65	332	711	838	1137	663	609	543	275	60	5232
% Difference	0.6%	−0.4%	0.3%	0.4%	−0.8%	0.2%	0.1%	0.3%	0.0%	1.2%	−0.02%

The average of the modified monthly HLC values is 0.376 kW/k, which is very close to the determined average of 0.371 kW/K. The results suggest that the underestimation of the mid-winter HLC affects the overall annual result more than the shoulder month (autumn and spring) overestimation. September and June are only partially heated months, although the degree-day model should intrinsically account for this as, in theory, those days without heating will also have zero degree-days (the casual gains providing any required space warming). However, the model may require a reduced number of heating days to better reflect actual energy use in these months.

Using the optimised predictor model the effect of changing control strategy was investigated for FST, NSB and OS. Figure 7 shows the results. Generally speaking the lower the average overnight temperature, the lower the energy consumption. The results suggest that under NSB the overnight temperature rarely drops below 17°C, so lower set back temperatures have little effect on saving, and act much the same as a 7 h FST operation. It is to be expected that longer off times (8 h and 9 h) lead to higher savings, but this will be at the expense of maintaining internal temperatures. The OS operation consumes more energy. This is because the system switches on at a time and temperature required to reach the set point at a prescribed time in the morning, giving a shorter off time and longer preheat period. Maintaining strict conditions therefore comes at added cost. This also indicates that under NSB for a heat pump, while internal temperatures stay relatively high over 24 h, they might not reach set point conditions until later in the morning, and the time at which this is achieved is dependent upon the prevailing outdoor temperature.

Figure 7.

Percentage change in predicted energy consumption for different control modes for IC260.

Summary of sensitivity analysis for six properties

Six properties were analysed using the predictor model for sensitivity to input variations. Three with gas boilers and three with heat pumps. Tables 7–9 show the modelled accuracies and sensitivity ranges for the five variables, with broad agreement in the overall trends and impacts. Only one property, IC63 showed an overestimation, while the others returned underestimations⁴. These were within the range ±8%. The sensitivity analyses show the same hierarchy of influence for all properties, with the set point temperature being the most critical, followed by HLC and COP/efficiency. The magnitude of the impacts is also very similar. This provides pointers for how modelling predicting residential energy use, cost and carbon should be conducted. This will be discussed in the following section.

Table 7.

Overall baseline model accuracy of predicted v actual energy consumption for six properties.

Property	Set point	% difference in modelled consumed energy with actual
IK04	19.0	−1.45%
IL26	18.7	−5.75%
IC260	20.0	−5.90%
ICB2	20.0	−1.04%
ICB6	20.0	−8.00%
ICH63	19.0	7.99%

Table 8.

Summary of modelled sensitivity of energy consumption (output) to four key input variables for six properties.

Property	Heating type	Efficiency/COP variation		HLC variation		Thermal capacity variation		Casual gain variation
Property	Heating type	−10%	+10%	−10%	+10%	−10%	+10%	−10%	+10%
IC260	Heat pump	10.5%	−8.6%	−11.9%	11.8%	−0.57%	0.57%	3.5%	−3.4%
ICB2	Heat pump	10.1%	−10.1%	−18.5%	13.7%	−0.54%	0.52%	6.1%	−6.6%
ICB6	Heat pump	10.2%	−8.4%	−12.8%	12.9%	−0.29%	0.30%	4.2%	−4.0%
IK04	Gas boiler	14.0%	−11.4%	−14.5%	14.4%	−0.5%	0.4%	2.5%	−2.5%
IL26	Gas boiler	10.5%	−8.6%	−13.5%	13.5%	−0.5%	0.4%	4.6%	−4.5%
ICH63	Gas boiler	12.0%	−9.8%	−13.1%	12.9%	−1.2%	1.0%	3.4%	−3.4%

Table 9.

Modelled variation in energy consumption per degree change of set point temperature.

Property	Set point temperature, °C	Variance in energy consumption per degree change in set point
IC260	20.0	11.6%
ICB2	20.0	11.6%
ICB6	20.0	10.1%
IK04	19.0	10.8%
IL26	18.7	13.3%
ICH63	19.0	11.8%

Discussion

Two significant issues arise from this work

• The key functionality that predictor models need to have to capture different system characteristics

• The quality of data collection and selection of input values to the model

The insights that such models can provide about the difference between heat pump and gas boiler design and operation will depend on both of the above.

Model functionality

Traditional sizing methods for heat generators, particularly at the residential level, have been based on steady state heat loss analysis at worst case conditions, and applying some oversize margin to allow for an expedient heat up time following an overnight shut down, and also to cover for uncertainties in the calculations. The oversize margin has not been particularly scientific and is often simply related to the nearest suitable output capacity in the manufacturer’s catalogue. In domestic properties this is limited further by the size range of combi-boilers, which are often dictated by DHW requirements, and leads to large spave heating oversize margins.

Heat pump operational efficiency is far more sensitive to levels of oversizing, with issues such as cycling rates, available emitter output capacities and system flow rates (due to lower operating temperatures) typically driving design selection towards minimum oversizing. The lack of ability to provide rapid heat-up times from cold start in turn drives control strategies that favour longer running times to keep indoor house temperatures higher over diurnal periods. This inevitably means that for a given house level of insulation and air tightness a heat pump will deliver more heat into the building than a gas boiler. This is reflected by the mean internal temperature of the building over time. An intermittently operated gas boiler will deliver lower mean internal temperatures than a more continuous heat pump.

This issue of intermittent versus more continuous running is central to the issue of running cost differentials between systems and is closely tied to the relative efficiency ratios of the two systems. In very crude terms the ratio of SCOP/η should be greater than the fuel cost ratio of (unit electricity price)/(unit gas price), although this does not fully take into account the additional running time (and warmer house conditions) for the heat pump.

Energy predictor models must therefore be able to capture both the quasi-dynamics of the system operation with respect to whole house temperatures over 24 h, and the variations in heat pump COP relative to changes in outdoor weather conditions. In addition, the models must be able to differentiate between fixed time start, night set back, and optimum start operations, which most steady state models cannot intrinsically do.

The above requirements add complexity and can lead to a propensity to use dynamic simulation software. But this is complex and requires large amounts of data input and model setup time, which is not cost effective for the residential application. In addition, dynamic simulation software does not lend itself readily to rapid sensitivity analysis because there are many input variables and the process becomes unwieldy. The models also do not explicitly return information such as effective thermal mass, nor do the control strategy algorithms use the same complexity as real life self-learning models (in the case of optimum start).

The premise of this work is therefore that a predictor model should be relatively simple to manipulate, with a reduced number of input variables that can be easily varied to assess the relative importance of these (as demonstrated in this paper). The model must also be sophisticated enough to allow for internal and external temperature variations, and more refined COP characteristics. The model presented here has attempted to combine all of these attributes in an attempt to assess relative costs and carbon emissions of running a heat pump versus a gas boiler. This will also be able to assess the impact of heat pump size selection.

Data collection and model inputs

This study has benefited from high quality and quantity data sets from a number of properties in order to measure HLCs, temperatures and energy consumption in order to validate the outputs of a specially developed predictor model for heat pump operation. Most real life situations will have neither the time nor the budget to conduct this level of detail when selecting heat pumps for residential retrofits. However, this paper has shown that it is crucial that the input data reflects as closely as possible the true operating parameters of the property under examination.

We have seen that there is a hierarchy of importance when deciding how much weight should be attached to the accuracy of an input variable. In order of importance these are: set point temperature, HLC, system efficiency (or COP), causal heat gains, thermal capacitance.

Set point is relatively straightforward to consider as there are a limited range of options, which are dictated by end user preference. However, it is important to convey to a user just how much difference a change in set point temperature can make to energy consumption. The values presented in this paper are in general agreement with accepted guidance in the UK (∼10%/K, see, for example,²⁶).

HLC is much more involved. It comprises two components: fabric and infiltration/ventilation heat losses. Fabric losses are well understood, and the methods for calculating these are proven. Infiltration losses can be much more of a guess and are in any case more variable depending on building orientation, leakiness and wind conditions. This study has looked at both calculated HLCs from property surveys and used an HLC measurement methodology based on regression analysis of actual space heating energy consumption and degree-days. Most of the property measurements returned HLC values lower (and sometimes very considerably lower) than the survey calculation. One gas boiler example studied here indicated approximately 30% overestimation of energy consumption if the surveyed HLC was used. This would not be an acceptable margin of error, and in any case could lead to oversizing of the system. This suggests all properties that are candidates for retrofit should conduct HLC measurements, and Day et al¹⁶ suggest a methodology for doing this. Other methods with similar approaches have been proposed,²⁷ but these do not use a degree-day methods.

Measuring the HLC has two downsides. Firstly, it takes time (a heating season is ideal), and secondly it requires significant data collection effort. This project used measured internal temperatures and disaggregated energy data (including heat metered space heating delivered). This is probably not realistic for many commercial operations, but if historic energy data is available, even at the monthly level, it is still possible to use the degree-day regression methods to make assessments of actual HLC. This can at least serve to show how these compare to the survey results but requires assumptions around system efficiency.

This paper has also shown a need for fully characterised seasonal heat pump COP curves (see equation (10)). These are not yet comprehensively available, but some attempt to use weather adjusted COP for different weather conditions is necessary, given the sensitivity of the model outputs to this variable. This is less important for gas boilers.

Casual gains are a significant, if second order, factor. Mean monthly solar insolation can be used in conjunction with solar gain factors, and coupled with other gains such as general electricity use to give general average gains in order to successfully run the model. Thermal capacity has less importance still, and an assumed value is probably acceptable. Erring on the high side will not have significant impacts.

In all cases it is strongly recommended that a sensitivity analysis conducted along the lines presented in this paper in order to understand the level of confidence in the results. Where the uncertainty range is high this provides a flag for which areas need more focus in terms of data collection, data quality, and processing.

Conclusion

This paper has presented a model for predicting the energy consumption of heat pumps in residential buildings. This builds on prior work where the energy performance of a number of houses has been characterised in detail, enabling the model to be validated by using well-defined input values, and comparing this to actual energy consumption. For the six properties shown here the energy estimations were within ±8% of actual. This is considered to be a good result in terms of the reliability of the predictor model used here. The model is not intended to replace existing industry standard models, but does present a case for simplified models (with reduced data input requirements) that can explicitly account for system control dynamics. Some further work on the model is necessary to reduce the large relative errors in the shoulder months, and also to explain and eliminate the observed bias in the model for (mostly) under-prediction.

Sensitivity checks revealed a hierarchy of importance in the accuracy of the input values, with set point, HLC and system efficiency being the most important. The project has provided evidence that calculated survey HLCs may differ significantly from the measured apparent HLC. Where a building is not to have significant fabric and other upgrades, the measured value should be used in any predictor estimations. In addition, the best available system efficiency and COP characteristics should be used, particularly in relation to variations in weather. This last point is of considerable importance for heat pumps and it is recommended that significant effort is made to publish a range of installed system COPs for different weather regimes and building typologies. At a minimum this should include typical intra-seasonal variations around the seasonal average COP.

All modelling contains uncertainties, and these limitations should be made as transparent as possible. Where the input values differ greatly from reality there is a potential for large errors in the estimates to arise, and these findings are applicable for all modelling methodologies. Similar sensitivity analysis should be conducted for other commonly used models to establish a general understanding of errors and uncertainties in all energy predictions for heat pumps. Poor modelling forecasts can have the effect of undermining customer confidence, particularly if running costs are higher than expected. Conducting sensitivity analysis on the results is important in order to inform the design, and to indicate to the end user the potential for variation, especially through user behaviours such as varying set points or opening windows and doors.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: DESNZ Heat Pump Ready Stream 2.

ORCID iDs

Tony Day

Issa Chaer

Notes

References

UK Government . Heat and buildings strategy. Department for Business, Energy & Industrial Strategy, 2021. https://www.gov.uk/government/publications/heat-and-buildings-strategy

BRE Group . Standard Assessment Procedure (SAP). https://bregroup.com/expertise/energy/sap/standard-assessment-procedure-2012. 2012.

BRE Group . SAP 10.2. https://bregroup.com/expertise/energy/sap/sap10. 2023.

Department for Energy Security and Net Zero . (2023). General summary of the Home Energy Model core calculation: a technical explanation of the methodology (HEM-TP-1). https://www.gov.uk/government/publications/home-energy-model-technical-documentation

Focus on space House Institute . Passive House Planning Package (PHPP). (n.d.). https://passivehouse.com/04_phpp/04_phpp.htm

Department for Energy Security and Net Zero . Home Energy Model validation: inter-model comparison summary report (HEM-1). https://www.gov.uk/government/publications/home-energy-model-validation-documentation. 2023.

Department for Energy Security and Net Zero . Comparison of HEM to IEA annex 58 measured data: focus on space heating consumption and internal air temperature in two test homes (HEM-2). Developed by Building Research Establishment (BRE) and consortium. 2023. https://www.gov.uk/government/publications/home-energy-model-validation-documentation

Department for Energy Security and Net Zero . Comparison of HEM to IEA annex 58 measured data (HEM-4). 2023. https://www.gov.uk/government/publications/home-energy-model-validation-documentation

Bauwens

Roels

. Co-heating test: a state-of-the-art. Energy Build 2014; 82: 163–172.

10.

Department for Business, Energy & Industrial Strategy . Technical evaluation of SMETER technologies (TEST) project. 2022. https://www.gov.uk/government/publications/smart-meter-enabled-thermal-efficiency-ratings-smeter-technologies-project-technical-evaluation

11.

Department for Energy Security and Net Zero . Demonstration of energy efficiency potential (DEEP), report 2.01: case study methods. 2024. https://www.gov.uk/government/publications/demonstration-of-energy-efficiency-potential-deep

12.

Jack

Loveday

Allinson

, et al. First evidence for the reliability of building co-heating tests. Build Res Inf 2017; 46(4): 383–401.

13.

Alzetto

Farmer

Fitton

, et al. Comparison of whole house heat loss test methods under controlled conditions in six distinct retrofit scenarios. Energy Build 2018; 168: 35–41.

14.

Brown

Chambers

Abate

, et al. SMITE: using smart meters to infer the thermal efficiency of residential homes. In: Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys '20). Virtual Event, Japan, 2020.

15.

Chambers

Oreszczyn

. Deconstruct: a scalable method of as-built heat power loss coefficient inference for UK dwellings using smart meter data. Energy Build 2019; 183: 443–453.

16.

Day

Morris

Chaer

, et al. An energy analysis methodology for residential heat pump retrofits. Build Serv Eng Res Tecnol 2024; 46(2): 177–188.

17.

Chartered Institution of Building Services Engineers . (2006). Technical memorandum 41: degree-days: theory and practice.

18.

Bogdanov

Satymov

Breyer

. Impact of temperature dependent coefficient of performance of heat pumps on heating systems in national and regional energy systems modelling. Appl Energy 2024; 371: 123647.

19.

Ruhnau

Hirth

Praktiknjo

. Time series of heat demand and heat pump efficiency for energy system modeling. Sci Data 2019; 6: 189.

20.

Priarone

Silenzi

Fossa

. Modelling heat pumps with variable EER and COP in EnergyPlus: a case study applied to ground source and heat recovery heat pump systems. Energies 2020; 13(4): 838.

21.

Abid

Hewitt

Huang

M-J

, et al. Domestic retrofit assessment of the heat pump system considering the impact of heat supply temperature and operating mode of control—A case study. Sustainability 2021; 13(19): 10857.

22.

Energy Systems Catapult . Electrification of heat demonstration project. 2024. https://es.catapult.org.uk/project/electrification-of-heat-demonstration/

23.

Energy Systems Catapult . Interim heat pump performance data analysis report. 2023. https://es.catapult.org.uk/wp-content/uploads/2023/03/EoH-Interim-Heat-Pump-Performance-Data-Analysis-Report-1.pdf

24.

Energy Systems Catapult . Interim insights from heat pump performance data. 2023. https://doc.ukdataservice.ac.uk/doc/9049/mrdoc/pdf/9049_eoh_Interim_insights_from_heat_pump_performance_data.pdf

25.