Validation of a multi-physics simulation platform for engine emissions modelling

Abstract

This paper introduces a comprehensive and systematic Design of Experiments based methodology deployed in conjunction with a multi-physics engine air-path and combustion co-simulation, leading to the development of a global transient simulation capability for engine out NOx emissions. The proposed multi-physics engine simulation framework couples a real-time one-dimensional air flow model with a Probability Density Function based Stochastic Reactor Model that accounts for detailed in-cylinder combustion chemistry to predict combustion emissions. The integration challenge stemming from the different computation complexities and time scales required to ensure adequate fidelity levels across multi-physics simulations was addressed through a comprehensive Design of Experiments methodology to develop a reduction of the slower Stochastic Reactor Model simulation to enable a transient simulation focussed on NOx emissions. The Design of Experiments methodology, based on Optimal Latin Hypercube design experiments, was deployed on the multi-physics engine co-simulation platform and systematically validated against both steady state and transient light-duty Diesel engine test data. The surrogate selection process included the evaluation of a range of metamodels, with Kriging metamodels selected based on both the statistical performance criteria and consideration of physical phenomena trends. The transient validation was carried out on a simulated New European Drive Cycle against the experimental data available, showing good capability to capture transient NOx emission behaviour in terms of trends and values. The significance of the results is that it proves the transient and drive cycle capability of the multi-physics simulation platform, suggesting a promising potential applicability for early powertrain development work focussed on drive cycle emissions.

Keywords

Engine modelling emission modelling SRM metamodel surrogate model

Introduction

Model-based engineering methods play a key role in the early stages of the powertrain development process, reducing costs and supporting faster introduction of advanced technologies. Addressing the increasing complexity of powertrains in an efficient and effective manner requires extensive use of virtual methods together with experimental data, as highlighted by the recently published Digitalisation Roadmap on behalf of UK Automotive Council.¹

CAE (computed-aided engineering) has been applied successfully to evaluate the performance of engine air path and aftertreatment, combustion or cooling.^2–5 However, the trade-off between fidelity and speed is still a challenge in CAE simulation, and this inhibits the more extensive use of CAE methods in the early stages of the product development process. In particular, multi-dimensional Computational Fluid Dynamic (CFD) was employed to model with adequate fidelity the physical processes across the engine air path and combustion systems. However, CFD approaches require significant amount of computational power and time for the simulations to converge. This makes CFD modelling being prohibitively expensive for the virtual prototyping tasks focussed on real time simulation of engine performance and emissions across different driving cycles. A common strategy to overcome this challenge is to substitute the computationally expensive simulations with metamodels or surrogate models,⁴ to capture complex behaviour with sufficient fidelity, and are much faster to evaluate. Metamodeling is commonly underpinned by response surface modelling techniques,⁶ which were initially introduced to enhance the efficiency of model development in conjunction with expensive physical experiments.⁷ Metamodeling techniques have proven to be an effective and efficient approach from both theoretical and practical perspective.^8–10 While design of experiments (DoE) have been successfully used in conjunction with CFD models to generate the required dataset for building sufficiently accurate metamodels,¹¹ the computational expense often limits the applicability to the full operational space.

Thermodynamic models, such as stochastic reactor engine model (SRM), are able to drastically reduce the engine cycle simulation times compared to multi-dimensional CFD modelling. PDF-based Stochastic Reactor Models are zero-dimensional and use probability density transport equation to capture the stochastic behaviour of the combustion process. In this context, zero-dimensionality implies that the variables are independent of the space, although statistical homogeneity is assumed, that is, the PDF transport equation remains the same across the space/volume. The in-cylinder variability is encapsulated by so-called ‘Stochastic particles’ which are used to represent statistically the information entropy relating to the temperature and mass fractions of the chemical species involved in the combustion process.

The interest towards PDF approaches from both industrial and academic research has increased recently due to relatively low requirements for time and computational expenses while maintaining adequate level of accuracy. Substantial results have been reported on applications of the PDF-SRM approach for both Diesel^12–18 and gasoline direct injection engines,^18–20 as well as for Homogeneous Charge Compression Ignition engines.^21–27 In particular, for Compression Ignition engines, the PDF approach was shown adequate fidelity for the in-cylinder pressure traces and heat release rate profiles,^14,15 as well as good performance for predicting NOx emissions.^12,13 However, accurate HC, CO and soot prediction is still challenging.^16,17

While the PDF-based SRM solvers demonstrate a good compromise between accuracy and computational time^28,29 to predict engine out emissions, the computation time is still high, and not compatible with the real time and transient simulations across multiple drive cycles at the early stage development process. This limitation could be overcome by employing metamodeling techniques, to replace the SRM solver with surrogate models of adequate fidelity and accuracy.

The motivation for the research work underpinning this paper stemmed from the industry need for virtual engine simulation capability to evaluate emissions performance across different drive cycles early in the engine application development cycle. This requires a multi-physics simulation with fast (compatible with real-time) running capability for transient engine-out emission prediction. An integrated Multi-Physics Engine Simulation (MPES) platform was developed for the co-simulation of two primary systems models of the compression ignition engine: (i) the air path, as a 1D engine model with real time simulation capability; and (ii) the combustion process, represented by a metamodel of the SRM model of the engine. The key objectives of the research were: firstly, to implement a surrogate modelling methodology for the SRM engine model with validated adequacy and accuracy; and secondly, to evaluate the performance of the MPES co-simulation framework for transient NOx emissions on the NEDC drive cycle. A case study approach, based on a specific light-duty Diesel engine, was employed to validate the modelling methodology and to evaluate the MPES performance.

In relation to the first objective, the lack of previous studies in this specific research field required a comprehensive understanding of key impacting parameters and their relationship to the air path conditions of the 1D model needed to develop the SRM engine metamodel. This required an efficient Design of Experiments (DoE) approach for the acquisition of the data necessary for the metamodel training and validation, as well as the evaluation of suitability of various statistical metamodeling techniques towards a Global Surrogate NOx model. A Global NOx Surrogate model here refers to a model that can predict NOx concentration in the exhaust gas based on the air path conditions predicted by 1D, covering the complete engine operating domain and transient air path conditions.

Within the automotive engine research and testing, the use of DoE strategies for metamodeling is reasonably well established. Examples of applications include various optimisation problems, such as valve timing calibration to favour engine performance,³⁰ wheel bearing design impact on bearing life time study,³¹ optimal water jacket flow distribution study³² and many others. Commonly used DoE approaches can be categorised into single-level and sequential strategies. In single-level DoE strategies the data is collected in one go based on a fixed DoE plan (‘one-shot’), followed by fitting the metamodel as a response surface model. The challenge with this is that when applied to systems with unknown behaviour, there is a risk of either over-sampling or under-sampling, resulting in possibility of waste of time and resources or model inaccuracy respectively.^33,34 As an alternative approach to single level DoE designs, the sequential DoE strategies consist of an initial DoE (usually an optimal space-filling design) that is iteratively augmented with additional test points (selected based on optimality criteria), until the desired quality of the metamodel is achieved.^33–35 The selection of a metamodeling approach is important and often a very challenging task.³⁴ Summary of the most common metamodeling techniques, along with their advantages and disadvantages, is presented in Table 1.

Table 1.

Advantages and disadvantages of various metamodeling techniques.

Modelling technique	Advantages	Disadvantages
Polynomial	- Simple structure, easy to understand and manipulate³⁶ - Low computational effort due to linearity in unknown parameters³³ - Lower probability of over-fitting – particularly useful for noisy data³⁷	- Prior knowledge about the response behaviour is required^38,39 - Time-expensive for complex responses as significant increase of model parameters occurs, also leading to requirement for more measurements in order to calculate the parameters³⁷
Kriging (Gaussian)	- Wide range of correlation functions are available, which adds flexibility³⁷ - Perform with high accuracy in global surrogate models^33,40 - Strong interpolation in-between the measured sample points allow lower requirements for number of data points⁴⁰	- ‘Curse of over-fitting’³³– in particular, for noisy data - Can be relatively time-consuming³⁷ - Performance is dependent on the type of the DoE used; less efficient in conjunction with classic DoEs (Full Factorial and Central Composite DoE)⁴¹
Neural Networks (RBF, Hybrid RBF)	- Capable of learning nonlinear complex relationship between input and output parameters⁴² - Good generalisation capability that allows confronting missing or noisy data⁴² - Less experiments needed compared to other modelling techniques due to built-in learning³³	- Over-fitting possibility, especially for noisy data³³

The contribution of this paper is that it introduces a comprehensive and systematic DoE based methodology deployed in conjunction with a multi-physics engine air-path and combustion co-simulation, leading to the development of a global transient simulation capability for engine out NOx emissions, validated against both steady state and transient engine test data. The significance of the results is that it proves the transient and drive cycle capability of the multi-physics simulation platform, with good potential applicability for early engine application development work focussed on drive cycle emissions.

The organisation of the paper is as follows: section 2 presents the overall research methodology, outlining the development and validation of the MPES co-simulation platform, in conjunction with engine case study for this research; section 3 presents the development and validation of the engine airpath model with both steady state and transient conditions; section 4 introduces the combustion SRM model and presents in detail the DoE experiments and metamodel fitting and evaluation methodology leading to the selection of a metamodel for the Global Surrogate NOx model; section 5 and 6 present the results of the validation of the MPES framework under both steady state and transient conditions, with a comprehensive discussion of the performance in the case study. The paper ends up with a discussion of the methodology and results, and a summary of the main conclusions and contributions.

Research methodology

This paper builds on previous research results that proved the capability of SRM to model with acceptable accuracy the in-cylinder conditions (pressure and heat release traces) and the NOx emissions from a light-duty Diesel engine.²⁸ In this previous research the simulation was performed with a stand-alone SRM solver, receiving the required model inputs from both experimental and 1D engine model, and the computational time required to solve one engine cycle was of the order of 2–3 min.

The aim of the current work was to develop a multi-physics engine simulation platform capable of simulate transient behaviour and emissions performance over a drive cycle, addressing the computational limitations of the SRM engine.

Multi Physics Engine Simulation (MPES) Framework

The proposed MPES simulation platform couples a 1D air path engine simulation model with a 0D PDF-based stochastic reactor model to predict engine-out emissions. Both the gas path and the combustion models were developed using commercial software packages – GT-suite and Kinetics, and SRM Engine Suite, respectively. The coupling of the two models means that SRM combustion solver is fed input parameters from the GT simulation, yielding as output predictions for the exhaust gas flow properties including composition, corresponding to the engine operating conditions. Specifically, this work focusses on validating the NOx emissions prediction capability under a transient drive cycle simulation.

The capability and performance of the MPES simulation platform was demonstrated against engine test data; in this research we have considered validation against both steady state and dynamic transient drive cycle data available from a specific engine case study.

The capability of SRM suite to be coupled with engine plant models has been discussed in previous studies.^43–45 However, these studies have focussed on steady-state or slow transient simulation, given that the SRM solver accounts for complex chemistry and charge inhomogeneity which incurs significant computation complexity. To support full transient simulations, the development of a surrogate model for the slower combustion solver is proposed herein. This metamodel predicts the engine-out NOx emissions based on the results of the experiments with the thermodynamic process and combustion reactions predicted by the SRM solver based on the inlet conditions at engine intake valve closing as inputs. The performance of both SRM and the surrogate model developed based on the SRM / MPES experiments also need to be validated against the engine data – both steady state and transient.

The hybrid MPES framework, obtained from the coupling of the GT air path model with the SRM surrogate model, will be capable of predicting NOx outputs in a real time simulation, and the performance of this framework in terms of adequacy of trends and accuracy needs to be evaluated against both steady state and transient engine test data.

Figure 1 summarises graphically the overall methodology for the development and validation of the proposed framework against test data (steady state and transient) available from engine case study.

Figure 1.

Methodology for development and validation of the MPES.

The methodology includes the following steps – also marked in Figure 1:

(1) Development and validation of 1D air path model: 1D air path model supplies the SRM combustion model with required inputs. It is important that the 1D model is capable of representing the performance of its physical counterpart accurately.

(2) Development, calibration, and validation of SRM combustion model: SRM combustion model outputs the NOx prediction, and therefore is required to accurately represent the combustion process.

(3) Development of NOx surrogate model based on SRM: the SRM Surrogate NOx model is built based on the SRM inputs and outputs. To cover the wide range of engine operating envelope, virtual DoE strategy is employed, which forces 1D model to run in various steady state settings, facilitating coverage of transient air path conditions. These conditions are further simulated within SRM environment to produce the NOx response.

(4) Validation of SRM Surrogate Model against test data: the validation process takes place for the steady state and transient drive cycle simulations, comparing the generated NOx results against the reference engine case study data.

Engine case study

The reference engine case study is a 2.0 L light duty Diesel engine for passenger car application. The basic information of the engine is described in Table 2. Both steady-state and transient experimental data were available. The steady-state data, based on 23 engine speed/load points, which cover the engine residence when performing low-to-moderate driving styles, is shown in Figure 2. These reference points were used for correlation of both the SRM-suite combustion model (described in detail in Korsunovs et al.²⁸) and the engine model. Regarding the transient experimental data, a New European Drive Cycle (NEDC) was measured on a transient engine dynamometer test bench in order to validate the MPES. In both tests, the main air path variables were recorded together with the NOx engine-out emissions.

Table 2.

Diesel engine basic information.

Parameter	Value
Bore	83 mm
Stroke	92.35 mm
Connecting rod length	140 mm
Compression ratio	15.5
Emissions standard	Euro 6c
Peak power	130 kW @ 4000 RPM
Peak torque	430 Nm @ 1750–2500 RPM

Figure 2.

Speed-load domain.

Development and validation of engine airpath model

The GT-Power fast running engine model (FRM) used and shown in Figure 3 is based on an accurate representation of the physical engine by means of a 1D solver. For the sake of brevity, the results of the engine model simplification approach are not shown in this paper, as it is outside of the main scope. However, as a summary, the main actions carried out to reduce the simulation time solver are described as follows:

Replacing pipes (1D elements) with volumes (0D elements) calibrated to mimic the pressure drop and thermal behaviour.

Using cylinder-slave option, where only one cylinder is calculated.

The combustion process is modelled imposing the fuel burned derived from the experimentally recorded in-cylinder pressure traces.

In-cylinder heat transfer has been simplified imposing the cylinder wall temperature as function of engine speed and load, based on the results provided by the detailed 1D model.

Figure 3.

GT-Power fast running engine model (FRM).

A virtual Energy Management Strategy has been implemented to control the fuel injected, VGT and EGR valves position to meet the desired brake torque, inlet manifold pressure and EGR rate respectively for every engine operating point. This engine control and the co-simulation has been built in Matlab Simulink.

Figure 4 shows the comparison results and box plot errors for air mass flow, intake manifold pressure, EGR rate, brake torque and intake manifold temperature between experimental and simulated (FRM) across the 23 selected steady-state points depicted in Figure 2.

E r r o r = \frac{S i m u l a t e d v a l u e - E x p e r i m e n t a l v a l u e}{S i m u l a t e d v a l u e} \times 100 %

(1)

Figure 4.

Comparison of modelled and measured variables under steady-state operating conditions.

The simulation of each is point is the result of requesting the objective engine speed and brake torque. The measured-modelled results show a general good agreement of the engine simulation platform. Although the EGR rate trends are predicted correctly, the estimated EGR presents higher errors than the rest of the variables compared. This might have an impact on overall NOx prediction as through analysis in Korsunovs et al.²⁸ it was found that EGR rate certainty is the most important factor that affects NOx prediction accuracy.

Issues have also been identified with the accuracy of intake manifold temperature prediction, it appears that FRM nearly consistently overpredicts the experimentally measured values. This might be related to previously mentioned EGR rate prediction problems, as those might also affect the high/low pressure EGR split and therefore affect the temperature. However, as it was found in Korsunovs et al.,²⁸ inlet temperatures inaccuracies do not lead to high variation in NOx results.

Figure 5 presents the GT-Power model performance throughout the NEDC drive cycle, compared against the experimentally recorded data. Considering the aforementioned steady state testing domain, shown in Figure 2, the main focus of this work is the engine operating envelope, enclosed by the limits of the steady state domain. Therefore, the GT-Power FRM was not calibrated outside of the desired operating region. In addition, to avoid additional error that might be introduced due to operation in uncalibrated regions, the FRM was deliberately forced to continue running at minimum load when engine stop condition was required by NEDC. These regions can be clearly seen in Figure 5 as sectors of significant variables mismatch between simulated and measured data. Otherwise, the FRM manages to replicate the real engine behaviour adequately, with the torque, intake manifold pressure, air mass flow and EGR prediction RMSE being 19.52 Nm, 0.06 bar, 13.56 kg/h and 11% respectively.

Figure 5.

Comparison of modelled and measured variables under transient operating conditions.

Development and validation of combustion SRM surrogate model

SRM model setup

The SRM combustion model has been built in CMCL SRM environment. The entire methodology for model building and calibration of the engine subjected to study is described in detail in Korsunovs et al.²⁸ As a summary, Table 3 contains the CMCL SRM combustion solver sub-models and their settings.

Table 3.

CMCL SRM sub-models settings.

Sub-section	Settings
Initial Mixture at the Inlet	- Oxidiser Composition (air surrogate): 76.7% nitrogen (N₂), 23.3% oxygen (O₂) - EGR composition auto-generated for the initial cycle, then updated from cycle to cycle
Fuel and Injection	- PDF injection model (Fuel evaporation is described by the PDF) - No wall impingement - Elkotb SMD correlation - Imposed Injection Rate Profile - Surrogate Diesel fuel: 80% n-Heptane (C7H16), 20% iso-octane (C₈H₁₈) - EN590 diesel fuel physical properties
Heat Transfer	- Woschni Heat Transfer Model - In-cylinder wall temperatures estimated from GT-Power
Turbulence	- Empirical k-ε Mixing Model

Surrogate SRM NOx model development

MPES fast running simulation capability can be achieved by replacing the time-expensive combustion chemistry solver (SRM) with a surrogate model. This surrogate model will provide statistics-based output, describing the engine-out NOx emissions as a result of the chemical reactions and thermodynamic processes during the combustion predicted by the SRM solver.

To develop a NOx Surrogate statistical model able to work in transient conditions, it is required to cover the NOx response via DoE at both fixed air path conditions of steady state speed-load points and deviations which might occur during transient operation. The engine model will be used to generate the boundary conditions for SRM solver, accounting for a certain deviation with respect to the reference value for all air path variables of each speed-load point. The engine control allows modifying the inlet manifold pressure and air mass flow targets via controlling the VGT position and both low and high pressure EGR valves. This will result in changes in EGR rate, intake manifold pressure and temperature (input parameters to SRM solver). The summary of the DoE-relevant engine model inputs/outputs are:

Inputs: Desired engine speed and brake torque, inlet manifold pressure target and air mass flow target.

Outputs (input to SRM): Actual engine speed and torque, pressure and temperature at intake valve close, EGR mass fraction.

An Optimal Latin Hypercube DOE design, consisting of Model Building (MB) and Model Validation (MV) points has been performed across the 23 steady state speed-load cases shown in Figure 2. The air mass flow DoE limits were selected as ±10% from the nominal value of the specific speed-load case. The inlet manifold pressure DoE limits were selected as ±0.05 bar from the nominal value of a certain speed-load case. For each speed-load case, the total test points for model building and validation was 40 and 10 respectively.

The entire DoE, consisting of 1150 points in total, were simulated first with the engine model and subsequently with the CMCL SRM solver. A set of NOx simulation results is now available for each of the 23 steady state speed-load cases shown in Figure 2. Each of the NOx reading values is linked to the unique set of GT-Power air path state variables that define the boundary conditions of SRM model. Via application of a metamodel to this data, it is expected to be possible to build a Global Surrogate NOx model that can predict the NOx result based on the state of the GT-Power air path model.

A sample of the DoE result can be seen in Figure 6, where both, the MB and the MV points are presented for speed-load Case 18 (indicated with magenta circle in Figure 2). The speed-load cases chosen for the DoE quality analysis based on the orthogonality and space-filling criteria are indicated with red diamonds in Figure 2.

Figure 6.

Case 18 DoE result and orthogonality data.

The orthogonality of the selected five cases DoEs was evaluated by calculating the Pearson correlation coefficient:

r = \frac{\sum_{i = 1}^{n} (X_{i}^{k} - \bar{X^{k}}) (X_{i}^{j} - \bar{X^{j}})}{\sqrt{\sum_{i = 1}^{n} {(X_{i}^{k} - \bar{X^{k}})}^{2}} \sqrt{\sum_{i = 1}^{n} {(X_{i}^{j} - \bar{X^{j}})}^{2}}}

(2)

Where: r – Pearson’s correlation coefficient; n – sample size; k and j – dimension (parameter), $k \neq j$ ; i – sample number; $X$ – sample along a dimension; $X$ – average of all N sample points along a dimension.

Figure 6 shows the Pearson correlation coefficient for speed-load Case 18 and the Least Square Reference Line. The Pearson correlation coefficient of -0.07 indicates a negligible negative correlation between the two factors and the design can be assumed to be quasi-orthogonal. The evaluation of other speed-load cases showed similar orthogonality results.

The space-filling property of the selected five cases DoEs was evaluated by calculating the Euclidian distance between the test points, using equation (3). The bar plot in Figure 7 shows the obtained Euclidian distance results for Case 18. The rest of the five cases show similar results. It can be concluded that the space-filling property of the DoEs is good and none of the points is located too close to each other.

D_{i} = minimum x_{i}, x_{j} \in n \sqrt{\sum_{k = 1}^{k} {| x_{i}^{k} - x_{j}^{k} |}^{2}}

(3)

Figure 7.

Case 18 Euclidean distance data.

A selection of three metamodeling techniques, that fulfil the requirement of this work in terms of applicability and dataset size, has been employed: Kriging (Gaussian), RBF and Hybrid RBF. Each of these techniques has several submodels, resulting in eight metamodels applied to each of the selected speed-load cases. Every metamodel has been built by means of the MATLAB Model-Based Calibration toolbox. The summary information about the metamodels used is presented in Table 4.

Table 4.

Types of metamodels used for global surrogate model building.

Metamodeling technique	Sub-model/correlation function/Kernel function	Label
Kriging (Gaussian)	Matern 3/2	Matern32
	Matern 5/2	Matern52
RBF	Gaussian	RBF-Gaussian
	Multiquadratic	RBF-Multiquadratic
	Thinplate	RBF-Thinplate
Hybrid RBF	Gaussian	HRBF-Gaussian
	Multiquadratic	HRBF-Multiquadratic
	Thinplate	HRBF-Thinplate

The performance of the Global NOx Surrogate models was evaluated based on the PRESS RMSE, RMSE and Validation RMSE:

RMSE = \sqrt{\frac{\sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2}}{n}}

(4)

Where: $n$ – total number of sample points; $y_{i}$ – i-th real (model training) value; $\hat{y_{i}}$ – model-predicted value.

PRESS RMSE = \sqrt{\frac{1}{df - 1} \sum_{i = 1}^{n} {[y (i) - \hat{y_{(i)}} (i)]}^{2}}

(5)

Where: $df$ – degrees of freedom (difference between minimum required number of measurement points and real number of measurement points $n$ ); $y (i)$ – i-th observation excluded from the regression process; $\hat{y_{(i)}} (i)$ – model-predicted i-th observation.

Validation RMSE = \sqrt{\frac{1}{v} \sum_{i = 1}^{v} {(y_{i} - \hat{y_{i}})}^{2}}

(6)

Further evaluation was supported by the NRMSE, CV (RMSE) and number of metamodel parameters data.

CV (RMSE) = \frac{RMSE}{\bar{y}}

(7)

Where: $y$ – mean of the data points.

NRMSE = \frac{RMSE}{y_{\max} - y_{\min}}

(8)

Where: $y_{\max}$ – maximum data point value; $y_{\min}$ – minimum data point value.

The process of Global NOx Surrogate models performance evaluation consists of five steps:

For the whole set of available MB points, build the candidate metamodels proposed and acquire the necessary statistical performance metrics mentioned earlier in this section.

For the built metamodels, shortlist the three best performing candidates based on Press RMSE, RMSE and Validation RMSE statistical performance metrics, using the MV points as validation data set. The underlying concept of this process is to prioritise ensuring that the selected surrogate model does not overfit or underfit the data, rather than selecting the metamodel that shows the best results in a certain statistical performance metric values. The basic measure for the fit, as suggested by MathWorks,⁴⁶ is RMSE. The cross validation is supported by PRESS RMSE and Validation RMSE. The main idea is that the PRESS RMSE should not be more than 20% higher than the RMSE, and the Validation RMSE should not be more than 20% higher than the PRESS RMSE. If none of the candidate metamodel meets the above criteria, the percentage tolerance is increased at 5% at a time, until at least one model for a speed-load case can be selected.

For the built metamodels, substitute the MV validation data set with the experimentally measured NEDC drive cycle NOx data. Evaluate the results. The underlying idea here is that the metamodels providing lowest Validation RMSE values might be able to perform better in the NEDC, If this methodology gives a significant benefit, this would mean that providing a limited amount of test data, it is possible to build reliable metamodel that would possibly work throughout various drive cycles.

Shortlist the three best performing candidate metamodels, based on the Validation RMSE values, in ascending order. If a shortlisted metamodel matches a metamodel selected in Step 2, select next best, so that a total of six metamodels is shortlisted after Step 2 and 3.

For the shortlisted candidate metamodels, evaluate the normalised statistical performance metrics (NRMSE and CV (RMSE)) to understand how the errors compare to the overall NOx response domain. Also, evaluate the ability of candidates to capture physical phenomena correctly by analysing the response trends.

The most important performance metrics of a metamodel is the evaluation of how it actually performs in the simulation application. The validation is supported by steady state and NEDC drive cycle real time simulation application.

The candidate metamodels described in Table 4 were built using all the DoE simulated NOx data obtained through the methodology discussed earlier. Statistical performance metrics for Global NOx Surrogate candidate metamodels are illustrated in Figure 8, followed by numerical data presented in Table 5.

Table 5.

Candidate Global NOx surrogate models statistical performance metrics, based on MB-MV data set; shortlisted candidates: Green – 1st candidate, Yellow – 2nd candidate, Orange – 3rd candidate.

Model:	No of parameters:	PRESS RMSE:	RMSE:	PRESS RMSE > RMSE	Validation RMSE	Validation RMSE > PRESS RMSE
Model:	No of parameters:	[ppmv]	[ppmv]	[%]	[ppmv]	[%]
HRBF-Thinplate	84	8.09	6.488	24.69	25.476	214.91
RBF-Thinplate	68	7.334	6.753	8.60	17.179	134.24
HRBF-Multiquadratic	80	10.205	6.983	46.14	15.293	49.86
RBF-Multiquadratic	103	11.489	7.302	57.34	17.363	51.13
RBF-Gaussian	74	12.118	10.79	12.31	18.302	51.03
Matern52	97.491	14.515	11.025	31.66	16.211	11.68
Matern32	102.41	14.987	11.504	30.28	17.016	13.54
HRBF-Gaussian	31	12.561	11.717	7.20	18.829	49.90

Figure 8.

Candidate Global NOx Surrogate models statistical performance metrics illustration.

The lowest RMSE values are associated with HRBF-Thinplate and RBF-Thinplate metamodels. RBF-Thinplate metamodel also benefits from the lowest PRESS RMSE, and hence it also corresponds to the lowest difference between the PRESS RMSE and RMSE, at PRESS RMSE being only 8.6% higher than RMSE. However, both metamodels have significant issues with predicting validation points, having Validation RMSE significantly higher than PRESS RMSE. The difference is more than 100%, which is an indicator of overfitting.

Despite having highest RMSE and PRESS RMSE values, as well as having the difference between these values of around 30%, both Kriging models, Matern52 and Matern32 show a very balanced correlation between PRESS RMSE and Validation RMSE. The difference between these values being around 16% is an indicator that these models can interpolate well in-between the model building data. The Kriging models also have their Validation RMSE values among the lowest, being outrun only by HRBF-Multiquadratic metamodel.

Therefore, as indicated in Table 5, the three shortlisted models based on the MB-MV statistical performance data, are Matern32, Matern52 and HRBF-Multiquadratic. However, it is important to note that in order for these models to satisfy the criteria outlined in Step 2, the difference tolerance had to be increased to 35%.

Following the procedure outlined in Step 3, the MV validation set for the built models has been changed to experimentally recorded NOx values from the NEDC drive cycle test run. The Validation RMSE values are presented in Table 6.

Table 6.

Candidate Global NOx Surrogate models Validation RMSE based on external validation set (NEDC experimental values); shortlisted candidates: Green – 1st candidate, Yellow – 2nd candidate, Orange – 3rd candidate.

Model:	Validation RMSE, ppmv
RBF-Multiquadratic	43.763
RBF-Thinplate	45.526
RBF-Gaussian	46.277
Matern32	48.685
Matern52	49.03
HRBF-Thinplate	53.296
HRBF-Multiquadratic	55.366
HRBF-Gaussian	56.624

The further three shortlisted candidate metamodels based on the lowest Validation RMSE of the NEDC external validation data set are RBF-Multiquadratic, RBF-Thinplate and RBF-Gaussian. Interestingly, there has been no cross-match with the first three shortlisted candidate metamodels. Therefore, the six candidate metamodels that have been shortlisted for further evaluation will be Matern32, Matern52, HRBF-Multiquadratic, RBF-Multiquadratic, RBF-Thinplate and RBF-Gaussian.

Normalised statistical performance metrics for these six candidate metamodels is illustrated in Figure 9. The NRMSE values are well below 1% for all of the shortlisted metamodels. Achieving such a low value was expected, as the range of the NOx DoE response for the entire engine operating domain for this engine is 1623 ppmv. The CV (RMSE) values are slightly higher, as the average DoE response NOx concentration value for the entire engine operating domain is 431 ppmv; however, all of the candidate metamodels show results deviating by less than 3%. Thus, it can be concluded that all the models perform well relative to the overall NOx domain.

Figure 9.

Normalised statistical performance metrics of the candidate Global NOx surrogate models.

Figure 10(a) and (b) show the predicted versus observed NOx values and studentized residuals respectively for Matern32 metamodel. Most of the residuals, 81.2%, are concentrated within the region of ±1σ (σ– standard deviation), 94% falling in the region of ±2σ, and only 6% of values are outside of the ±2σ region, being still contained within ±3σ. No obvious pattern can be identified in the residuals plot, meaning that the predictions are not biassed. Similar results have been obtained for the rest of the candidate metamodels.

Figure 10.

Matern32 predicted versus observed NOx values (a) and Studentized residuals (b).

In order to avoid unrealistic or non-physical results provided by the metamodels, the global response trends for the shortlisted candidate metamodels are illustrated in Figure 11. These trends have been obtained by fixing the variables at the mean values throughout the domain. Most of metamodels can correctly identify the physical phenomena that have an effect on engine NOx. Matern32, Matern52 and RBF-Thinplate show very similar trends, followed by RBF-Multiquadratic. RBF-Gaussian, despite showing the correct trend from the effect of inlet manifold pressure, appears to compromise on it and is biassed towards more aggressive EGR effect. HRBF-Multiquadratic metamodel fails to capture the inlet pressure effect completely and implements even more aggressive EGR effect than RBF-Gaussian.

Figure 11.

Candidate Global NOx Surrogate models response trends for complete engine operating domain.

The selection process results presented above demonstrate that selecting the best candidate metamodel based purely on statistical performance metrics is inappropriate. It is important to consider the ability of metamodel to capture the physical phenomena behind the process it represents. Therefore, there needs to be a balance between the numerical selection approach and engineering judgement.

MPES validation under steady-state operating conditions

The evaluation of the candidate metamodels performance across the steady state speed-load cases was done based on NOx concentration prediction error against the experimental values:

Error = \frac{NO x_{Exp} - NO x_{Mod}}{NO x_{Exp}}

(9)

Where: $NO x_{Exp}$ is measured NOx [ppmv] and $NO x_{Mod}$ is meta-model predicted NOx [ppmv].

Further analysis of the prediction error was supported by the error standard deviation study, calculated as per equation (10):

St . D . = \sqrt{\frac{\sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{2}}{n - 1}}

(10)

Where: $x_{i}$ is error for $i^{th}$ speed-load case; $\bar{x}$ is mean error; $n$ is total number of speed-load cases.

The six candidate metamodels selected were employed in this validation process: Matern32, Matern52, HRBF-Multiquadratic, RBF-Multiquadratic, RBF-Thinplate and RBF-Gaussian. The prediction capability of each metamodel was evaluated across the complete engine operating domain, represented by the 23 speed-load operating points shown before.

Average prediction error and standard deviation for the candidate metamodels and SRM are shown in Figure 12. The data shows a consistent under-prediction of all candidate models with respect to the experimental NOx values following the SRM trend. Matern52 metamodel outperforms other candidate metamodels and demonstrates the lowest NOx concentration prediction error throughout the complete engine operating domain. As can be seen from the plot, the average error of the metamodel is comparatively close to the error demonstrated by SRM.

Figure 12.

Average metamodels and SRM NOx concentration error (a) and Standard deviation of metamodels and SRM NOx concentration prediction (b).

Response surface plots of Matern52 and SRM NOx concentration prediction is illustrated in Figure 13. Overall, the metamodel captures well the NOx concentration trend and the response closely matches the SRM one. It can be seen, nevertheless, how the metamodel slightly over-predicts the results of SRM at low load region and under-predicts it for mid-high load range.

Figure 13.

NOx concentration surface plot comparison between Matern52 and SRM.

Once the metamodels performance have been verified against SRM results (source of data for metamodel building), the simulation accuracy under steady-state conditions is now evaluated with respect to experimental data. Figure 14 shows the NOx mass flow surface plot comparison between the metamodel Matern52 and the experimental results. Overall, the metamodel tends to slightly over-predict the data recorded. This difference does get noticeable at high engine load and speed. The reason of this behaviour comes from the stand-alone SRM correlation used as a source to generate the metamodel, well explained in Korsunovs et al.²⁸ However, the response nearly overlaps in the region of low to mid speeds and load. Considering that most of the NEDC drive cycle operation falls in this region, it can be concluded that Matern52 metamodel predicts the experimental data very well and stands a high chance of providing high fidelity NOx emissions prediction throughout the NEDC drive cycle simulation.

Figure 14.

NOx mass flow emissions surface plot comparison between Matern52 and experiments.

Steady state validation results demonstrate high ability of metamodeling techniques to cover and represent performance of SRM combustion model, providing limited error to the source model results. These results revealed that high accuracy can be achieved at a negligible computational time thanks to the simplicity of the metamodel building.

MPES validation under transient operating conditions

The evaluation of candidate metamodels performance under real time drive cycle simulation was performed based on the comparison of metamodel responses with the experimentally recorded data in terms of both instantaneous and cumulative NOx across the entire cycle.

Noteworthy, at the engine stop conditions, which were outside the engine steady state operating domain, as discussed previously in Section 3, exhaust mass flow was artificially switched off in order to replicate experimentally recorded NOx mass flow.

Figure 15(a) shows the cumulative NOx error results using the GT-Power predicted exhaust mass flow. The experimental cumulative NOx for this engine NEDC drive cycle was 1.8g. All the metamodels over-predict the experimental result, which was expected, considering that the metamodels slightly over-predict the steady state experimental NOx concentration trends as discussed before in Section 5.

Figure 15.

Cumulative NOx results using GT-Power predicted exhaust mass flow (a) and experimental exhaust mass flow (b).

An additional source of error comes from the inaccuracy of the air path model to replicate the actual exhaust mass flow, presenting an average overestimation of 6.5%. In order to isolate this error, an additional test imposing the actual exhaust mass flow was carried out. The results are shown in Figure 15(b), showing evidence that the flow to experimental greatly reduces the error. The Matern52 prediction error (the most accurate metamodel) drops from just below 20% to just over 5%.

Visual representation of a selection of metamodels simulation run results is illustrated in Figure 16, employing the experimental exhaust mass flow in the simulation. Visually, Matern52 adheres closer to the experimental curve; very similar result was achieved by Matern32. The RBF-Gaussian and RBF-Thinplate also show good results with few more problematic regions.

Figure 16.

Instantaneous (a) and cumulative (b) NOx emissions across the NEDC cycle: Metamodels vs experimental.

The simulation carried out revealed that MPES platform is limited timewise only be the performance of the GT-Power model. The metamodel computational time is negligible in comparison to GT-Power model computational time. In this particular case, the GT-Model runs in real time, which allows the MPES to run in real time as well. The NOx prediction present also high levels of accuracy, especially considering the uncertainties in the EGR rate and exhaust mass flow prediction capability of GT-Power model. Imposing experimental exhaust mass flow demonstrates the potential of MPES platform to reach significantly higher accuracy levels through the reduction of uncertainties.

Conclusions

This paper has introduced a comprehensive and systematic DoE based methodology deployed in conjunction with a multi-physics engine air-path and combustion co-simulation, leading to the development of a global transient simulation capability for engine out NOx emissions, validated against both steady state and transient engine test data.

A key conclusion from this work is that the MPES multi-physics engine simulation framework that enables the co-simulation of a 1D air-path engine model and a surrogate of the PDF-based Stochastic Reactor Model that accounts for detailed in-cylinder combustion chemistry provides the transient simulation capability needed for drive-cycle emissions assessment early in the powertrain development cycle with acceptable adequacy and accuracy.

The methodology proposed for the metamodel selection process included both the statistical performance criteria and consideration of physical phenomena trends, with validation against steady state and transient engine test data. The proposed DoE for metamodeling building captures air path variables deviation from the predefined set-up, which is a requirement to enable virtual transient calibration. From the range of metamodeling techniques considered in the study, Kriging surrogates were selected based on their ability and accuracy in predicting NOx trends. This was judged based on the evidence of average error under steady-state conditions comparatively close to that seen on the stand-alone Stochastic Reactor Model combustion simulation, and also ability to identify correctly the physical phenomena that impacts engine-out NOx.

The transient validation exercise revealed that the proposed framework can reproduce in an acceptable manner the transient engine-out NOx, that is a function not only of the engine speed and load, but also different actuators. The comparison with experimental NEDC data shows that this framework can achieve errors in the order of 5% if high fidelity levels of exhaust mass flow prediction are reached.

The significance of the results is that it proves the transient and drive cycle capability of the MPES multi-physics simulation platform, suggesting a promising potential applicability for early powertrain development work focussed on drive cycle emissions.

Footnotes

List of abbreviations

BMEP Brake Mean Effective Pressure

CAD Crankshaft Angle Degrees

EGR Exhaust Gas Recirculation

EMS Engine Management Strategy

PRESS Predicted Residual Error Sum of Squares

RMSE Root Mean Squared Error

SSE Sum of Squared Errors

VGT Variable Geometry Turbine

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors acknowledge funding for the research presented in this article from Jaguar Land Rover under research collaboration with the University of Bradford, project named ‘Multi-Physics Engine Simulation Framework’.

ORCID iDs

Oscar Garcia-Afonso

Felician Campean

References

IDE. Digitalisation roadmap. 2021. https://roadmap.ide.uk/?theme=digital-engineering

Piano

Millo

Di Nunno

Gallone

. Numerical analysis on the potential of different variable valve actuation strategies on a light duty diesel engine for improving exhaust system warm up. SAE technical paper 2017-24-0024, 2017.

Malkhede

Khalane

. Maximizing volumetric efficiency of IC engine through intake manifold tuning. SAE technical paper 2015-01-1738, 2015.

Galindo

Serrano

Piqueras

García-Afonso

. Heat transfer modelling in honeycomb wall-flow diesel particulate filters. Energy 2012; 43(1): 201–213.

Raut

Mallikarjuna

. Effects of direct water injection and injector configurations on performance and emission characteristics of a gasoline direct injection engine: a computational fluid dynamics analysis. Int J Engine Res 2020; 21(8): 1520–1540.

Singh

Campean

Seale

Grove

Cary

. Evaluation of Camshaft control strategies using a multi-evolutionary genetic algorithm. In: Röpke

(ed.) Design of experiments in engine development, Expert Verlag, 2007, pp. 165–183.

Röpke

Baumann

Köhler

B-U

Schaum

Lange

Knaak

. Engine calibration using nonlinear dynamic modeling. In: Alberer

Hjalmarsson

delRe

(eds) Identification for automotive systems. London: Springer, 2012, pp.165–182.

Tizzard

Kleijnen

JPC

. Statistical tools for simulation practitioners. J R Stat Soc Ser A 1988; 151: 374.

E. P. Box

Hunter

. Statistics for experimenters: an introduction to design, data analysis, and model building. New York: John Wiley & Sons, 1978.

10.

Simpson

Poplinski

Koch

Allen

. “Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 2001; 17(2): 129–150.

11.

Kajero

Thorpe

Chen

Wang

Yao

. “Kriging meta-model assisted calibration of computational fluid dynamics models. AIChE J 2016; 62(12): 4308–4320.

12.

Bernard

Scaife

Bhave

Ooi

Dizy

. Application of the SRM engine suite over the entire load-speed operation of a U.S. EPA tier 4 capable IC engine. SAE technical paper 2016-01-0571, 2016.

13.

Bernard

Lebas

Demoulin

F-X

. A 0D phenomenological model using detailed tabulated chemistry methods to predict diesel combustion heat release and pollutant emissions. SAE technical paper 2011-01-0847, 2011.

14.

Mauviot

Albrecht

J. J. Poinsot

. A new 0D approach for diesel combustion modeling coupling probability density function with complex chemistry. SAE technical paper 2006-01–3332, 2006, pp. 776–790.

15.

Bordet

Caillol

Higelin

Talon

. A physical 0D combustion model using tabulated chemistry with presumed probability density function approach for multi-injection diesel engines. SAE Technical Paper 2010-01-1493, 2010.

16.

Pasternak

Mauss

Bensler

. Diesel engine cycle simulation with a reduced set of modeling parameters based on detailed kinetics Michał Pasternak and Fabian Mauss. SAE technical paper 2009-01-0676, 2009.

17.

Franken

Mauss

. Development of methodology for predictive diesel combustion simulation using 0D stochastic reactor model. SAE technical paper 2016-01-0566, 2016.

18.

Smallbone

Bhave

Coble

, et al. Simulating PM emissions and combustion stability in gasoline/diesel fuelled engines. SAE technical paper 2011-01-1184, 2011.

19.

Pasternak

Mauss

Sens

Riess

Benz

Stapf

. Gasoline engine simulations using zero-dimensional spark ignition stochastic reactor model and three-dimensional computational fluid dynamics engine model. Int J Engine Res 2016; 17(1): 76–85.

20.

Vasos

. Multi-physics co-simulation of engine combustion and exhaust aftertreatment system. Bradford, United Kingdom, University of Bradford, 2016.

21.

Mosbach

Kraft

Bhave

Mauss

Mack

Dibble

. Simulating a homogeneous charge compression ignition engine fuelled with a DEE/EtOH blend. SAE technical paper 2006-01-1362, 2006.

22.

Mosbach

Kraft

, et al. Dual injection homogeneous charge compression ignition engine simulation using a stochastic reactor model. Int J Engine Res 2007; 8: 41–50.

23.

Vikhansky

Mosbach

, et al. A computational study of an HCCI engine with direct injection during gas exchange. Combust Flame 2006; 147(1–2): 118–132.

24.

Bhave

Balthasar

Kraft

Mauss

. Analysis of a natural gas fuelled homogeneous charge compression ignition engine with exhaust gas recirculation using a stochastic reactor model. Int J Engine Res 2004; 5: 93–104.

25.

Aldawood

Mosbach

Kraft

. HCCI combustion phasing transient control by hydrogen-rich gas: investigation using a fast detailed-chemistry full-cycle model. SAE technical paper 2009-01-1134, 2009.

26.

Mosbach

Aldawood

Kraft

. Real-time evaluation of a detailed chemistry HCCI engine model using a tabulation technique. Combust Sci Technol 2008; 180(7): 1263–1277.

27.

Mosbach

Celnik

Raj

, et al. Towards a detailed soot model for internal combustion engines. Combust Flame 2009; 156(6): 1156–1165.

28.

Korsunovs

Campean

Pant

Garcia-Afonso

Tunc

. Evaluation of zero-dimensional stochastic reactor modelling for a diesel engine application. Int J Engine Res 2020; 21: 592–609.

29.

Lai

Parry

Mosbach

Bhave

Page

. Evaluating emissions in a modern compression ignition engine using multi-dimensional PDF-based stochastic simulations and statistical surrogate generation. SAE technical paper 2018-01-1739, 2018.

30.

Lach

Kien

Hans

Siebertz

Platzbäcker

. (2005) DoE application within analytical valve train development. In: Röpke

(ed.) Design of Experiments (DOE) in Engine Development. Expert Verlag, pp. 14.

31.

Lee

. Bearing life evaluation for automotive wheel bearings using design of experiments. SAE technical paper 2018-01-1903, 2018.

32.

Panagiotopoulos

Iqbal

Mourelatos

Papadimitriou

. Optimal water jacket flow distribution using a new group-based space-filling design of experiments algorithm. SAE technical paper 2018-01-1017, 2018.

33.

Kianifar

. Application of multidisciplinary design optimisation frameworks for engine mapping and calibration. Bradford, United Kingdom, University of Bradford, 2014.

34.

Kianifar

Campean

Wood

. Application of permutation genetic algorithm for sequential model building–model validation design of experiments. Soft Comput 2016; 20(8): 3023–3044.

35.

Crombecq

DeTommasi

Gorissen

Dhaene

. (2009, December 13-16). A novel sequential design strategy for global surrogate modeling [Conference paper]. Proceedings of the 2009 Winter Simulation Conference (WSC), Austin, TX, USA. IEEE.

36.

Hartmann

Baumann

Nelles

. (2013). Axes-oblique partitioning of local model networks for engine calibration. In Röpke

(ed.) Design of Experiments (DOE) in Engine Development. Expert Verlag, pp. 92–106.

37.

Khan

MAZ

. Transient engine model for calibration using two-stage regression approach. Loughborough, United Kingdom, Loughborough University, 2010.

38.

Myers

Khuri

Carter

Khuri

. Response surface methodology: 1966–1988. Technometrics 1989; 31: 137.

39.

Morris

Mitchell

. Exploratory designs for computational experiments. J Stat Plan Inference 1995; 43: 381–402.

40.

Miller

Grove

Baker

Taylor

Pates

. Application of advanced modelling techniques in the development of a high specific output 3 cylinder gasoline engine. SAE technical paper 2019-01-1136, 2019.

41.

Meckesheimer

Barton

Simpson

Limayem

Yannou

. Metamodeling of combined discrete/continuous responses. AIAA J 2001; 39(10): 1950–1959.

42.

Hagan

Demuth

(1999, June 2–4). Neural networks for control [Conference paper]. Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251), San Diego, CA, USA. IEEE.

43.

Etheridge

Bhave

Smallbone

Coble

Mosbach

Kraft

. Optimisation of injection strategy, combustion characteristics and emissions for IC engines using advanced simulation technologies. SAE technical paper 2011-26-0080, 2011.

44.

Coble

Smallbone

Bhave

, et al. Implementing detailed chemistry and In-cylinder stratification into 0/1-D IC engine cycle simulation tools. SAE technical paper 2011-01-0849, 2011.

45.

Fröjd

Perlman

Tunér

Mauss

. (2011, January 26–27). 1D engine modeling with detailed reaction kinetics [Conference paper]. Swedish-Finnish Flame Days 2011, Piteå, Sweden. IFRF. http://www.ffrc.fi/FlameDays_2011/Session_2_Engines/17frojd.pdf

46.

MathWorks. Guidelines for selecting the best model fit, 2019, https://uk.mathworks.com/help/mbc/model/model-selection-guide.html?searchHighlight=OverfittingandUnderfitting&s_tid=doc_srchtitle (accessed 25 August 2019).