A data-driven multi-criteria decision-making approach for assessing new product conceptual designs

Abstract

The surge in competition among companies to acquire a more significant portion of the market as well as respecting customer preferences in high quality and diverse products result in a reduction of product life cycles. Accordingly, companies are under enormous pressure to introduce new high quality and diverse products on time. Assessing new product designs at the primary phases of new product development (NPD) is a necessary and complex activity that can considerably reduce the time and cost of introducing new products to the market. The current methods of evaluating new product conceptual designs, including employing decision-making methods based on subjective opinions of experts, utilizing simulation packages, and following trial-and-error approaches in prototyping, may be inefficient, very time-consuming, and costly. To overcome this issue, this paper develops a quantitative data-driven Multi-Criteria Decision-Making (MCDM) approach founded on the combination of an Artificial Neural Network (ANN) method and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), to assess the new conceptual designs. So that the ANN method is utilized to predict the performance characteristics of new designs based on the related existed data of similar products, and TOPSIS is employed to score and rank different proposed alternatives designs. Finally, a case study of evaluating new product conceptual designs in an automotive research and development company is considered to demonstrate the performance and applicability of the proposed approach.

Keywords

Artificial neural network multi-criteria decision making new product development design evaluation automotive industry

Introduction and literature review

In an increasingly fierce competitive market, due to globalization, the advent of new technologies, and multiple channels of introducing products to customers, companies are under enormous pressure to at least keep their position in the market. New Product Development (NPD) is a valuable strategy that aids companies in enhancing their core competence and development. As a result, successful implementation of NPD results in substantial benefits for companies, while a poor implementation can put their survival at risk.^1–3 For this reason, companies are seeking ways to reduce the time and cost of introducing a new product to the market with respecting customer preferences.^4–6

Assessing conceptual design alternatives at the primary stages of NPD is very crucial since it can considerably influence the subsequent stages of designing and manufacturing, and eventually impact the quality, cost, and the acceptance of newly introduced products in the market.⁷ Therefore, a careful assessment of conceptual designs at the early stages of NPD leads to a meaningful time and cost savings.⁸

Following conventional methods of evaluating conceptual designs, the evaluation may be inefficient due to the imprecise and indistinctive information regarding a new product design at the early stages of NPD, and subjective and qualitative judgments of the decision-makers.^7,9 Consequently, these results cannot be dependable. Moreover, although employing simulation software before producing prototypes has widely been accepted, when the number of alternatives increases, this approach is also inefficient owing to a large amount of time and computer processing resources needed to perform simulation executions. Under these situations, prediction methods like regression model and data mining techniques can be beneficial if enough information about the technical and performance characteristics of related products exist. So, the relevant data are utilized to predict the performance of the new conceptual designs. As a result, developing decision-making models based on these quantitative attributes results in a more reliable and efficient assessment of new product conceptual designs.

Real decision-making problems involve with multiple criteria in their nature. Aghamohagheghi et al.¹⁰ considered the MCDM and the cost-benefit analysis methods to access sustainable transport projects with the monetary and non-monetary criteria under uncertainty. They used an interval-valued Pythagorean trapezoidal fuzzy set as an attractive uncertain modeling tool for a real-case study. Fattahi et al.¹¹ proposed a new fuzzy MCDM method to access the risks of various failure modes, in which the weights of the failure modes, risk factors, and decision-makers were considered. They proposed the modified fuzzy weighted MULTIMOORA (Multi-Objective Optimization on the basis of Ratio Analysis plus full multiplicative form) and the modified fuzzy Analytic Hierarchy Process (AHP) methods for a steel-making factory as a real-case study. Khalaj et al.¹² proposed a new extension of the cross-entropy measure of belief values and presented a new aspect of belief functions. They used this measure based on the Dempster-Shafer theory between two belief sets in MCDM with belief valued information. Thus, developing Multi-Criteria Decision-Making (MCDM) models for assessing conceptual design alternatives are of importance, as well.

Applying data mining techniques,^13–17 artificial neural network,^18,19 and machine learning^20,21 in different stages of NPD has demonstrated promising results. Nevertheless, assessing conceptual designs on the early stages of NPD using quantitative approaches is at its infancy period. Yang et al.²² devised a belief rule-based model to predict customer preference according to new product attributes. Zhang et al.⁷ provided a quantitative method based on Support Vector Machine (SVM) and decision-making methods for examining design alternatives in the primary stages of NPD, in which a Decision-Making Trial and Evaluation Laboratory (DEMATEL) method is employed for capturing expert judgments about performance characteristic weights, the SVM for prediction of the values of performance characteristics, and finally a VIKOR (Višekriterijumska Optimizacija I Kompromisno Rešenje) method for ranking the alternatives. Also, they validated their proposed model through a real case study on a blow molding machine.

Also, some researchers focused on solely applying decision-making methods to score conceptual alternatives. Ying et al.²³ took into consideration different kinds of information, including linguistic, interval, and crisp, in the process of a concept selection. They presented a Multi-Attribute Decision-Making (MADM) method with hybrid information found on cumulative prospect theory and validated the method on a case for selecting conceptual alternatives of an automatic dishwasher. Hayat et al.²⁴ introduced a hybrid decision model based on soft sets, TOPSIS, and Shannon entropy, in which Shannon entropy is employed for weighting customer preferences and the TOPSIS method for ranking the concept designs.

In contrast to the above studies, which mostly focused on either predicting an evaluation metric or ranking the alternatives based on qualitative data, this study aims to devise a data-driven MCDM method to assist companies in evaluating conceptual designs to reduce the time and cost of NPD and increase the level of its success. To do this, two Artificial Neural Network (ANN) methods, namely Multilayer Perceptron (MLP) and Radial Basis Function (RBF), are employed for predicting unknown performance characteristics based on the existed data of similar products in the market. Since the number of technical attributes of a design may be numerous, a Principal Component Analysis (PCA) method is applied to reduce the dimensions of the problem and determine the most influential attributes to be considered in the prediction algorithms. Afterward, according to these predicted values, a Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method is utilized to score and rank conceptual design alternatives. Then, some of the top-ranked alternatives can be selected for more assessment by a simulation package or prototype production. To the best of the authors’ knowledge, for the first time, this paper implements this approach on a case study in the automotive industry to assess the new conceptual design alternatives.

The rest of this paper is structured as follows. In Section 2, the proposed data-driven MCDM method for assessing conceptual design alternatives is thoroughly elaborated. Then, in Section 3, the case study in which the proposed method is implemented is described. Moreover, in Section 4, the computational results regarding executing all stages of the proposed approach to the case study are expressed in detail. Finally, Section 5 concludes this research and provides some recommendations for future studies.

Proposed method

Problem definition

The evaluation of the new product design involves complex studies, including the prediction of performance characteristics of new alternatives and ranking them. In this study, a new approach based on the combination of data science and MCDM approach is developed to determine some key performance characteristics of automobile alternatives, rank and narrow them, so that instead of performing very time consuming and costly simulations or producing a large number of prototype products, according to all alternatives, a limited number of them will proceed these steps. Therefore, the cost and time needed for introducing a new product to the market will be drastically decreased.

To do this, first, the main attributes and performance characteristics of the final products must be determined based on the opinions of experts in different associate departments of the company. Then, a list of conceptual alternatives is provided by designers, and after primary examinations, some infeasible designs are removed because of some reasons (e.g. the high price of production and the lack of supplier to provide parts). Afterward, according to available data of similar products, either produced by the company or its competitors, the performance characteristics of new alternatives are predicted using ANN methods. Finally, the alternatives will be scored and ranked based on a TOPSIS method, then some of the top-ranked alternatives are considered for simulation and consequently for producing prototype products.

Steps of the proposed approach

In this subsection, the steps of the proposed approach are elaborated.

Step I: Determining alternatives

Firstly, engineers provide several conceptual alternatives regarding the targeted final product. Each alternative must include technical attributes so that assessing their feasibility is possible. Feasible alternatives are those that their products are attainable according to the level of required technology, finance, materials, and other resources.

Step II: Data gathering

To assess each alternative, the technical attributes and performance characteristics of the related or similar existed products in the market are gathered.

Step III: Data cleansing

In this step, missing, noisy, inaccurate, and outlier data are detected, and then the related records will be corrected or removed.

Step IV: Data preparation

After data cleansing, the input and output variables are determined. Due to the high dimensionality of the model and a large number of input variables (i.e. technical attributes), PCA is employed to reduce the dimensions of the model so that most influential input variables are considered for the next steps. Afterward, the data are divided into three sets of training, validation, and test data.

Step V: Selecting the best configuration and method of prediction

In this study, MLP and RBF methods are exploited to predict the value of output variables (i.e. performance characteristics). The procedure for selecting the best configuration of each method and choosing the best method is as follows.

Determining different configurations for RBF and MLP methods by setting their parameters.

Calculating the Mean Squared Error (MSE) measurement associated with each performance characteristic for each configuration of RBF and MLP methods, $MS E_{ij}$ , based on Equation (1), where $i \in {1, \dots, n}$ are the index of configurations, $j \in {1, \dots, m}$ the index of performance characteristics, and $Y_{k}$ and ${\hat{Y}}_{k}$ are the real value and predicted value of a performance characteristic related to test data k.

Calculating the Weighted Sum of Normalized MSE (WSNMSE) for each configuration of RBF and MLP methods based on equations (2) and (3), where $W_{j}$ is the weight of performance characteristic j, and $NMS E_{ij}$ the normalized value of $MS E_{ij}$ .

Selecting the configuration with the minimum value of the WSNMSE, as the best configuration for both RBF and MLP methods.

Setting different architectures based on various percentages of training, validation, and test data.

Selecting the best prediction method according to the value of the WSNMSE for each architecture by using the paired t-test.

\begin{matrix} MS E_{ij} = \frac{1}{n} \sum_{k = 1}^{K} {(Y_{k} - {\hat{Y}}_{k})}^{2} \forall i \in {1, \dots, n}, \\ j \in {1, \dots, m} \end{matrix}

(1)

\begin{matrix} NMS E_{ij} = \frac{MS E_{ij} - min_{i} MS E_{ij}}{max_{i} MS E_{ij} - min_{i} MS E_{ij}} \forall i \in {1, \dots, n}, \\ j \in {1, \dots, m} \end{matrix}

(2)

WSNMS E_{i} = \sum_{j = 1}^{m} W_{j} NMS E_{ij} \forall i \in {1, \dots, n}

(3)

Step VI: Predicting performance characteristics for each alternative

In this step, the values of performance characteristics are predicted by using the best method selected in the previous step.

Step VII: Scoring and ranking the alternatives

In this step, first, the weight of each performance characteristic is specified by the experts’ opinions. Then, the TOPSIS technique is employed to score and rank the alternatives. Afterward, several high-ranked alternatives are reported for simulation and making prototype products. The schematic of the mentioned steps is depicted in Figure 1.

Figure 1.

Steps of the proposed approach.

Principal component analysis (PCA)

Determining impactful attributes is one of the most essential activities in machine learning and pattern recognition science.²⁵ The PCA method is a practical tool for extracting influential attributes and reducing the dimensions of a problem with the least data loss for avoiding dimensional disaster.²⁶ This method is very helpful for problems, in which achieving labeled data is complicated.²⁵ The PCA method is a statistical orthogonal linear transformation that converts a set of attributes and variables, which might be correlated, to a set of non-correlated attributes called principal component.²⁶ The number of principal components is equal or less than the primary attributes. The first principal component represents the highest data variance, the second principal component, the second greatest one, etcetera. To extract impactful attributes, the attributes with the most influence on data variance are chosen.²⁷ Mi et al.²⁵ and McWhirter et al.²⁷ presented comprehensive information about the PCA method.

Artificial neural network (ANN)

The ANN is a renowned machine learning tool, which can learn and detect patterns in data.^28,29 This system includes input, hidden, and output layers, and processing entities are called neurons.³⁰ ANNs provide non-linear equations between input and output variables without the need to know the precise information and formulation about their relationship.³¹

In ANNs, input layer neurons are not operational and only receive data from the outside environment and deliver them to the next layers. The neurons of other layers are operational, and after processing, send output signals to the next layers.³² The neurons are connected through different links, which have different associated weights. In ANNs, in contrary to statistical methods, the weights are modified automatically to reach better performance.³³

Selecting the proper configuration, type of training data, and appropriate learning algorithm have a considerable impact on reducing the complexity of a problem and enhancing the performance of ANNs. The configuration setting can include determining the number of layers and the number of neurons in each layer.³⁴

Training, validation, and test are three key processes of ANNs. In a training process, the relationship between input and output is presented. Also, validation is a critical phase in ANNs used to improve the training quality and prevent overfitting. Then, in the test process, some output variables are predicted, and the performance of the network is assessed.^28,35

So far, several methods of ANNs have been introduced. In this paper, two of the most popular models, namely MLP and RBF, are employed. These methods have different calculation structure but the same applications.³⁶

In the MLP method, there are some layers between the input and output layers, which are called hidden layers. In comparison to single-layer ANNs, by using different modeling, MLP can better demonstrate the relationship between input and output data.³⁵ Figure 2 illustrates the MLP model. For more information on this method, the readers can refer to Sharafi et al.,³⁷ Das et al.³⁸ and Khayat and Afarideh.³⁹

Figure 2.

Basic schematic of an MLP method.

The RBF method was introduced by Broomhead and Lowe,⁴⁰ in which three layers, including input, hidden, and output layers, form this model of ANN. The structure of the RBF method is more straightforward compared to the MLP method, which results in a faster learning process.³⁶ The network of an RBF method is depicted in Figure 3. The hidden layer as network neurons consists of activation functions of RBF, whose role is a non-linear conversion of input vectors at the input layer. The function of expressing nonlinearity in a new space is performed by the output layer.³⁶ The process of learning in the RBF method includes two distinct stages. The first stage involves a self-organized process for adjusting the weight of each neuron and assigning radial basis functions, which is considerably dependent on the input data. In the second stage, the weight of output neurons is set by utilizing a delta rule.⁴¹ For a better understanding of the RBF method, the studies of Vahdani et al.,⁴² Nguyen et al.,⁴³ and Khayat and Afarideh³⁹ are recommended.

Figure 3.

Basic schematic of an RBF method.

TOPSIS

TOPSIS is a well-known MCDM method developed by Hwang and Yoon,⁴⁴ which involves scoring alternatives according to positive and negative ideal solutions. Solutions that are further than a negative ideal solution and closer to a positive ideal solution are better.⁴⁵ The effectiveness of the TOPSIS method is higher when the number of alternatives is more limited.⁴⁶

The steps of the TOPSIS method proceed below.

I. Creating decision matrix $R = {r_{ij}}$ , where $r_{ij}$ is the value of criterion (i.e. performance characteristic) j of alternative i, $i \in {1, \dots, m}, and j \in {1, \dots, n}$ .

II. Normalizing matrix R and achieving normalized matrix $R' = {r'_{ij}}$ according to the order of magnitude and dimension of each criterion.

III. Calculating weighted normalized matrix $U = {u_{ij}}$ according to Relation (4), where $w_{j}$ is the weight of criterion i.

u_{ij} = w_{j} r'_{ij}

(4)

IV. Determining positive and negative ideal solutions based on Relations (5) and (6), respectively, where J and J′ are the set of criteria that must be maximized and minimized, respectively.

A^{+} = {(\max_{i} u_{i j} | j \in J), (\min_{i} u_{i j} | j \in J') | i \in {1, \dots, m}} = {u_{1}^{+}, u_{2}^{+}, \dots, u_{n}^{+}}

(5)

A^{-} = {(\min_{i} u_{i j} | j \in J), (\max_{i} u_{i j} | j \in J') | i \in {1, \dots, m}} = {u_{1}^{-}, u_{2}^{-}, \dots, u_{n}^{-}}

(6)

V. Calculating $d_{i}^{+}$ and $d_{i}^{-}$ distances of each alternative from positive and negative ideal solutions according to Relations (7) and (8).

d_{i}^{+} = \sqrt{\sum_{j = 1}^{n} {(u_{ij} - u_{j}^{+})}^{2}}

(7)

d_{i}^{-} = \sqrt{\sum_{j = 1}^{n} {(u_{ij} - u_{j}^{-})}^{2}}

(8)

VI. Calculating the Relative Closeness Coefficient (RCC) for each criterion according to Relation (9).

RC C_{i} = \frac{d_{i}^{-}}{d_{i}^{+} + d_{i}^{-}}

(9)

After calculating RCC, the alternatives are ranked based on the RCC values. The greater value of RCC means the better corresponding alternative.⁴⁶

Case study and description

The automotive industry is one of the major economic sectors in the world regarding revenue and employment measures. Due to the nature of products, the highly evolving demands of customers, and increasing competition in the global market, this industry is amongst the most complex industries. As a result, automakers seek new ways to enhance their operations and remain profitable.^{47, 48} Therefore, employing a new approach in NPD procedures can bring a meaningful impact on the automakers’ performance.

In this study, an automotive research and development company is considered as a case study that involves the reduction of CO₂ gas emission. During the process of an NPD, primarily, several conceptual designs are proposed by the experts according to the specific performance characteristics that are determined by the automotive manufacturing company as the customer. Then, these conceptual design alternatives must be roughly evaluated and narrowed to a small number of designs. Consequently, the narrowed designs must be further assessed by using simulation, and finally, a limited number of alternative designs are prototyped, and a more comprehensive assessment will be undergone.

A data set of 2200 automobiles are used for training, validating, and testing of the proposed MLP and RBF methods. Several attributes including Wheelbase (Wb), Track Front (TF), Track Rear (TR), Length (L), Width (W), Height (H), Kerb Weight (KW), Bore (B), Stroke (S), Cylinders Arrangement (CA), Number of Cylinders (NC), Cylinders Volume (CV), Valve Train Type (VTT), Number of Valve Per Cylinder (VPC), Compression Ratio (CR), Coolant Liquid (CL), Unitary Capacity (UC), Aspiration (A), Engine Location (EL), Drive (D), Tyre Size (TS), Transmission Type (TT), Top Gear Ratio (TGR), Final Drive Ratio (FDR), Fuel System (FS), and RAC Rating (RR) are considered as input attributes, and CO₂ emission, maximum power, and 0–100 km/h time are regarded as performance characteristics.

Result and discussion

In this section, the steps of implementing the proposed data-driven decision model on evaluating automobile conceptual design alternatives are illustrated. As mentioned earlier, three performance characteristics, namely CO₂ gas emission (p₁), maximum power (p₂), and 0–100 km/h time (p₃) of an automobile, are considered as the output of the prediction models. The proposed approach is implemented through the following four major phases.

Proposing a list of conceptual design alternatives and their corresponding attributes

First, a list of conceptual design alternatives for automobile NPD is provided by the experts and engineers in the company respecting the predefined requirements of the customer. Since some of these alternatives are not feasible due to the lack of required technology, obstacles to supply required materials and parts, and the high cost of production, they are omitted from the primary list. Then, 14 remained alternatives are considered for more assessments in the next phases to predict their p₁, p₂, and p₃ attributes, and rank them.

Determining influential attributes using the PCA method

Because the number of technical attributes of the problem is enormous, considering all of these attributes leads to a reduction in accuracy and performance of the model, and an increase in its computational time as well. As mentioned earlier, the PCA method is one of the most practical and useful methods of determining influential attributes and reducing the dimensions of a problem. The influential attributes of the problem, according to the PCA method, are listed in Table 1. The attributes that their impact on the variation of data are greater than 0.1% are considered as influential attributes. As it is clear, a cylinder volume is the most influential attribute based on Table 1. Accordingly, the 14 alternatives and their corresponding attributes are reported in Table 2, in which the quantitative attributes are normalized).

Table 1.

Result of the PCA method.

Attributes	Value of the PCAmethod (%)
Number of cylinders	0.12905
Stroke	0.15297
Bore	0.42411
RAC rating	0.81224
Track front	3.26226
Track rear	3.75338
Width	4.92133
Wheelbase	9.28909
Kerb weight	22.91965
Length	23.14911
Cylinders volume	93.65280

Table 2.

Proposed design alternatives with their corresponding normalized attributes.

ID	Wb	TF	TR	L	W	KW	B	S	NC	CV	RR
A1	0.576	0.643	0.643	0.608	0.506	0.458	0.448	0.857	4	0.226	0.155
A2	0.576	0.643	0.643	0.608	0.506	0.445	0.448	0.857	4	0.226	0.155
A3	0.576	0.643	0.643	0.608	0.506	0.455	0.448	0.857	4	0.226	0.155
A4	0.691	0.845	0.796	0.736	0.684	0.494	0.490	0.462	6	0.332	0.330
A5	0.691	0.845	0.796	0.736	0.684	0.550	0.448	0.676	6	0.365	0.311
A6	0.539	0.651	0.659	0.691	0.652	0.432	0.392	0.774	4	0.197	0.140
A7	0.691	0.845	0.796	0.736	0.684	0.512	0.448	0.676	6	0.365	0.311
A8	0.539	0.651	0.659	0.691	0.652	0.393	0.392	0.774	4	0.197	0.140
A9	0.691	0.845	0.796	0.736	0.684	0.529	0.490	0.462	6	0.332	0.330
A10	0.495	0.576	0.575	0.605	0.520	0.581	0.490	0.710	8	0.566	0.492
A11	0.495	0.576	0.575	0.605	0.520	0.601	0.490	0.710	8	0.566	0.492
A12	0.577	0.685	0.666	0.729	0.612	0.517	0.476	0.643	6	0.369	0.322
A13	0.577	0.685	0.666	0.729	0.612	0.399	0.476	0.645	4	0.202	0.162
A14	0.577	0.685	0.666	0.729	0.612	0.499	0.476	0.643	4	0.201	0.162

Note: Wb = wheelbase; TF = track front, TR = track rear; L = length; W = width; KW = Kerb weight; B = bore; S = stroke; NC = number of cylinders; CV = cylinders volume; RR = RAC rating.

Predicting the performance characteristics by employing MLP and RBF methods

In this paper, two methods of the ANN, namely MLP and RBF, are applied for prediction. Firstly, MLP and RBF methods are executed under different configurations based on a trial-and-error approach, and the corresponding MSE for MLP and RBF methods are shown in Tables 3 and 4, respectively. To choose the best configuration for each method, the value of MSE for each performance characteristic is normalized, then the WSNMSE is calculated as a measurement to compare the configurations. The weights for p₁, p₂, and p₃ are considered 0.4, 0.3, and 0.3, respectively, based on the opinion of the experts regarding the nature of the case study. According to the results, the selected configuration for the MLP method is in row 3 of Table 3, and for the RBF method in row 2 of Table 4.

Table 3.

Different configurations for the MLP method.

Row	No. of hiddenlayers	No. of neurons		Transferfunction	Learningrule	MSE
Row	No. of hiddenlayers	Layer 1	Layer 2	Transferfunction	Learningrule	CO₂emission	Maximum power	0–100 km/h time
1	1	3	–	Sig	LM	0.007990	0.018377	0.006586
2	1	6	–	Sig	LM	0.006895	0.015726	0.006088
3	1	9	–	Sig	LM	0.006384	0.014985	0.005929
4	2	3	3	Sig	LM	0.008292	0.019908	0.007142
5	2	3	6	Sig	LM	0.011260	0.016690	0.005980
6	2	6	3	Sig	LM	0.007167	0.015700	0.006040
7	2	6	6	Sig	LM	0.006988	0.016389	0.006190
8	2	3	9	Sig	LM	0.007730	0.016930	0.006201
9	2	9	3	Sig	LM	0.006745	0.017272	0.006439
10	2	9	9	Sig	LM	0.006974	0.015385	0.006615
11	2	6	9	Sig	LM	0.007067	0.017664	0.007390
12	2	9	6	Sig	LM	0.007263	0.019939	0.010753

Table 4.

Different configurations for the RBF method.

Row	Number of the cluster centers	MSE
Row	Number of the cluster centers	CO2 emission	Maximum power	0–100 km/h time
1	5	0.011478	0.015397	0.022727
2	10	0.010074	0.015213	0.021315
3	15	0.010587	0.016342	0.023540
4	20	0.011192	0.015872	0.022467
5	25	0.010800	0.015977	0.022682
6	30	0.010761	0.015562	0.022324
7	35	0.011046	0.016642	0.023496
8	40	0.011142	0.015732	0.021847
9	45	0.011610	0.015791	0.022096
10	55	0.011594	0.016121	0.022692

After selecting the best configuration for each method, one of these methods should be selected for predicting the performance characteristics of the alternatives. For this purpose, each method is executed based on different percentages of training, validation, and test data, and the corresponding normalized MSE for each architecture is listed in Table 5. Over-fitting (overtraining) is one of the serious errors that may occur while using ANN methods. It means that if an ANN method is employed in a wrong way it may tend to adapt to any data, even noisy data. To prevent this error, in this paper, the percentage for validation data is set to 15%⁴⁹ for all architectures. Moreover, the percentages for training and testing data are set to the range of 40%–84% and 1%–45%, respectively. It should be noted that since an adequate number of records are needed for training the ANN algorithm, 40% is considered as the lowest percentage for the training set.

Table 5.

Normalized MSE values for different percentages of training, validation and test data.

Architecture No.	% oftraining	% ofvalidation	% of test	Normalized MSE for theMLP method			Normalized MSE for theRBF method
Architecture No.	% oftraining	% ofvalidation	% of test	CO₂emission	Maximumpower	0–100 km/htime	CO₂ emission	Maximumpower	0–100km/h time
1	84	15	1	0.3512	0.4211	0.1327	0.3657	0.4178	0.1539
2	82	15	3	0.2945	0.1053	0.1571	0.9253	1.0000	0.8940
3	80	15	5	0.5655	0.2866	0.1283	0.8568	0.6922	0.6233
4	78	15	7	0.3654	0.0520	0.1398	0.8961	0.8746	1.0000
5	76	15	9	1.0000	0.0000	0.0000	0.7735	0.6912	0.5848
6	74	15	11	0.7333	0.0643	0.0880	0.7922	0.6601	0.4438
7	72	15	13	0.6640	0.0533	0.1188	0.8154	0.7108	0.7314
8	70	15	15	0.6475	0.2088	0.2735	0.9389	0.7052	0.7505
9	68	15	17	0.5729	0.0902	0.1387	0.9086	0.6438	0.5786
10	65	15	20	0.3953	0.2143	0.2754	0.9212	0.6857	0.5443
11	63	15	22	0.5794	0.1552	0.1732	1.0000	0.7294	0.6002
12	60	15	25	0.0000	0.0681	0.1361	0.7421	0.6278	0.0000
13	58	15	27	0.4406	0.1551	0.2492	0.8198	0.6163	0.4785
14	56	15	29	0.2931	0.1104	0.1512	0.8711	0.6673	0.6136
15	55	15	30	0.4380	0.0835	0.1367	0.0000	0.6148	0.3364
16	53	15	32	0.3357	0.1946	0.2508	0.7524	0.6928	0.4588
17	50	15	35	0.3251	0.1509	0.2220	0.8566	0.0000	0.3641
18	48	15	37	0.3895	0.1240	0.1916	0.8995	0.7264	0.6782
19	45	15	40	0.3052	0.1233	0.1687	0.7806	0.6191	0.3990
20	40	15	45	0.5750	1.0000	1.0000	0.7829	0.6623	0.4165

For the sake of comparing the performance of MLP and RBF methods, and select the best method for prediction, a paired t-test is applied to WSNMSE of each method according to the different architectures of Table 5. This test seeks to find whether any meaningful statistical difference exists between MLP and RBF methods. The results of the paired t-test are reported in Table 6. Since the p-value is less than 0.05, the performance of the proposed method is different base on the WSNMSE measurement. Consequently, the method with the minimum average of the WSNMSE enjoys a better performance. According to Table 6, it is evident that MLP outperforms RBF.

Table 6.

Result of the paired t-test method.

	Paired differences
	Mean	Std.deviation	Std. errormean	95% confidence interval ofthe difference		t	d.f.	p_value(2-tailed)
				Lower	Upper
WSNMSE_MLP-WSNMSE_RBF	−0.3670	0.2211	0.0494	−0.4704	−0.2635	−7.42	19	0.000

Next, the MLP method is employed to predict p₁, p₂, and p₃ for each of the proposed alternatives, as shown in Table 7.

Table 7.

Prediction values of the performance characteristics and TOPSIS results for scoring and ranking the alternatives.

Type of criteria		Criteria			$RC C_{i}$	Rank
		CO₂ emission	Maximum power	0–100 km/h time
		Min	Max	Min
Alternative	Weight	$w_{1} = 0.4$	$w_{2} = 0.3$	$w_{3} = 0.3$
A1		152.99129	163.52620	8.236	0.620595313	6
A2		154.03643	161.44128	8.278	0.615427382	7
A3		152.54433	163.26824	8.231	0.620951804	5
A4		193.29165	257.67265	6.595	0.72024453	3
A5		180.30213	249.08092	6.757	0.743574332	1
A6		149.74895	157.29102	9.805	0.591393322	10
A7		141.36994	207.67795	6.593	0.72918276	2
A8		144.14920	156.16864	9.732	0.598977976	9
A9		204.91603	243.99416	6.744	0.657424837	4
A10		333.42200	317.24090	6.916	0.40430833	14
A11		306.70013	306.00052	6.975	0.431571936	13
A12		202.02830	216.10185	7.576	0.606907544	8
A13		201.52514	150.63793	9.897	0.476892289	12
A14		175.49060	152.12466	10.076	0.536198767	11

Scoring and ranking the alternatives by employing the TOPSIS method

When performance characteristics are predicted by the MLP method, the TOPSIS method is employed to score and rank the proposed alternatives. The results of the TOPSIS method are reported in Table 7. Regarding this table, alternative A5 is ranked first, followed by A7, A4, A9, and A3. These top-ranked alternatives are considered for further assessments and undergoing the next steps of NPD.

Conclusion and future research directions

This study dealt with the assessment of the conceptual designs of a new product. Since the current methods of evaluation are either very time and cost consuming or inefficient because of the subjective opinion of experts, developing a quantitative and reliable method to reduce the time and cost of the evaluation is of much importance. Therefore, this paper provided a data-driven MCDM method based on the combination of MLP and RBF methods with the TOPSIS method, in which MLP and RBF methods are responsible for the prediction of unknown performance characteristics of new design alternatives based on the existed information about similar products in the market, and TOPSIS for scoring and ranking the alternatives. Because the number of technical attributes may be significant, the PCA method is applied to specify the most impactful attributes to be considered in the prediction algorithms. A case study of 14 conceptual design alternatives provided by an automotive research and development company were considered to implement the proposed method of assessment. Different configurations and architectures were explored to determine the best method of prediction and their corresponding parameters. According to the results, MLP was selected to predict the performance characteristics of the design alternatives, and then the TOPSIS method scored and ranked the alternatives based on the predicted results. The final results revealed the satisfying applicability and performance of the proposed approach.

Applying data-driven methods in NPD procedures is at its novice stages; therefore, this area is a hot and promising topic for future research both in academic and industrial communities. Incorporating the prediction of some criteria like customer satisfaction, market demand, environmental, economic, and social effects of new product design in the proposed method can be a promising direction for future research. Moreover, applying other prediction and MCDM methods based on the nature of a case and the amount of existed data, including machine learning, deep learning, support vector machine, data envelopment method, is strongly recommended.

Footnotes

Acknowledgements

The authors would like to thank the Editor-in-Chief and anonymous referees for their helpful comments, which greatly improved the presentation of this paper. Also, the support of the Iranian Operations Research Society is highly acknowledged by the last author, as the board member of this society.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Seyedhossein Iranmanesh

Reza Tavakkoli-Moghaddam

References

Chan

. A dynamic decision support system to predict the value of customer for new product development. Decis Support Syst 2011; 52: 178–188.

Song

Ming

. Integrating Kano model and grey–Markov chain to predict customer requirement states. Proc IMechE, Part B: J Engineering Manufacture 2013; 227: 1232–1244.

Huang

Y-S

Liu

L-C

J-W

. Decisions on new product development under uncertainties. Int J Syst Sci 2015; 46: 1010–1019.

Gao

. A process model for concurrent design in manufacturing enterprise information systems. Enterp Inf Syst 2008; 2: 33–46.

K-T

Shen

C-Y

. Combining tolerance design and monitoring process capability in a design-manufacturing integration procedure. Proc IMechE, Part B: J Engineering Manufacture 2009; 223: 1389–1394.

Ren

Yan

, et al. Conceptual design method driven by product genes. Proc IMechE, Part B: J Engineering Manufacture 2019; 2019: 0954405419876195.

Zhang

Z-J

Gong

Jin

, et al. A quantitative approach to design alternative evaluation based on data-driven performance prediction. Adv Eng Inf 2017; 32: 52–65.

Nikander

Liikkanen

Laakso

. The preference effect in design concept evaluation. Des Stud 2014; 35: 473–499.

C-C

Wang

Chao

K-M

. A fuzzy group-preferences analysis method for new-product development. Expert Syst Appl 2006; 31: 826–834.

10.

Aghamohagheghi

Hashemi

Tavakkoli-Moghaddam

. An advanced decision support framework to assess sustainable transport projects using a new uncertainty modeling tool: Interval-valued Pythagorean trapezoidal fuzzy numbers. Iran J Fuzzy Syst 2020; 2020: 53–73.

11.

Fattahi

Tavakkoli-Moghaddam

Khalilzadeh

, et al. A novel FMEA model based on fuzzy multiple criteria decision making methods for risk assessment. J Enterp Inf Manag. Epub ahead of print 2020. DOI: 10.1108/JEIM-09-2019-0282.

12.

Khalaj

Tavakkoli-Moghaddam

Khalaj

, et al. New definition of the cross entropy based on the Dempster-Shafer theory and its application in a decision-making process. Commun Stat Theory Methods 2020; 49(4): 909–923.

13.

Jiao

Zhang

Helander

. A Kansei mining system for affective design. Expert Syst Appl 2006; 30: 658–673.

14.

Bae

Kim

. Product development with data mining techniques: A case on design of digital camera. Expert Syst Appl 2011; 38: 9274–9280.

15.

Yang

J-B

Wang

Y-M

D-L

, et al. Belief rule-based methodology for mapping consumer preferences and setting product targets. Expert Syst Appl 2012; 39: 4749–4759.

16.

Lee

Song

Park

. Design of convergent product concepts based on functionality: An association rule mining and decision tree approach. Expert Syst Appl 2012; 39: 9534–9542.

17.

Song

Ming

Han

. Prioritising technical attributes in QFD under vague environment: a rough-grey relational analysis approach. Int J Prod Res 2014; 52: 5528–5545.

18.

Roy

Mahapatra

Rani

, et al. Robust feedforward and recurrent neural network based dynamic weighted combination models for software reliability prediction. Appl Soft Comput 2014; 22: 629–637.

19.

Göçken

Özçalıcı

Boru

, et al. Integrating metaheuristics and artificial neural networks for improved stock price prediction. Expert Syst Appl 2016; 44: 320–331.

20.

Yang

C-C

Shieh

M-D

. A support vector regression based prediction model of affective responses for product form design. Comput Ind Eng 2010; 59: 682–689.

21.

Wang

K-C

. A hybrid Kansei engineering design expert system based on grey system theory and support vector regression. Expert Syst Appl 2011; 38: 8738–8750.

22.

Yang

Chen

Y-W

, et al. A belief rule based expert system for predicting consumer preference in new product development. Knowl Based Syst 2016; 94: 105–113.

23.

Ying

C-S

Y-L

Chin

K-S

, et al. A new product development concept selection approach based on cumulative prospect theory and hybrid-information MADM. Comput Ind Eng 2018; 122: 251–261.

24.

Hayat

Ali

Karaaslan

, et al. Design concept evaluation using soft sets based on acceptable and satisfactory levels: an integrated TOPSIS and Shannon entropy. Soft Comput 2019; 2019: 1–35.

25.

J-X

Zhang

Y-N

Lai

, et al. Principal component analysis based on nuclear norm minimization. Neural Netw 2019; 18: 1–16.

26.

Hotelling

. Analysis of a complex of statistical variables into principal components. J Educ Psychol 1933; 24: 417.

27.

McWhirter

Hussain

Al-Jumeily

, et al. Classifying periodic astrophysical phenomena from non-survey optimized variable-cadence observational data. Expert Syst Appl 2019; 131: 94–115.

28.

Amalnick

Habibifar

Hamid

, et al. An intelligent algorithm for final product demand forecasting in pharmaceutical units. Int J Syst Assur Eng Manage 2019; 2019: 1–13.

29.

Yazdanparast

Hamid

Azadeh

, et al. An intelligent algorithm for optimization of resource allocation problem by considering human error in an emergency department. J Ind Syst Eng 2018; 11: 287–309.

30.

Boateng

Pillay

Davis

. Predicting the level of safety performance using an artificial neural network. In: International conference on human systems engineering and design: future trends and applications. New York, NY: Springer, 2018, pp.705–710.

31.

Fausett

. Fundamentals of neural networks: architectures, algorithms, and applications. Upper Saddle River, NJ: Prentice-Hall, Inc., 1994.

32.

Behnood

Golafshani

. Predicting the compressive strength of silica fume concrete using hybrid artificial neural network with multi-objective grey wolves. J Cleaner Prod 2018; 202: 54–64.

33.

Acheampong

Boateng

. Modelling carbon emission intensity: application of artificial neural network. J Cleaner Prod 2019; 225: 833–856.

34.

Abidoye

Mahdi

Idris

, et al. ANN-derived equation and ITS application in the prediction of dielectric properties of pure and impure CO₂. J Cleaner Prod 2018; 175: 123–132.

35.

Raza

Khosravi

. A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings. Renewable Sustainable Energy Rev 2015; 50: 1352–1372.

36.

Kardani

Baghban

Sasanipour

, et al. Group contribution methods for estimating CO₂ absorption capacities of imidazolium and ammonium-based polyionic liquids. J Cleaner Prod 2018; 203: 601–618.

37.

Sharafi

Iranmanesh

Amalnick

, et al. Financial management of public private partnership projects using artificial intelligence and fuzzy model. Int J Energy Stat 2016; 4: 1650007.

38.

Das

Matolak

Das

. Spectrum occupancy prediction based on functional link artificial neural network (FLANN) in ISM band. Neural Comput Appl 2018; 29: 1363–1376.

39.

Khayat

Afarideh

. Performance evaluation of multilayer perceptron, radial basis function, fuzzy inference system, and an adaptively tuned fuzzy wavelet neural network in parameter prediction of multiphase flow measurement instrumentation. Expert Syst 2019; 36: e12386.

40.

Broomhead

Lowe

. Radial basis functions, multi-variable functional interpolation and adaptive networks. Malvern: Royal Signals and Radar Establishment Malvern, 1988.

41.

Yan

Tang

, et al. Drag coefficient prediction for non-spherical particles in dense gas–solid two-phase flow using artificial neural network. Powder Technol 2019; 354: 115–124.

42.

Vahdani

Iranmanesh

Mousavi

, et al. A locally linear neuro-fuzzy model for supplier selection in cosmetics industry. Appl Math Modell 2012; 36: 4714–4727.

43.

Nguyen

T-T

Mia

Dang

X-P

, et al. Green machining for the dry milling process of stainless steel 304. Proc IMechE, Part B: J Engineering Manufacture 2019; 2019: 0954405419888126.

44.

Hwang

C-L

Yoon

. Multiple attribute decision making: methods and applications a state-of-the-art survey (Vol. 186). Berlin, Germany: Springer Science & Business Media, 2012.

45.

Lozano

Calzada-Infante

. Efficiency ranking using dominance network and multiobjective optimization indexes. Expert Syst Appl 2019; 126: 83–91.

46.

Chen

. Effects of normalization on the entropy-based TOPSIS method. Expert Syst Appl 2019; 136: 33–41.

47.

Oliveira

Nunes

Afonso

. New product development in the context of industry 4.0: Insights from the automotive components industry. In: International joint conference on industrial engineering and operations management. Cham, Switzerland: Springer, 2018, pp.83–94.

48.

Relich

. A declarative approach to new product development in the automotive industry. In: Environmental issues in automotive industry. Cham, Switzerland: Springer, 2014, pp.23–45.

49.

Hagan

Demuth

Beale

. Neural network design. Boston, MA: PWS Publishing Co., 1997.