A quality analysis method for three-dimensional model in aircraft structural parts design

Abstract

To analyze the quality of three-dimensional (3D) model for aircraft structural parts designed by model-based definition (MBD) technology, an approach combining analytic hierarchy process (AHP), hesitant fuzzy linguistic term set (HFLTS), and fuzzy synthetic evaluation is proposed. According to all levels of quality standards and part specification-tree elements, a quality assessment index system is constructed from four sub-models of parts 3D model: design model, process model, tooling model, and test model. In addition, the weight of each index is calculated using the AHP. Then the assessment model is established by using a configurable index system model, HFLTS, fuzzy synthetic evaluation, and assigning uniformly and quantitatively the index system through quality grade division rules of indexes and triangular fuzzy numbers. Finally, a case application is used to illustrate the proposed method. The application of this method can make the quality analysis of parts 3D model more effective, accurate, and efficient. This paper can not only help enterprises identify higher-weight and error-prone design factors, but also guide designers in modeling.

Introduction

With the application of model-based definition (MBD) technology in aviation manufacturing enterprises, three-dimensional (3D) model is usually the only data source in the design and manufacturing process. ¹ It affects engineering analysis, virtual assembly, digital prototype, and NC machining.^2,3 Moreover, MBD is also a key technology for implementation of model-based enterprise, ⁴ product lifecycle management, ⁵ and digital twin. ⁶ Therefore, the analysis of product data quality (PDQ) plays a crucial role in avoiding product defects, which is also one of the key problems to be solved in the application of MBD technology. Aircraft structural parts are the main components of aircraft frame and aerodynamic shape, and it is necessary to analyze the quality of 3D model for aircraft structural parts. The aims of quality analysis for parts 3D model are to control PDQ in the product research and development cycle, guide modeling in real time, and improve design efficiency. However, the characteristics of MBD technology and aircraft structural parts complicate the quality analysis of parts 3D model. Hence, how to analyze the quality of parts 3D model effectively and comprehensively becomes more and more important.

By studying a large amount of literature, the research methods of quality analysis for parts 3D model mainly include manual assessment, manufacturability evaluation, and system inspection.^7–9 Manual assessment is the earliest method used. According to the standards of various organizations, the enterprises formulate the own data quality standards, then assess the quality of parts 3D model manually. For example, the national standard GB/T 18784 for CAD/CAM data quality was put forward by China National Institute of Standardization. According to the national standard, Guangxi Yuchai Machinery Co., Ltd. formulated the 3D modeling standard Q/YC 5033, which was used as the basis for manual assessment. However, through the investigation, it is found that some quality problems are easy to be omitted in the process of manual assessment, which leads to the non-standard problems in the published 3D model. In short, the standardization process of design specifications for most enterprises develops slowly, and most of them still stay in the state of standard text.

Manufacturability evaluation, which usually adopts the method of establishing inference engine based on database, has been widely used in process examination.^10,11 Wang et al. ¹² used three-level manufacturability evaluation system to carry out an assessment for gears 3D model. Sato et al. ¹³ integrated fictitious physical models with topology optimization for manufacturability evaluation of molds. Stolt et al. ¹⁴ presented an integrated method for manufacturability assessment of aircraft engine components, which can find the best parameter settings in 3D model. Tang et al. ¹⁵ established a quality controllability evaluation model for composite components. Moreover, Hsiao et al. ¹⁶ used gray number design evaluation to solve forecast product production problem. However, there is no comprehensive assessment index system by this method, the indexes assessed mainly focus on the manufacturability.

The widely used method in enterprises is system inspection. Gupta et al. ¹⁷ developed a computer-aided tool for the quantitative evaluation of prismatic machining parts. Son et al. ¹⁸ developed an automated product data quality verification and management system, which only focused on quality influencing factors in design phase. Huang et al. ¹⁹ proposed an automatic 3D model errors detection approach for aircraft structural parts, and developed a prototype system. However, this method has poor scalability and configurability, which cannot meet the assessment requirements for parts 3D model designed by MBD technology.

These existing methods have realized the assessment of some indexes for parts 3D model from different perspectives. However, there are problems such as few assessment indexes, poor scalability, poor configurability, and inflexible application. Therefore, it is imperative to study a more flexible method for comprehensive quality assessment of parts 3D model. The quality assessment of 3D model for aircraft structural parts is complex considering it involves multi-factor and human decision-making process. In terms of this problem, the commonly used methods include hesitant fuzzy linguistic term set (HFLTS), analytic hierarchy process (AHP), and fuzzy synthetic evaluation. HFLTS is an effective tool in the process of decision making, which has been swiftly advanced on various fronts over the past 5 years.^20–24 Huang et al. ²⁵ used proportional HFLTS to convert requirements into engineering characteristics in the product design. In response to the problem with both qualitative and quantitative criteria in the context of HFLTSs, Liao et al. ²⁶ proposed a new multi-criteria decision making method. Moreover, Xu ²⁷ evaluated the product design quality with dual hesitant fuzzy information. Wang and Li ²⁸ developed a new multiple attribute decision making method based on the interaction operational laws of Pythagorean fuzzy numbers. It can be seen that HFLTS can help to enhance the accuracy of assessment results.

The AHP is a famous method for making decisions, which has been widely used in engineering fields.^29,30 It can help decision makers find the most important factors. Fuzzy synthetic evaluation is an effective assessment method, which is affected by multiple factors. ³¹ The current applications in aerospace are mainly focused on manufacturing resource allocation, ³² assembly safety evaluation, ³³ tolerance allocation, ³⁴ and remanufacturability evaluation. ³⁵ Through these applications, it can be seen that the fuzzy synthetic evaluation has great superiority in the assessment process. In practical application, AHP and fuzzy synthetic evaluation are often used together.^36,37 However, the assessment indexes of 3D model will change for different aircraft structural parts. In terms of this problem, Xu et al. ³⁸ researched a hybrid model based on dynamic fuzzy synthetic evaluation. Li et al. ³⁹ established a dynamic evaluation framework for ambient air pollution monitoring. Wang et al. ⁴⁰ researched a fuzzy synthetic evaluation algorithm with dynamic weight.

Compared with these studies, although many studies have been conducted to assess the quality of parts 3D model, no comprehensive index system and inflexible methods exist for the quality assessment of parts 3D model in aircraft structural parts design. In this study, we analyzed the design factors that affect the quality of 3D model for aircraft structural parts according to all levels of quality standards and part specification-tree elements, and then developed a series of evaluation indexes. By the combination of AHP, HFLTS, and fuzzy synthetic evaluation, a quality assessment of parts 3D model is carried out. As the parts 3D model information cannot be directly used for assessment, it is converted by quality grade division rules of indexes and represented by triangular fuzzy numbers. Moreover, a configurable index system model is proposed to make the assessment process more flexible.

The purpose of this paper is the proposed quality analysis method for 3D model in aircraft structural parts design. Specifically, it can effectively identify the factors influencing the quality of 3D model for aircraft structural parts and then help aviation manufacturing enterprises carry out the quality control in the parts design. Subsequently, the quality assessment index system and the quality grade division rules of indexes can provide modeling reference for designers. Finally, the proposed assessment model ensures the accuracy of the data source in aircraft research and development, and helps enterprises to implement MBD technology efficiently.

Quality assessment index system

The establishment of index system plays an important role in the quality assessment of 3D model for aircraft structural parts. Due to the variety and complexity of aircraft structural parts, enterprises need to pertinently select the indexes according to the assessed parts. In view of this requirement, a general quality assessment index system need be established to meet various types of parts. The factors considered in establishing the assessment index system are as follows:

1. All levels of standards and artificial experience. The former mainly includes international standards, national standards, and enterprise standards. The later mainly refers to the experience of design technicians, process technicians, and manufacturing technicians. Through the investigation of an aircraft manufacturing enterprise, the quality standards of parts 3D model are shown in Table 1. The generality of quality assessment index system can be ensured according to all levels of standards and artificial experience.

2. The part specification-tree elements. In 3D model, the part specification-tree is used to organize and manage all elements. By analyzing the part specification-tree elements, the parts 3D model data is divided into two categories: geometry elements and non-geometry elements. ⁴¹ Geometry elements includes engineering geometry, external references, construction geometry, reference geometry, part body, and axis systems. Non-geometry information includes standard notes, part notes, annotation notes, annotation set, material description, approval status, and publication. The integrity and accuracy of quality assessment index system can be ensured according to the part specification-tree elements.

Table 1.

Quality standards for parts 3D model of an aircraft manufacturing enterprise.

Category	Mechanism	Number	Name
International standard	The American Society of Mechanical Engineers	Y14.41	Digital product definition data practices
	International Organization for Standardization	ISO16792	Technical product documentation, digital product definition data practices
	Boeing	BDS-600	Application specification based on MBD
National standard	China National Institute of Standardization	HB 7756.1	CATIA modeling requirements
		GB/T 18784	Assurance method of CAD/CAM data quality
		GB/T 24734	Digital product definition data practices
Enterprise standard	An aircraft manufacturing enterprise	MBD	General requirements
			Geometry elements requirements
			Non-geometry elements requirements
Artificial experience	Technicians	Experience	Experience of design technicians, process technicians, and manufacturing technicians

The digital manufacturing process based on MBD technology includes parts design, process design, tooling design, and parts test.^42,43 Correspondingly, parts 3D model will derive four sub-models, including design model, process model, tooling model, and test model. The design of each sub-model is reasonable to ensure the parallel control in parts design and manufacturing process. Therefore, the first-level indexes are established based on four sub-models of parts 3D model. Then, according to all levels of standards and artificial experience, the secondary-level indexes are established, as shown in Table 2. In practical application, based on the part specification-tree elements, the tertiary- to N-level indexes should be established.

Table 2.

Quality assessment index system of 3D model for aircraft structural parts.

Target layer	First-level indicators	Second-level indicators	Third- to N-level indicators
Quality of 3D model for aircraft structural parts X	Design model X1	Name and properties X11	The part specification-tree elements
		Technical requirement X12
		FT/A X13
		Coordinate system X14
		Feature information-tree X15
		Structure of part specification-tree X16
		Feature parameters X17
		Lightweight structure X18
	Process model X2	Annotation and properties X21	The part specification-tree elements
		NC program X22
		Processing method X23
		Machining settings X24
		Machining parameters X25
		Machining efficiency X26
	Tooling model X3	Coordinate system X31	The part specification-tree elements
		Fixing-clamping devices X32
		Manufacturing cost X33
	Test model X4	Test items X41	The part specification-tree elements
		Test path X42
		Measuring tools X43

Design model index (X₁)

Design model is the basis of product digital manufacturing process. ⁴⁴ Based on the classification of parts 3D model data, the quality of design model mainly includes geometry standardization and non-geometry standardization. Non-geometry standardization can be divided into name and properties standardization, technical requirements standardization, and functional tolerancing and annotation (FT/A) standardization. Geometry standardization can be divided into coordinate system standardization, feature standardization, structural elements standardization. Thus, the secondary-level indexes of design model are established.

Process model (X₂)

Process model is an integrated three-dimensional digital model with complete process information to guide the parts machining. ⁴⁵ It describes in detail the processing sequence of NC machining, the geometric features of 3D process model, the annotation, and properties of multi-procedures. ⁴⁶ Geometric features data describes the geometry information formed after machining. The annotation data describes the process constraint information necessary for production, such as machining size, tolerance range, and accuracy requirements. Attribute data describes the built-in information, specifically, including raw material specification, analysis data, and processing operation information. Machining data describes processing settings, processing parameters, and processing efficiency.

Tooling model index (X₃)

The tooling model is the basis for manufacturing fixing and clamping devices. ⁴⁷ The design quality of tooling model affects the product machining efficiency, which determines the perfection, applicability, and inheritance of the process equipment. Tooling model data contains all geometric and non-geometric information of the process equipment. Therefore, the quality assessment indexes of tooling model mainly include the unity of coordinate system, characteristics of process equipment, and manufacturing cost. In the tooling design, by analyzing the quality of the tooling model, it is possible to pre-evaluate the subsequent manufacturing and use of the tooling digital prototype, and make design changes in advance.

Test model index (X₄)

The test model provides a carrier for the expression and application of inspection information.^48,49 The test model mainly includes the tolerance information of the part, the geometric feature information of the tolerance, the measuring tool information required by the tolerance, and the correlation information among the tolerance, geometry, and measuring tools. Therefore, based on the inspection information model and inspection relationship model, the test model indexes are divided into test items, test paths, and measuring tools.

Establishment of the quality assessment approach

The AHP and HFLTS-fuzzy synthetic evaluation methods are used to assess the quality of parts 3D model. Firstly, the parts 3D model data is extracted by feature technology. Secondly, a configurable index system model is used to configure objective assessment tree and modify the general assessment index system. Then, AHP is used to determine the weight. Finally, the fuzzy linguistic terms and fuzzy synthetic evaluation are used to carry out an assessment of the quality for parts 3D model. The quality assessment process used in this paper is shown in Figure 1.

Figure 1.

Quality assessment process.

Assessment information extraction

The parts 3D model data is the basis for the accuracy of subsequent quality assessment. In order to provide the correct input information for quality assessment, redundant or incomplete information should be removed to unify the format of extracted assessment information.

3D modeling based on MBD technology is closely related to feature information modeling.^50,51 In order to ensure the standardization and completeness for information transmission of aircraft structural parts 3D model, feature technology is used to reorganize and extract the parts 3D model data. On the one hand, the 3D model data is extracted by different application programming interface and stored in XML format. On the other hand, this paper models the extracted data to unify the digital structure of the parts 3D model data. By reading the XML file, the extracted data is reorganized to obtain the appropriate assessment information. The final assessment information consists of four types of information: overall information, feature description, feature parameters, and index node information.

Objective assessment tree configuration

For different assessment requirements and parts types, if the general assessment index system is directly used for the quality assessment, it will affect the accuracy of the assessment results and reduce the assessment efficiency. Therefore, the structure of index system should be modifiable and extensible, that is, the objective assessment tree should be dynamically configured according to the assessed parts.

In order to avoid the mutual influence among indexes and meet the configuration of the objective assessment tree, the selection node is inserted into the traditional multi-layer assessment index system, then establishing a configurable index system model, as shown in Figure 2. Therefore, the configurable index system model includes index node and selection node. The selection node is used to configure the objective assessment tree and modify the general assessment index system. According to the common relationship among indexes, the selection node is divided into three types, including AND type, XOR type, and OR type.

Figure 2.

Configurable index system model.

AND type: All sub-indexes must be assessed, generally, which mainly are mandatory or general indexes.

XOR type: Sub-indexes are parallel indexes, and only one of them can be assessed, generally, which mainly are same indexes of different types of structures or indexes with different assessment rules of the same feature.

OR type: Sub-indexes belong to multi-choice indexes, and one or more of them can be assessed, generally, which mainly are sub-indexes of the tertiary-level index.

Firstly, GI is defined as a set of all indexes nodes in the general assessment index system. TI is defined as a set of all indexes nodes in the objective assessment tree. Based on the knowledge expression of the assessed information and assessment index, the comparison between the assessed information and the assessment index is transformed into the similarity assessment problem, and then the target assessment index retrieval is realized. ⁵² Finally, the indexes nodes in GI and TI are orderly arranged according to the top-down and breadth-first search method. The configuration process of the objective assessment tree is shown in Figure 3.

• Step 1: According to the top-down, breadth-first search method, GI is obtained. If GI is not empty, going to step 2, otherwise going to step 7.

• Step 2: Traversing all indexes in GI. If the parent node is AND type, going to step 3; if it is XOR type, going to step 4; if it is OR type, going to step 5.

• Step 3: Deleting self-node, going to step 6.

• Step 4: If the index node is not selected, deleting self-node and child nodes; otherwise, deleting all parallel nodes, then deleting self-node and going to step 6.

• Step 5: If the index node is not selected, deleting self-node and child nodes; otherwise, deleting self-node and going to step 6.

• Step 6: Adding the selected index node to the end of TI.

• Step 7: According to the top-down, breadth-first search method, TI is converted to the objective assessment tree.

Figure 3.

Configuration process of the objective assessment tree.

As the selection node is inserted into the index system model, the general assessment index system has the characteristics of tailoring and scalability. With the development of design technology, enterprises can constantly modify the general assessment index system.

Weight determination

The AHP is an effective method to analytically determine the weight of each index in a multi-objective and decision-making problem. ⁵³ Fortunet et al. ⁵⁴ used the AHP method in the design of aircraft structural parts. In this study, we employ this method to get the weights of each level of indexes by expert’s opinion as making pairwise comparisons. The specific steps are as follows:

• (1) Constructing the judgment matrix. Based on actual experience analysis, the indexes are compared in pairs, then obtaining the judgment matrix:

$$A=\left[\begin{array}{cccc}{a}_{11}\hfill & a_{12}\hfill & \dots \hfill & a_{1n}\hfill \\ \multicolumn{1}{c}{a_{21}}& a_{22}\hfill & \dots \hfill & a_{2n}\hfill \\ \multicolumn{1}{c}{\dots }& \dots \hfill & a_{\mathrm{ij}}\hfill & \dots \hfill \\ \multicolumn{1}{c}{a_{n1}}& a_{n2}\hfill & \dots \hfill & a_{\mathrm{nn}}\hfill \end{array}\right]$$

(1)

where a_ij is the importance ratio of index X_i compared to index X_j through the nine-level scaling method, n is the number of indexes.

• (2) Calculating weight. By calculating the geometry mean value of each line elements, the weight ω _i of the index X_i is obtained.

$${\omega }_i=\frac{\sqrt[n]{\underset{j=1}{\overset{n}{\Pi }}{a}_{\mathrm{ij}}}}{\sum _{i=1}^n\sqrt[n]{\underset{j=1}{\overset{n}{\Pi }}{a}_{\mathrm{ij}}}}$$

(2)

• (3) Consistency check. The consistency ratio CR is calculated.

$$\{\begin{array}{c}\mathrm{CI}=\frac{{\lambda }_{max}-n}{n-1}\hfill \\ \mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}\hfill \end{array}$$

(3)

where λ_max is the maximum eigenvalue of the judgment matrix; CI is the consistency index of the judgment matrix; RI is the average random consistency index.

When CR < 0.1, the judgment matrix has satisfactory consistency, otherwise it should be adjusted so that it has satisfactory consistency.

In order to ensure the accuracy of the weight, this paper uses the method of multiple experts’ assessment. The final weight of each index is the average value, and m is the number of experts.

$${\omega }_i^{\prime }=\frac{1}{m}\sum _{j=1}^m{\omega }_{\mathrm{ij}}$$

(4)

Single index assessment

In quality assessment, if the index has no sub-indexes in the objective assessment tree, it is necessary to carry out single index assessment based on the corresponding assessment information. However, the quality assessment indexes of 3D model for aircraft structural parts are incommensurable, that is, there is no uniform measurement standard for each index. If the extracted assessment information is directly used as the quality of indexes, it is not convenient to analyze and compare the indexes. Therefore, before the single index assessment, it is necessary to quantify the characteristic values of indexes. In this paper, fuzzy linguistic term and fuzzy number are used to deal with problems that are difficult to quantify. The single index assessment process is as follows:

1. The assessment quality of index for parts 3D model is uniformly divided into four grades: non-standard, general standard, relative standard, and extremely standard.

2. According to the experts’ opinions, the quality grade division rules of indexes are formulated to assess single index by linguistic terms.

3. The membership function values of linguistic variable are represented by triangular fuzzy numbers.

Triangular fuzzy number is a method of converting fuzzy and uncertain language variables into definite values. ⁵⁵ A triangular fuzzy number, denoted as t = (l, m, u), has the following membership function:

$$\mu (x)=\{\begin{array}{cc}0,\hfill & x\in (-\infty ,l);\hfill \\ \multicolumn{1}{c}{(x-l)/(m-l),}& x\in [l,m];\hfill \\ \multicolumn{1}{c}{(u-x)/(u-m),}& x\in (m,u];\hfill \\ \multicolumn{1}{c}{0,}& x\in (u,+\infty )°\hfill \end{array}$$

(5)

where l, m, u (l ≤ m ≤ u), respectively, are the smallest possible value, the most promising value, and the largest possible value of the fuzzy number.

Due to the assessment results are made by linguistic terms, it’s not available to calculate. Triangular fuzzy number is used to convert the linguistic terms. The relationship between the linguistic terms and the numerical values is shown in Figure 4, where non-standard (N) = [0, 0, 0.33], general (G) = [0, 0.33, 0.67], relatively standard (RS) = [0.33, 0.67, 1], extremely standard (ES) = [0.67, 1, 1].

Figure 4.

Triangular fuzzy numbers conversion relationship.

Fuzzy synthesis evaluation

Fuzzy synthetic evaluation is a method based on fuzzy mathematics, and synthetically assessing all relevant factors. Since each quality factor of parts 3D model affects the assessment results, the weighted mean model M (•, ⊕) is used as the calculation operator of evaluation model. Moreover, this paper uses the four-level quality division rules of indexes to quantify the extracted assessment information uniformly, and uses the degree of membership theory to convert four-level quality assessment into quantitative assessment. By calculating the index weight and triangular fuzzy number, the quality of parts 3D model is assessed. Fuzzy synthetic evaluation is divided into three phases.

First, the assessment vectors of the bottom indexes obtained by single index assessment are used as the membership matrix of the upper level (k-1 level), and this level fuzzy evaluation is conducted, where k is the number of layers for indicators in the objective assessment tree.

$$R_i=\sum _{i=1}^n{\omega }_{{\mathrm{x}}_{\mathrm{ij}}}{M}_{{\mathrm{x}}_{\mathrm{ij}}}$$

(6)

where R_i is the assessment vector of k-1 level index X_i, ω _ij refers to the weight of the bottom index X_ij, M_ij is a triangular fuzzy number obtained by single index assessment.

Then, the assessment vectors of child-level indexes are used as the new membership matrix to carry out the upper fuzzy evaluation.

Finally, the assessment vector of each index in the objective assessment tree is calculated from the bottom up, which is converted to a corresponding linguistic term to describe the quality grade. In order to comprehensively and accurately measure the quality grade of each index, this study will not use the traditional treatment method based on the closeness of triangular fuzzy numbers, but treat the assessment vector by means of a fuzzy envelope, which is a language interval. By this method, the final results will be the minimum linguistic term interval that contains assessment vector. Liu and Rodríguez ⁵⁶ found that this method can avoid the loss of information due to data over-processing.

After the quality assessment of parts 3D model, the enterprises conduct design feedback and design change based on the assessment results. Recommendations for different quality levels are as follows:

Non-Standard: There are problems such as inability to machine, parts 3D model need be returned to redesign.

General Standard: There are partial problems, parts 3D model need to be made design changes.

Relatively Standard: There are some minor problems, parts MBD model need to be made simple changes.

Extremely Standard: Parts 3D model can be published.

Case study

This section takes the quality assessment of 3D model for an aircraft structural part as an example and invites 20 experts determine the weights and quality grade division rules of indexes. Based on these, fuzzy synthesis evaluation is carried out. This section aims to identify the higher-weight and error-prone design factors in aircraft structural parts design, and proposes corresponding preventive measures.

Assessment information extraction

The assessed part and extracted assessment information are shown in Figure 5. The example shown in the figure is the information about the feature groove in the main beam. On the one hand, the overall information of the main beam is described, and the feature groove and its parameters are described. On the other hand, the corresponding assessment index node is described.

Figure 5.

The assessed part and extracted assessment information.

Objective assessment tree configuration

According to the enterprise requirements and the extracted assessment information, the assessment indexes are selected from the general assessment index system. Then, according to the top-down and breadth-first search method, the objective assessment tree is configured, as shown in Figure 6.

Figure 6.

The objective assessment tree for an aircraft structural part 3D model.

Weight determination and single index assessment

In quality assessment, the choice of experts is essential to the assessment results. Therefore, it is necessary to focus on the number of experts and the selection process. To ensure the credibility of the assessment results, this study selects 20 experts. These experts have the following characteristics: (1) Engaging in aircraft research and development. (2) Having rich experience in the design/manufacturing of aircraft structural parts. (3) Having rich knowledge of engineering assessment. The details of the experts are shown in Table 3.

Table 3.

Details of the experts.

Classification	Subclass	The number of experts
Work-type	Product design	10
	Process planning	5
	Quality inspector	5
Work seniority	1–5 years	2
	6–10 years	15
	11-years	3
Number of research projects	1–3	2
	4–6	16
	7-	2

The work of experts has two aspects: (1) Formulating the quality grade division rule of each index. For example, the quality grade division rule of the index groove depth (X₁₇₃) is formulated according to the ratio of groove depth to corner radius, denoted as D/R, where ES’ rule is D/R≤3.5, RS′ rule is 3.5 < D/R ≤ 4, G′ rule is 4 < D/R ≤ 4.5, and N′ rule is 4.5 <D/R. The quality grade division rules of indexes in the objective assessment tree are shown in Table 4. (2) Weight assignment. Taking the judgment matrix of the first-level indexes in the objective assessment tree constructed by an expert as an example for calculation. Through the comparison of four indexes, the judgment matrix is constructed as follows.

$$A_1=\left[\begin{array}{cccc}1\hfill & 2\hfill & 3\hfill & 5\hfill \\ \multicolumn{1}{c}{1/2}& 1\hfill & 2\hfill & 3\hfill \\ \multicolumn{1}{c}{1/3}& 1/2\hfill & 1\hfill & 2\hfill \\ \multicolumn{1}{c}{1/5}& 1/3\hfill & 1/2\hfill & 1\hfill \end{array}\right]$$

Table 4.

Quality grade division rules of indexes in the objective assessment tree.

Index	Extremely standard	Relatively standard	General	Non-standard
X11	*AXS-C = absolute coordinate system && *AXS-LOCAL = relative coordinate system && *AXS-PART = *AXS-C && NO(AXS-LOCAL) ≤3	*AXS-C = absolute coordinate system && *AXS-LOCAL = Relative coordinate system && *AXS-PART = *AXS-C && NO(AXS-LOCAL) >3	*AXS-C = absolute coordinate system && *AXS-LOCAL = relative coordinate system && *AXS-PART = AXS-LOCAL	*AXS-C ≠ absolute coordinate system||*AXS-LOCAL ≠ relative coordinate system
X161	NO (first-level node) = 12	NO (first-level node) = (10–11) && NO(geometry node) = 4	NO (first-level node)  =  (8–9) && NO (geometry node)  = 4	NO (geometry node) <4
X162	NO (partbody) = 1 && similar axis/reference plane ∈ same open_body && NO (axis) ≤50	NO (partbody)  = 1 && similar axis/reference plane ∈ same open_body && NO (axis) >50	NO (partbody) = 1 && similar axis/reference plane ∉ same open_body	NO (partbody) ≠ 1
X163	NO (non-allowed reference features)  = 0	NO (non-allowed reference features) = 1	NO (non-allowed reference features) = 2	NO (non-allowed reference features) > 2
X171	Rmin (curvature)  < THK(shell) < R(HB)	Rmin (curvature) < THK(shell) <  1.1 R(HB)	Rmin(curvature) < THK(shell) < 1.2 R(HB)	Rmin(curvature) > THK(shell)
X172	NO (cusp) = 0	0 < NO(cusp) ≤ 2	2 < NO(cusp) ≤ 5	5 < NO(cusp)
X173	D/R (corner) ≤ 3.5	3.5 < D/R(corner) ≤ 4	4 < D/R(corner)  ≤ 4.5	4 < D/R(corner)
X174	NO (cavity) = 0 && R(draft) ≥ 90°	NO (cavity) = 0 && R(draft) ≤ 85°	NO (cavity) = 0 && R(draft) ≤ 80°	NO (cavity) > 0
X175	0 < NO (fillets angle) ≤ 5	5 < NO (fillets angle) ≤ 10	10 < NO (fillets angle) ≤ 15	15 < NO (fillets angle)
X221	Integrity rate = 100%	97%≤Integrity rate<100%	95%≤Integrity rate<97%	Integrity rate<95%
X222	NO (non-allowed PPWords) = 0	0<NO (non-allowed PPWords) ≤ 5	5<NO (non-allowed PPWords) ≤ 10	10 < NO (non-allowed PPWords)
X223	NO (repeated program) = 0	NO(repeated program) = 1	NO(repeated program) = 2	NO (repeated program) > 2
X241	NO (safety plane) = 1 && Z(blank) < Z (safety plane) < Z (machine origin)	NO (safety plane) = 1 && Z(safety plane) = Z(machine origin)	NO (safety plane) = 1&& Z(safety plane)) < Z(safety plane)	NO (safety plane) = 0
X242	NO (nonexistent tool) = 0	NO (nonexistent tool) = 1	1 < NO (nonexistent tool) ≤ 3	3 < NO (nonexistent tool)
X243	NO (nonexistent machine) = 0	NO (nonexistent machine) = 1	1<NO (nonexistent machine) ≤ 3	3 < NO (nonexistent machine)
X251	ΔZ = Zmin	Zmin < ΔZ < Cutting depth	0 < ΔZ < Zmin	ΔZ = 0||ΔZ ≥cutting depth
X252	NO (interference) = 0	NO (interference) > 0	NO (interference) > 0	NO (interference)  > 0
X31	*AXS-LOCAL = absolute coordinate system*AXS-TO = *AXS-LOCAL && NO (AXS-LOCAL)  ≤ 3	*AXS-LOCAL = absolute coordinate system*AXS-TO = *AXS-LOCAL && NO (AXS-LOCAL)  > 3	*AXS-LOCAL = absolute coordinate system*AXS-TO ≠ *AXS-LOCAL	*AXS-LOCAL = relative coordinate system
X321	NO (interference) = 0	NO (interference) > 0	NO (interference) > 0	NO (interference) > 0
X322	NO (freedom) = 0	NO (freedom) > 0	NO (freedom) > 0	NO (freedom) > 0
X323	NO (process convex) = 6||8	8 < NO(process convex) ≤ 12	0 < NO(process convex) < 6	NO (process convex) = 0
X411	NO(coordinate system) = 1	NO(coordinate system) = 0	NO(coordinate system) = 0	NO (coordinate system) = 0
X412	NO (non-tested dimension) = 0	NO (non-tested dimension) = 1	NO (non-tested dimension) = 2	NO (non-tested dimension) > 2
X421	NO (lightweight model) = 1	NO (lightweight model) = 0	NO(lightweight model) = 0	NO(lightweight model) = 0
X422	NO (repeated path) = 0	NO (repeated path) = 1	NO (repeated path) = 2	NO (repeated path) > 2
X423	NO (interference) = 0	NO (interference) = 1	NO (interference) = 2	NO (interference) > 2

According to equation (2), the weights of the first-level indexes in the objective assessment tree can be obtained.

[0.48, 0.27, 0.16, 0.09] ^T

According to equation (3), the consistency check is followed.

λ_max1 = 4.013

CR₁ = 0.048

RI = 0.90

CR = 0.005

Because CR < 0.1, the judgment matrix is considered to have satisfactory consistency. Then, by averaging the weights of 20 experts, the final weights of the first-level indexes in the objective assessment tree are obtained. The weight of design model (X₁) is 0.52, the weight of process model (X₂) is 0.26, the weight of tooling model (X₃) is 0.13, and the weight of test model (X₄) is 0.09. The weights of other level indexes in the objective assessment tree are as follows:

$${[{\omega }_{{\mathrm{x}}_{11}},{\omega }_{{\mathrm{x}}_{16}},{\omega }_{{\mathrm{x}}_{17}}]}^T={[0.20,0.29,0.51]}^T$$

$${[{\omega }_{{\mathrm{x}}_{22}},{\omega }_{{\mathrm{x}}_{24}},{\omega }_{{\mathrm{x}}_{25}}]}^T={[0.43,0.36,0.21]}^T$$

$${[{\omega }_{{\mathrm{x}}_{31}},{\omega }_{{\mathrm{x}}_{32}}]}^T={[0.27,0.75]}^T$$

$${[{\omega }_{{\mathrm{x}}_{41}},{\omega }_{{\mathrm{x}}_{42}}]}^T={[0.6,0.4]}^T$$

$${[{\omega }_{{\mathrm{x}}_{161}},{\omega }_{{\mathrm{x}}_{162}},{\omega }_{{\mathrm{x}}_{163}}]}^T={[0.48,0.35,0.17]}^T$$

$$\begin{array}{c}{[{\omega }_{{\mathrm{x}}_{171}},{\omega }_{{\mathrm{x}}_{172}},{\omega }_{{\mathrm{x}}_{173}},{\omega }_{{\mathrm{x}}_{174}},{\omega }_{{\mathrm{x}}_{175}}]}^T\hfill \\ ={[0.20,0.20,0.20,0.20,0.20]}^T\hfill \end{array}$$

$${[{\omega }_{{\mathrm{x}}_{221}},{\omega }_{{\mathrm{x}}_{222}},{\omega }_{{\mathrm{x}}_{223}}]}^T={[0.58,0.15,0.27]}^T$$

$${[{\omega }_{{\mathrm{x}}_{241}},{\omega }_{{\mathrm{x}}_{242}},{\omega }_{{\mathrm{x}}_{243}}]}^T={[0.60,0.20,0.20]}^T$$

$${[{\omega }_{{\mathrm{x}}_{251}},{\omega }_{{\mathrm{x}}_{252}}]}^T={[0.50,0.50]}^T$$

$${[{\omega }_{{\mathrm{x}}_{231}},{\omega }_{{\mathrm{x}}_{232}},{\omega }_{{\mathrm{x}}_{233}}]}^T={[0.41,0.35,0.24]}^T$$

$${[{\omega }_{{\mathrm{x}}_{411}},{\omega }_{{\mathrm{x}}_{412}}]}^T={[0.30,0.70]}^T$$

$${[{\omega }_{{\mathrm{x}}_{421}},{\omega }_{{\mathrm{x}}_{422}},{\omega }_{{\mathrm{x}}_{423}}]}^T={[0.22,0.25,0.53]}^T$$

By comparing the quality grade division rules of indexes shown in Table 4 with the extracted assessment information, the single index assessment is conducted. Then, the assessment results are represented by triangular fuzzy numbers. All indexes without sub-indexes are assessed, and the assessment results are shown in Table 5.

Table 5.

The results of single index assessment.

Index	Assessment results
	Linguistic term	Triangular fuzzy number
X11	ES	[0.67, 1, 1]
X161	RS	[0.33, 0.67, 1]
X162	ES	[0.67, 1, 1]
X163	RS	[0.33, 0.67, 1]
X171	RS	[0.33, 0.67, 1]
X172	ES	[0.67, 1, 1]
X173	RS	[0.33, 0.67, 1]
X174	ES	[0.67, 1, 1]
X175	RS	[0.33, 0.67, 1]
X221	ES	[0.67, 1, 1]
X222	G	[0, 0.33, 0.67]
X223	RS	[0.33, 0.67, 1]
X241	RS	[0.33, 0.67, 1]
X242	G	[0, 0.33, 0.67]
X243	G	[0, 0.33, 0.67]
X251	RS	[0.33, 0.67, 1]
X252	RS	[0.33, 0.67, 1]
X31	ES	[0.67, 1, 1]
X321	RS	[0.33, 0.67, 1]
X322	ES	[0.67, 1, 1]
X323	G	[0, 0.33, 0.67]
X411	RS	[0.33, 0.67, 1]
X412	ES	[0.67, 1, 1]
X421	RS	[0.33, 0.67, 1]
X422	ES	[0.67, 1, 1]
X423	RS	[0.33, 0.67, 1]

Fuzzy synthesis evaluation

According to equation (6), the assessment vectors of the secondary-level indexes in the objective assessment tree are calculated as follow:

$$R_{{\mathrm{x}}_{16}}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{16j}}{M}_{{\mathrm{x}}_{16j}}=[0.428,0.786,1]$$

$$R_{{\mathrm{x}}_{17}}=\sum _{j=1}^5{\omega }_{{\mathrm{x}}_{17j}}{M}_{{\mathrm{x}}_{17j}}=[0.466,0.802,1]$$

$$R_{{\mathrm{x}}_{22}}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{22j}}{M}_{{\mathrm{x}}_{22j}}=[0.418,0.81,0.95]$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_{24}}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{24j}}{M}_{{\mathrm{x}}_{24j}}=[0.198,0.534,0.868]\hfill \end{array}$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_{25}}=\sum _{j=1}^2{\omega }_{{\mathrm{x}}_{25j}}{M}_{{\mathrm{x}}_{25j}}=[0.33,0.67,1]\hfill \end{array}$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_{32}}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{32j}}{M}_{{\mathrm{x}}_{32j}}=[0.34,0.704,0.921]\hfill \end{array}$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_{41}}=\sum _{j=1}^2{\omega }_{{\mathrm{x}}_{41j}}{M}_{{\mathrm{x}}_{41j}}=[0.568,0.811,1]\hfill \end{array}$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_{42}}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{42j}}{M}_{{\mathrm{x}}_{42j}}=[0.415,0.753,1]\hfill \end{array}$$

The assessment vectors of the secondary-level indexes are used as the new membership matrix, and the assessment vectors of the first-level indexes in the objective assessment tree are obtained as follows:

$$R_{{\mathrm{x}}_1}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{1j}}{M}_{{\mathrm{x}}_{1j}}=[0.496,0.837,1]$$

$$R_{{\mathrm{x}}_2}=\sum _{j=1}^3{\omega }_{{\mathrm{x}}_{2j}}{M}_{{\mathrm{x}}_{2j}}=[0.32,0.681,0.931]$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_3}=\sum _{j=1}^2{\omega }_{{\mathrm{x}}_{3j}}{M}_{{\mathrm{x}}_{3j}}=[0.423,0.778,0.941]\hfill \end{array}$$

$$\begin{array}{c}{R}_{{\mathrm{x}}_4}=\sum _{j=1}^2{\omega }_{{\mathrm{x}}_{4j}}{M}_{{\mathrm{x}}_{4j}}=[0.507,0.788,1]\hfill \end{array}$$

The assessment vectors of the first-level indexes are used as the new membership matrix, and the assessment vector of the quality for this aircraft structural part 3D model is obtained as follows:

$$R_{\mathrm{x}}=\sum _{j=1}^4{\omega }_{{\mathrm{x}}_{2j}}{M}_{{\mathrm{x}}_{2j}}=[0.44,0.783,0.974]$$

The assessment vectors are shown in Figures 7 and 8, and the assessment language value is obtained according to the principle of the minimum assessment linguistic term interval containing assessment vector. Then the results are further analyzed as follows:

1. As noted in Figure 7, it can be concluded that the quality of this aircraft structural part 3D model is relatively standard. The 3D model can only be published after simple modification. As noted in Figure 8, the sources of non-standard factors are mainly from process model (X₂), which are mainly caused by the designers’ lack of experience in process design. This is consistent with the judgment in the referenced.^57–59 Therefore, for designers, due to the lack of process knowledge, it is easy to cause quality problems in the process and tooling model design. The quality of design model is superior to the other three sub-models, that is, because the design standards for aircraft structural parts formulated by aviation manufacturing enterprises are mostly aimed at the design model (X₁) and they are easy to control.

2. The design model is the key to 3D model for aircraft structural parts, which is also the basis for the design of process model, tooling model, and test model. Therefore, the design model has a high weight. This is consistent with the judgment in the referenced.^60,61 The process model is the basis for machining parts, and its weight comes second. Test model is an important factor to ensure product quality, its weight comes third.

3. Although the structure of part specification-tree (X₁₆) affects the quality of design model (X₁), it is not the main factor for the quality of design model, so it has less weight. The main factor is the parameters for different features (X₁₇), for example, the larger groove depth affects the subsequent machining. In process model, NC program (X₂₂) is critical to the parts machining, so it has a high weight for parts quality.^45,57 In terms of tooling model, if the design of fixing-clamping devices (X₃₂) is wrong, the machine and blank will be damaged, so they have a high weight. ⁴⁷

4. Furthermore, the proposed method makes it easier to identify the factors influencing the quality of 3D model for aircraft structural parts and they are mainly from higher-weight and error-prone indexes. Designers should pay attention to these indexes. And the quality grade division rules of indexes can also provide design reference.

Figure 7.

The assessment vector of the quality for part 3D model.

Figure 8.

The assessment vector of the quality for the first-level index.

Comparison and discussion

• (1) Use of configurable index system model. In order to prove the accuracy and efficiency of the assessment process by using the configurable index system model, the assessment results are compared with those of using a fixed assessment index system.

According to different parts types and enterprise requirements, this study established 50 kinds of assessment indexes for feature parameters (X₁₇). If a fixed assessment index system is used, all sub-indexes of index X₁₇ should be assessed. In addition, since there is no corresponding assessment information, the assessment results of these indexes are all non-standard, which is represented by a triangular fuzzy number as [0, 0, 0.33]. To ensure the comparability of the assessment results, the same experts are invited to formulate the quality grade division rule and weight of each index. Through fuzzy synthesis evaluation, the assessment results of the quality for part 3D model (X) and design model (X₁) have changed. The calculation results are shown in Figures 9 and 10.

Figure 9.

Change of the quality assessment result for part 3D model using fixed assessment index system.

Figure 10.

Change of the quality assessment result for index X₁ using fixed assessment index system.

Comparing the two assessment results, it is found that the quality grade of part 3D model (X) is reduced from relative standard to between general and relative standard, and the quality grade of design model (X₁) is reduced from relative standard to general standard. The main source of non-standard factors changed from process model (X₂) to design model (X₁). In this way, it is difficult for enterprises to accurately identify error-prone design factors. Moreover, the enterprises will receive wrong design feedback and design change. Therefore, it is obvious that using a configurable index system model can improve the accuracy of assessment results and the efficiency of assessment process.

• (2) Use of hesitant fuzzy linguistic term set. In order to prove the effectiveness and accuracy by using HFLTS, the assessment results are compared with those of using a fixed assessment values.

First, the relationship between the linguistic terms and the fixed linguistic values is obtained, where non-standard (N) = 0, general (G) = 0.33, relatively standard (RS) = 0.67, extremely standard (ES) = 1. Then, the quality grade division rule and weight of each index remain unchanged. Finally, fuzzy synthesis evaluation is carried out. The final quality grade of each index is determined by the semblance function.

As can be seen from Table 6, the assessment results using fixed assessment values and HFLTS are roughly the same. However, the assessment results will be a fixed value using the fixed linguistic values. In this way, it may make the assessment results tend to be too high or too low, and then affect the judgment of enterprises. For example, the quality grade of process model (X₂) is relatively standard by this method, which may lead designer to ignore this non-standard factor. However, according to the assessment result using HFLTS, the quality grade of process model (X₂) is between general standard and relatively standard. In this way, the sources of non-standard factors cannot be fully tapped, which leads to the designer cannot modify the part 3D model in time, and reduces design efficiency. Therefore, this study suggests that using HFLTS can make the assessment results more accurate and the assessment process more effective.

Table 6.

Change of the quality assessment result for the first-level index and part 3D model using fixed linguistic values.

Index	Assessment results
	Quality values	Fixed linguistic values	HFLTS
X1	0.841	ES	Between RS and ES
X2	0.681	RS	Between G and RS
X3	0.778	RS	Between RS and ES
X4	0.841	ES	Between RS and ES
X	0.791	RS	Between RS and ES

Conclusion

Focusing on the 3D model for aircraft structural parts, this study uses AHP and HFLTS-fuzzy synthesis evaluation methods for the first time to conduct a quality assessment in aviation manufacturing enterprises, and draws the following conclusions:

1. Based on all levels of quality standards and part specification-tree elements, a quality assessment index system of 3D model for aircraft structural parts is established considering design model, process model, tooling model, and test model.

2. The AHP is used to determine the weight of each index. As the experts have different work-type, seniority, and project experience, the weights in the index system are more objective and reasonable.

3. A configurable index system model is established by inserting the selection node into the traditional multi-layer assessment index system, which makes the assessment results more accurate and assessment process more efficient.

4. A fuzzy evaluation model is proposed based on the combination of HFLTS and fuzzy synthesis evaluation. By using HFLTS, the assessment results are more accurate, and the assessment process is more effective.

5. The quality of 3D model for an aircraft structural part is assessed. The assessment results indicate that design model (X₁) owns the largest weight, followed by process model (X₂). The error-prone design factors mainly come from process model (X₂). It is expected that this approach may identify the higher-weight and error-prone design factors. The quality assessment index system and quality grade division rules of indexes can also be used as a reference for parts design.

In the future, this approach can be applied to the quality analysis of 3D model for aircraft assembly, and the index system needs be adjusted accordingly.