Abstract
In order to improve the maintenance training effect of complex mechanical product, a virtual maintenance system was developed. The system framework was proposed, and the main functional modules were elaborated. A multilevel information representation model for complex mechanical product was put forward, and the flowchart of model transformation technology was designed. Moreover, a collision detection method based on hierarchical bounding volume was proposed, and the maintainability analysis and evaluation solution based on maintenance knowledge was presented. Finally, a prototype system was developed, and the proposed system was provedto be efficient through an example of hydraulic winch.
1. Introduction
Nowadays, for the complex mechanical product with integration of mechanical, electronic and computer technology, and so forth, the disassembly/assembly process is more and more complicated in structure. It is hard to carry out the maintenance training tasks through a physical prototype not only due to the great limitation of work environment, but also due to the long maintenance time and high maintenance cost. In order to achieve the maintenance training effect through physical prototype, virtual maintenance system based on virtual reality technology (VR technology) is applied to the maintenance training tasks for complex mechanical product. Due to advantages such as no space limitation, short maintenance time, low maintenance cost, and risk [1–3], virtual maintenance system is widely applied in many engineer fields such as aerospace [4, 5], weapon equipments [6], and medical equipments [7, 8].
At present, many researchers have worked on the problem and proposed different solutions in order to realize high efficiency and visualization of product maintenance process. In [9–11], disassembly sequence planning approach was investigated to simulate machine assembly maintenance process. In [12], a framework capable of performing complex tasks depending on the user's order was proposed, and this framework enabled VR applications to animate from dozens of characters in real time for complex constraints. In [13], a framework for synthesizing new character motions from existing motion capture data based on a frame-based solver was presented, and this ensured animating hundreds of humanoids with different morphologies in real time. In [14], Xie et al. proposed a virtual maintenance training system (VMTS) based on multiagentsystem, and the system is composed of seven collaborative agents. VMTS can be transformed to the development of multi-agent system, thus the robustness, flexibility, and reusability of VMTS were improved. In [15], Liu et al. presented a virtual assembly oriented multilayer exact collision detection algorithm based on the analysis of the problem in the virtual assembly environment. In [16], a novel collision evaluation algorithm based on generalized penetration depth, approximation information, and tolerance information was proposed to solve the problem in which collisions between polygonal models may not reflect collisions between real objects in reality. In [17], a bounding volume hierarchy based on tetrahedron and spheres for collision detection between complex polyhedra was presented. The method drastically reduced the number of intersection tests between the features of both polyhedra. In [18], a new scene graph structure was presented to remove the limitations of traditional scene graph structure inthe development of virtual maintenance training system (VMTS). In [19], Liu et al. presented a multilayer component information expression structure model in virtual assembly. This method solved the data transformation between CAD system and virtual assembly system very well. In [20], the therblig characteristics required by virtual maintenance simulation were analyzed, and the maintenance therbligs were classified and defined, respectively. The modeling method of virtual human was studied, such as fatigue model of virtual human upper limb [21], regulation of human locomotion [22], and modeling of skeletal muscles coupled with fatigue [23]. In [24], the implementation of basic body, retrieval body, and control body was studied, and the architecture and theory of multibody structure maintenance information modeling for multilayer representation were proposed. In [25], the information model and directed graph expression were studied, and the collaborative manufacturing task decomposition process covering the design of LMU and LMP was put forward. Moreover, the design methods for LMU and LMP were provided. In [26], the problem of how to build a virtual prototype (VP) used for virtual maintenance simulation was investigated, and a maintenance-task-oriented virtual maintenance prototype (VMP) modeling was addressed to represent geometry modeling and human-object interaction modeling. Moreover, other systems were established based on virtual reality [27–30].
Bearing the above observations in mind, we present a virtual maintenance system for complex mechanical product, and the rest of this paper is organized as follows. Section 2 presents the system framework and elaborates the main functional modules. Section 3 describes the key technologies such as model transformation technology, collision detection method, and a maintainability analysis and evaluation solution based on maintenance knowledge. Section 4 develops a prototype system based on the above-mentioned key technologies. Section 5 summarizes our conclusions and future work.
2. Framework of the Proposed System
According to the structure and maintenance training demand of complex mechanical product, a virtual maintenance system should meet the following requirements.
A virtual maintenance training scene with high credibility should be modeled.
The integration between CAD system and virtual maintenance system should be realized, and as much as possible CAD model information should be reserved.
Natural, simple input/output interfaces should be provided.
Various maintenance tools can be applied in the virtual maintenance system.
Maintenance knowledge management and maintainability analysis should be provided to improve the automation and intelligence of maintenance process.
According to the above requirements, a framework composed of several functional modules was proposed as shown in Figure 1.

Framework of the proposed system.
There are several functional modules in the virtual maintenance system, and the functionalities of main functional modules are given as follows.
System modeling: it includes virtual maintenance scene modeling, virtual prototype modeling, and virtual human/virtual hand modeling. The virtual prototype models for complex mechanical product are modeled in CAD system and transformed into the virtual maintenance system with a multilevel information representation model.
Virtual maintenance simulation operation: it mainly includes interactive operation, collision detection, and disassembly sequence planning. User sends the maintenance operation orders through the data glove, and the virtual human carries out the maintenance tasks. Meanwhile, the virtual maintenance system executes the collision detection operation.
Maintainability analysis and evaluation solution: it includes the maintenance inspection, maintainability analysis, and maintenance data management. The virtual maintenance system judges the correctness of user operations timely and prompts users the right guidance based on the maintenance knowledge.
3. Key Technologies
3.1. Model Transformation Technology
A complete product model should contain the following aspects of information: geometrical information, topological information, hierarchical structure information, and assembly constraint information. Usually, the virtual prototype models are transformed from CAD models, and the geometry information of CAD models, are expressed by accurate mathematical form. Because of the large amounts of time to display CAD geometric model, it is very difficult to achieve the complex graphic's real-time display in virtual maintenance system. Generally, CAD models can be converted into triangular patch models which can be applied in virtual maintenance system. However, during the transformation process, virtual prototype models also lost the topology information and geometry information. In order to solve this problem, a multilevel information representation model for complex mechanical product is put forward. According to the product information of different levels of abstraction, product information can be divided into five layers which are product layer, component layer, part layer, geometric surface layer, and display layer as shown in Figure 2.

Multilevel information representation model.
Product layer can be described as follows:
where P is product nodes. The information of product nodes includes name information, function information, quality information, fault information, materials information, and other management information.
Component layer can be described as follows:
where A are component nodes, M c is the hierarchical mapping relationship between component and product, and F c is the relationship of assembly constraints and relative position among the components.
Part layer can be described as following:
where E are part nodes. The part nodes mainly include part name information, physical properties information, technical information, and management attributes. M p is the hierarchical mapping relationship between part and component. F p is the relationship of assembly constraint and coupling among parts such as connection relationship of bolt and splines.
Geometric surface layer can be described as follows:
where G are geometric surface nodes, M g is the hierarchical mapping relationship between geometric surface layer and part layer, and F g is the constraint relationship of geometric surface.
Part is composed of the geometric surface which can be accurate to express the geometric shape and size of a part though keeping internal geometric constraint. The part position can be identified precisely based on the geometric constraint in a virtual maintenance system.
Display layer can be described as follows:
where D are triangular patch nodes. The triangular patch nodes mainly include geometric properties of triangular patch such as coordinates and color. M d is the hierarchical mapping relationship between display layer and geometric surface layer. It was applied in model visualization and collision detection in a virtual maintenance system.
The information of product layer, component layer, part layer, and geometric surface layer can be obtained through the second development of CAD system, and the information of display layer can be obtained through the neutral format files such as STEP and STL format which are output from the CAD system through the model transformation interface. The model transformation process can be shown in Figure 3.

Flowchart of model transformation.
3.2. Collision Detection Method
Collision detection is not only the basic elements of a virtual maintenance system but also the premise for virtual maintenance operation. Virtual maintenance system should possess the function of accurate real-time collision detection. For complex mechanical product, the model coupling degree is very high which would increase the amount of calculation for collision detection greatly. Thus, a quick and efficient collision detection method should be provided to realize the authenticity of a virtual maintenance system.
According to the requirements of collision detection in a virtual maintenance system, a collision detection method based on hierarchical bounding volume is proposed, and the overall process of the proposed method is shown as in Figure 4.

Flowchart of collision detection method.
In virtual maintenance system, the part models are generally composed of triangles, and a collision can be recognized through judging whether the triangles have intersection points. The collision detection algorithm based on triangles can be elaborated as follows.
Supposepair trianglesD K = (L1, L2) exist.
Where L1 = (p1, p2, p3), L2 = (q1, q2, q3).p1, p2, and p3 are the vertexes of triangles L1, and q1, q2, and q3 are the vertexes of triangles L2. L1′, L2′ are the planes of pair triangles.
Determinant:
If det(p1, p2, p3) < 0, p1, p2, and p3 should be arranged clockwise, and if det(p1, p2, p3) > 0, p1, p2, andp3 should be arranged counterclockwise.
n1, n2, and n3 are dot product of
Judgment of the relative place for two triangles in the three-dimensional space can be described as follows.
If n1, n2, and n3 have the same sign, triangles are disjoint.
If only one of n1, n2, and n3is zero and the other two have the same sign, only one vertex of the L2 is in the plane L1′. For this situation, the judgment of relative place for two triangles should refer to the situation f. If the other two have different signs, the judgment of relative place for two triangles should refer to the situation e.
If only two of n1, n2, and n3 are zero, the judgment of relative place for two triangles should also refer to the situation e.
If all of n1, n2, and n3 are zero, L1 and L2 are in the same plane. The judgment of relative place for two triangles should also refer to the situation g.
If n1, n2, and n3 have different signs, the line segment v1v2 which is one intersection of L1 and L2′ to the other intersection of L1 and L2′ is figured out, and the line segment v3v4 which is one intersection of L2 and L1′ to the other intersection of L2 and L1′is also figured out. Triangles have the same joint in the condition of the line segment v1v2 and the line segment v3v4 having overlap. Otherwise, triangles do not have joint.
Judgment of the relative place for point and triangle [31]. Set one point r in the plane L1′;
If det(p1, p2, p3) > 0, r is in the triangle L1 in the case of t i < 0, i = 1,2, 3. Ifdet(p1, p2, p3) < 0, r is in the triangle L1 in the case of t i > 0, i = 1,2, 3.
Judgment of the relative place for two triangles in the same plane [32]. Two triangles in the same plane are disjoint only if there is an edge of a triangle making the two triangle vertexes on the opposite side. In other words, two triangles are disjoint in two conditions, and one condition can be described as follows.
There is an edge R
i
of the triangle L1, and
The other condition can be described as follows.
There is an edge Q
j
of the triangle L2, and
Otherwise, they are intersection.
3.3. Maintainability Analysis and Evaluation Solution
For a complex mechanical product, maintenance tasks are complicated and difficult to be carried out by a common engineer. Usually, the task of maintainability analysis should be carried out, and the evaluation results and modification proposals including the factors of object accessibility, object visibility, maintenance time, and maintenance environment should be provided to guide the maintenance operations. In order to improve the intelligence and automation of maintenance process, a maintainability analysis and evaluation solution based on maintenance knowledge are proposed as shown in Figure 5.

Maintainability analysis and evaluation solution.
The maintainability analysis and evaluation solution for complex mechanical product are composed of reachability analysis, maintenance time estimation, error analysis, maintenance safety analysis, disassembly sequence automatic planning, and dismantling path automatic detection. Maintenance knowledge can be acquired from product information, relative standard, and expert experience knowledge. According to product model information features, maintenance knowledge can be divided into three domains: structure domain, maintenance domain, and analysis domain.
Structure domain mainly expresses product structure information such as size, color, weight, and assembly relationship. For a more reality training effect, it is important to make the size and color of the model as far aspossible as same as the reality object. Assembly relationship induces topological relationship, tolerance information, and spatial position among parts and is prepared for maintenance simulation.
Maintenance domain mainly includes maintenance process for product operation knowledge such as dismantling path planning, testing, therblig, and selection of maintenance tools. In order to realize maintenance simulation in nonimmersion virtual maintenance system, the virtual human movement must be decomposed. Maintenance activities are decomposed into therbligs based on motion analysis method for engineering, and common maintenance activity is composed of a series of therbligs. The therbligs are divided into three classes: moving class, static class, and operation class. The detail information of therblig classes can be described in Table 1.
The detail information of therblig classes.
Analysis domain mainly focuses on the maintenance analysis and evaluation such as maintenance fault analysis, maintenance time estimation, and maintenance reachability analysis. Maintenance fault analysis acquires test data in the maintenance process to judge failure reason, failure type, the failure effects, and so forth. The test data includes the collision detection, distance detection, and tool change frequency. Maintenance time estimation mainly determines spending the real time of a therblig in reality maintenance process which is based on the time of a therblig in the virtual maintenance process and the zoom factor based on the moving element maintenance difficulty level and the human posture. Maintenance reachability analysis mainly analyses maintenance personnel space, maintenance position, and visual range in the maintenance process.
4. An Example
Based on EON studio platform, a prototype of the virtual maintenance system for complex mechanical product was developed, and an application in the mechanical product of hydraulic winch was providedas shown in Figure 6.

Application of the virtual maintenance system.
The 3D models in this example were set up by the CAD software UG and were converted to the virtual maintenance system through the model transformation interface as shown in Figure 6(a). In the proposed system, the bill of material for hydraulic winch can be organized according to the assembly constraints and the multilevel information representation model as shown in Figure 6(b). The system carried out real-time detection of collision whether or not it occurred. If collision occurred between two parts, the color of two parts was changed to a different color, meanwhile, the system issued a warning in the form of sounds, as shown in Figure 6(c) and enlarged Figure 6(d). The maintenance activities were composed of therbligs, and the maintenance activities of gear-shaft installation that consist of therbligs were shown as in Table 2. The components put on the table, as that shown in Figure 6(e). Virtual human was adjusting the position of gear shaft as shown in Figure 6(f). The system can automatically analyze maintainability such as tool change frequencies, maintenance space, and maintenance visibility, as shown in Figure 6(g). Finally, the disassembly results and maintainability analysis report were output as shown in Figure 6(h).
The therbligs of gear-shaft installation.
5. Conclusions and Future Work
A virtual maintenance system for complex mechanical product was designed and implemented to improve the maintenance training effect. The framework and its key technologies of the proposed system were elaborated. Finally, an example was provided, and the proposed system was proved to be feasible and efficient.
Ongoing and future work will focus on the improvement and extension of the proposed system. Furthermore, the applications in other complex mechanical products are also an important research for the authors.
Conflict of Interests
The authors declare that they have no conflict of interests.
Footnotes
Acknowledgments
The support of the National Natural Science Foundation of China (nos. 51005231, 51209203), the National Natural Science Foundation of Jiangsu Province (no. BK2012131), and the Priority Academic Program Development of Jiangsu Higher Education Institutions in carrying out this research is gratefully acknowledged.
