Process planning and scheduling for loop layout robotic workcell using virtual reality technology

Abstract

The adoption of virtual reality in manufacturing system simulation had proved its effectiveness in bridging up to the gap between different areas of expertise, especially in product design and manufacturing. Virtual reality had enclosed human–machine interface by enabling the user to be immersed into the virtual environment and experience real-time interaction with the virtual objects. In this article, an implementation of virtual reality in cellular manufacturing system simulation is presented. By utilizing the features of visualization and real-time interaction of virtual reality technology, the manufacturing process of a product had been visualized while the real-time control on the product traveling path based on the user’s input was performed and the corresponding activities that related to the change of traveling path had been predicted in the virtual environment. Through the study, simulation of the manufacturing system in virtual reality showed its potential as a powerful decision support system in process planning and scheduling. Various process planning and schedules can be planned through the virtual environment, while the product traveling distance can be obtained from the developed system.

Introduction

The rapid development of computer technology has led the simulation method of complex manufacturing systems which are developed from 2D sketch to 3D model and finally to virtual reality (VR). The term virtual manufacturing has frequently appeared in recent studies as the result of increasing interest in using VR for system simulation.^1,2 It has generally been defined as a “system which abstract prototypes of manufacturing objects, processes, activities, and principles evolve in a computer-based environment to enhance one or more attributes of the manufacturing process.” ³ Application of VR in the manufacturing system is not limited to visualization purposes. It also provides real-time interaction and control with the virtual prototype, thus enhancing the overall manufacturing system quality from product design, manufacturing process, machining operation, facility layout, and material handling system to all factory entities. Therefore, it has great potential to be a decision-supporting tool for manufacturing system planning and is expected to gain time–cost benefits from its application.

Related works

A detailed literature review and analysis on 290 research papers related to applications in manufacturing has been conducted by Negahban and Smith. ⁴ The most important finding in this study was the significant growing trend of the simulation to be used as a tool to solve different problems in manufacturing systems. In earlier researches, simulation is mainly used as a prediction and simple performance evaluation tool. As the development in computer science in the past decade has been rapidly growing, researchers have shown the interest of merging optimization and simulation for the search of optimal policy that is applied in manufacturing systems. From the observation of the current research trend, the application of simulation in manufacturing system design and operation is expected to continue growing and evolving in the future. ⁴ As the importance of simulation application in manufacturing system continues to grow, researchers have applied VR for the purpose of simulating multiple manufacturing activities such as visualizing and analyzing current systems, improving and redesigning them, and searching for optimal solutions in the emerging problems.

Generally, the application of VR in the manufacturing system can be classified into three main areas which are design process, operation management, and manufacturing process. This can further be divided into three different forms of VR systems according to the level of immersion such as non-immersive VR, semi-immersive VR, and full-immersive VR. ⁵ Through the review on the application of VR systems in all areas, VR systems are not only limited to visualizing the possible problems in manufacturing design and product design but they also allow real-time interaction in the virtual environment to solve the problems effectively and efficiently. ⁵

The idea of adopting VR in manufacturing systems was further expanded through the suggestion of applying VR as a collaboration tool for factory planning based on scenario technique. ⁶ The study presented the idea of continuous VR system support throughout all phases of factory planning which enables the impact prediction of the current decision on process planning in the future development of the entire factory. Several kinds of research have been conducted based on the idea of using the VR system as a visualization tool in manufacturing systems.^7,8,9 VR has been used to build a die-casting virtual factory ⁷ where two different scheduling approaches were applied in the same virtual prototype platform under the same virtual environment so that a comparison between the two approaches could be observed and studied. Through the combination of VR system and discrete event simulation (DES), ⁸ the accuracy of system simulation in the virtual world has greatly improved, which opens the door for analysis to be conducted in the virtual world for energy consumption, resource breakdowns, and human performance analyses. VR systems appear to be a powerful tool to assist in the implementation of value stream mapping (VSM) in US-based small to medium enterprises (SMEs) through visualization. ⁹ VR has also been used to build virtual process flow in both current and future state of VSM, where the time taken for each process is recorded in a VR platform and is used as the comparative variable to visualize the improvement of process planning through VSM. VR is also proven to be a powerful tool for human–robot interaction that is able to develop an offline programming system with real-time simulation. A study by Yap et al. ¹⁰ had used VR to create a virtual teaching system to track user’s hand movement for palletizing (move a pallet from a location to another location); then, this movement path is used as robotic arm movement reference. Finally, the command for robotic arm movement based on simulation is generated, and this command is expected to be applicable in real industrial robots for palletizing movement. The work was further expanded, in which the VR had been used for robotic layout design, and an assembly process of an electronic product consisting of three components was performed by the robotic arm within the virtual robotic cell. ¹¹ The result had shown that the developed VR robotic cell can be used in industrial robotic workcells based on the observed accuracy and effectiveness in the case study.

As VR’s usage in the field of manufacturing systems simulation is continuously expanding, a virtual factory framework has been developed, which aims to achieve the sharing of resources, knowledge, and information to support the design and the management of the entire factory entities at all phases. The virtual factory concept has been applied in recent studies,^12,13 even though the proposed concept and model needs further development in order to deal with the complex manufacturing system network. However, the success of interoperability between software tools supporting the design, management, and performance evaluation of manufacturing system through VR platform has been observed in the work.

One of the main benefits of using VR technology in manufacturing system simulation is the ability to visualize the virtual prototype of workplaces in its real operation environment and real-time interaction within the virtual environment, thus eliminating the need of real mock-ups or real system testing which is time–cost effective and directly impacts the competitiveness of a company. As the virtual factory is able to simulate the real environment, the ergonomic study based on VR platform has been conducted.^14,15,16 In virtual prototypes of workplaces, the ergonomics and work safety of the product manufacturing process can be studied through the integration of VR hardware such as programmable hand tracking and gesture recognition gloves. ¹⁴ Similar works have been done ¹⁵ in order to study the assembly operation action performed by the operator, within an almost real assembly environment where every operation is based on operator’s reaction and is recorded and mapped as a standard assembly operation in the real workplace. An assembly test case was presented by Chryssolouris et al. ¹⁶ where a prototype virtual experimentation environment has been created that uses a planning tool for the assembly process by considering the ergonomic condition.

Application of VR in manufacturing has shown a potential to enhance manufacturing systems in a study by Yang et al. The VR simulation applies noise investigation, engineering change management, and virtual cutting tool with chip formation simulation. By applying changes virtually, the negative impact of changes in real manufacturing systems can be minimized, resulting in improved quality in planning and avoiding production shutdown. ¹⁷ A recent study by Darmoul et al. ¹⁸ in 2015 has performed the application of VR in the robotic cell as well. The virtual environment for the robotic cell was successfully used for layout planning, and a feasible solution from the simulation is then set up as a real robotic workcell.

Through the reviews of previous works, the possibility of applying VR in different levels of the manufacturing system design is high with current computer science and VR software applications, giving a positive result in achieving the initially designed objective. This article aims to study the possibility of applying a VR system in a robotic workcell and the effectiveness of the virtual robotic workcell to act as decision-supporting tools in process planning.

Virtual robotic cell development

This article presents the idea of applying VR system for the robotic cell. Through a scenario technique, the VR system is able to act as a decision-supporting tool for robotic cell planning by selecting a suitable drop–pick position in terms of minimizing process cycle time and travel distance. There are two phases involved in the virtual environment creation. The first phase is the conception of structural design and the second phase is the development of virtual models based on the initial structural design.

Conception of structural design

Initial layout design has been proposed by adopting a small-scaled real conveyor system mock-up available in the manufacturing system lab in which the loop layout system has two computer numerical control (CNC) machines which are denoted as M1 (CNC lathe machine) and M2 (CNC milling machine) as fixed machine placement in the middle of the loop layout robotic workcell that are located separately at two opposite sides of a single gripper robotic arm, R, as shown in Figure 1. The loop layout system has the machine flexibility which provides three suitable positions for the machine placement around the circular conveyor system; thus, in the VR simulation, three dummy machines denoted as M3, M4, and M5 would be placed in these positions for process planning and scheduling analysis. The material would be fed in from the input station; go through three machining sequences, in which the first and third stage of machining is completed by either M3, M4, or M5, while the second stage must be completed by either M1 or M2; and the part would leave the robotic cell at the output station as it had completed all three stages of machining. The single gripper robotic arm functions as the material handling system to transfer the part from the conveyor system to M1 or M2 or vice versa for the second-stage machining process. The complete structural design for the robotic layout is illustrated in Figure 1.

Figure 1.

Robotic cell layout.

Few assumptions were made in proposing a loop layout robotic workcell:

1. One robot holds one part per time (single gripper robotic arm).

2. One machine involved per manufacturing process.

3. Each machine can only hold one part at a time.

4. Each part only performs a maximum of three operations in the cell (the second operation is limited to M1 or M2).

5. No buffer for intermediate storage between the stages within the cell.

6. The robotic cell has an input station and an output station only.

Virtual robotic cell development

The stages to build the virtual robotic workcell are shown in Figure 2. The first stage is the computer-aided design (CAD) modeling process, where the measurement of the real mock-up conveyor system is recorded for the CAD model construction using 3D modeling software (SolidWorks). After the completion of the 3D model, it is exported to the 3DS Max software for 3D model detailing and geometry plane calibration with the utilization of computer graphic software. Next, all 3D models will be transferred into a virtual environment of a VR software (EON Reality) to construct a complete virtual robotic manufacturing system. In the fourth stage, KUKA robot programming script and the conveyor control script are required to simulate the desired motion for each part such as the continuous flow of parts and the KUKA robotic arm motion. To enhance the automation within the VR, pre-programmed virtual sensors were set up to trigger some functions of KUKA robotic arm such as pick up and drop-down product at a certain position. The fifth stage is to create user interfaces that enable real-time interaction of the user with the simulation. The interface shows buttons on the screen and preset commands. The input device is the mouse or the keyboard. The final stage is the most important stage, which is the feedback system design where all the virtual manufacturing data are detected, collected, and delivered to the user for further analysis to assist in decision making. This framework is designed to achieve a full real-time interaction with the virtual objects and visualize the impact of changes within the virtual loop layout system with detailed data.

Figure 2.

Virtual robotic cell development framework.

Architecture of virtual robotic cell simulation

The simulation model is prepared in two modes which are the auto mode, in which the software will read the machine sequence from the text file, or the manual mode, which requires user’s instruction. In the first mode, the simulation window will read the text file based on the given path, and then the part will be initiated to move from the input port to the desired Mi location (only machines arranged around the conveyor loop can be selected). Once the part is in place, the stage 2 operation Mi + 1 will be read from the text file where the second operation is limited to machines within the loop (M1 or M2); thus, the robotic arm will unload parts from Mi to Mi + 1. Then, the operation duration will be displayed as a linear movement from the loading area of Mi + 1 to the unloading area of Mi + 1. Once the part arrives at the unloading area, the third operation stage Mi + 2 reads from the file and thus the robotic arm will move parts from Mi + 1 to Mi + 2. After the part arrives at the final position, the part will move to the output port. In the second mode, the same sequence and procedure occur, but they require full interaction with the user. The simulation will not run automatically; the user needs to select their next desired operation machine Mi + 1, as the part has reached the unloading area of the current machine. The full architecture is shown in Figure 3.

Figure 3.

Simulation architecture.

In order to obtain a comparative result for process planning and scheduling, a distance calculation script was built to obtain the total travel distance of a part for completing its manufacturing process. The main factor affecting the part’s total travel distance is the process sequence that is decided by the location of the machines around the conveyor belt, while the part’s travel distance during the robotic arm’s operation is not considered in the programming script. The travel distance between two workstations is determined by rectilinear measurement, which is the length of the straight lines connecting two points while traveling in only one axial direction at a time. ¹⁹ The rectilinear distance is expressed in the absolute value of x-distance and y-distance and is expressed as in equation (1)

$$|\mathrm{x}-{\mathrm{a}}_{\mathrm{i}}|+|\mathrm{y}-{\mathrm{b}}_{\mathrm{i}}|$$

(1)

There were a few assumptions made in the script algorithm:

1. The distance calculation is limited to the part moving on the conveyor belt only.

2. The distance of a part moved by the robotic arm is excluded.

3. The part starts at the input port and exits at the output port.

4. The part performs three stages of machining to complete a production cycle but the same machine can only be used once.

The distance calculation script algorithm is applied to calculate the total travel distance of the part. As stated, the part would enter the robotic cell at the input station, go through three processing machines, and exit the workcell at the output station; the part’s traveled distance on the conveyor belt would be calculated through the following procedures:

1. As the simulation started, the initial position of the part at the input station would be read and saved as array data as

$${\mathrm{p}}_{\mathrm{i}}=[{\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}]$$

(2)

where p_i is the initial position of the part x_i and y_i is the geometry coordinate.

1. Then, as the part arrives at the pickup point of the first dummy machine for the first machining process, the position of part at that specific pickup point would be read and recorded in the array as

$${\mathrm{p}}_{\mathrm{mi}}=[{\mathrm{x}}_{\mathrm{mi}},{\mathrm{y}}_{\mathrm{mi}}]$$

(3)

where p_mi is the position of the part at the first machine pick up point x_mi and y_mi is the geometry coordinate.

1. The distance traveled by the part from the input station to the first machine would be calculated by equation (4) using the information from equations (2) and (3)

$${\mathrm{d}}_1=\sqrt{{({\mathrm{x}}_{\mathrm{mi}}-{\mathrm{x}}_{\mathrm{i}})}^2+{({\mathrm{y}}_{\mathrm{mi}}-{\mathrm{y}}_{\mathrm{i}})}^2}$$

(4)

where d₁ is the first calculated distance.

1. After completing the first machining process, the part would unload by the dummy machine at the same pickup point, and the robotic arm would pick the part from the conveyor belt to either M1 (CNC lathe) or M2 (CNC milling) that are located at the center of the robotic cell. As the second machining process is completed, the robotic arm would pick the part from the unloading station to the pickup point of the third machine from processing. The position changing of the part was completed by a robotic arm in these processes, not the conveyor.

2. Then, as the part completed the machining process of the third machine, it would unload to the drop-off point of the dummy machine at the conveyor; the position of that drop-off point would be read and recorded in the array as

$${\mathrm{p}}_{\mathrm{mi}+1}=[{\mathrm{x}}_{\mathrm{mi}+1},{\mathrm{y}}_{\mathrm{mi}+1}]$$

(5)

where p_mi+1 is the position of the part at the first machine pick up point x_mi+1 and y_mi+1 is the geometry coordinate.

1. As the part has completed the three processes, it would be sent to the output station through the conveyor; the final position of the part (output station) before leaving the robotic workcell would be recorded as

$${\mathrm{p}}_{\mathrm{o}}=[{\mathrm{x}}_{\mathrm{o}},{\mathrm{y}}_{\mathrm{o}}]$$

(6)

where p_o is the final position of the part x_o and y_o is the geometry coordinate.

1. The distance of travel by the part from the third machine to output station would be calculated by equation (7) using the information from equations (5) and (6)

$${\mathrm{d}}_2=\sqrt{{({\mathrm{x}}_{\mathrm{o}}-{\mathrm{x}}_{\mathrm{mi}+1})}^2+{({\mathrm{y}}_{\mathrm{o}}-{\mathrm{y}}_{\mathrm{mi}+1})}^2}$$

(7)

where d₂ is the second calculated distance.

1. The total travel distance of the part is obtained through equation (8) using information from equations (4) and (7)

$$\mathrm{D}={\mathrm{d}}_1+{\mathrm{d}}_2$$

(8)

where D is the total travel distance.

The script flow is shown in Figure 4.

Figure 4.

Distance calculation script algorithm.

Simulation result and discussion

Figure 5 showed the complete virtual loop layout robotic workcell in VR software. According to the assumption made, there is a choice of three different machine selections at the first stage, two machine selections at the second stage (however, since there is robotic movement while handling the part, the distance for selecting either M1 or M2 is assumed to be the same, which has less or negligible effects on the total travel), and two machine selections at third stage, which means that there are a total of six sequences available for a part shown in equation (9)

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{P}\left(\mathrm{Process}\mathrm{Sequence}\right)=\mathrm{P}\left(\mathrm{first}\mathrm{machine}\right)}\\ \multicolumn{1}{c}{\times \mathrm{P}\left(\mathrm{second}\mathrm{machine}\right)\times \mathrm{P}\left(\mathrm{third}\mathrm{machine}\right)}\end{array}$$

(9)

where P(first machine) = 3, three dummy machines available for the process sequence planning; P(second machine) = 1, two CNC machines available for the process sequence planning, but the choice of the second machine does not affect the total travel distance as the material handling system used to load and unload the part to the second machine was a robotic arm. Thus, the possible output would be 1.

Figure 5.

Virtual loop layout robotic workcell.

P(third machine) = 2, two dummy machines available for the process sequence planning; it is because of the assumption that a part would not enter the same machine twice, and as one of the dummy machines has been selected as the first processing machine, two dummy machines are left available for completing the third machining process.

From equation (9)

$$\mathrm{P}\left(\mathrm{Process}\mathrm{Sequence}\right)=3*1*2$$

The distance for each possibility was recorded. Since it is in a virtual environment, the distance taken is in the virtual unit that had been calibrated as 1 mm in physical loop layout system which is equal to 1 virtual unit in a virtual loop layout system. Table 1 shows the total travel distance of each travel path and the comparison of alternative paths involving same machines but in different processing sequences using the percentage of difference formulated as in equation (10)

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{Percentage}\mathrm{of}\mathrm{difference}}\\ \multicolumn{1}{c}{=\frac{\mathrm{Higher}\mathrm{value}-\mathrm{Lower}\mathrm{value}}{\mathrm{Lower}\mathrm{value}}*100\mathrm{\%}}\end{array}$$

(10)

Table 1.

Data comparison for different process plans.

Part sequence	Distance (virtual unit)	Part sequence	Distance(virtual unit)	Percentage ofdifference (%)
	2808.724		5507.886	96.10%
	1282.580		6943.949	441.96%
	2662.172		5624.044	111.26%

The greatest comparison observed is the part sequence of M3–M1–M5 (1282.580 mm) and the part sequence M5–M1–M3 (6943.949 mm), in which the travel distance of M5–M1–M3 is the time of M3–M1–M5. This may be because M3 is the first machine from input station and M5 is the last machine before output station; if the part travels according to the sequence M5–M1–M3, the part almost travels the distance of two loops to complete the manufacturing process. While the percentage of difference for the other two pairs is relatively lower, it may be due to the machines in the first stage and third stage located next to each another, so the repeated travel distance is lower.

The distance travel and scheduling of a manufacturing system can be done with simple simulation programs such as ARENA and Visual Slam. However, with the aid of the VR system, engineers can immerse in the system and thoroughly review the movement of the robot of products in the simulated virtual environment. Some extra aspects such as the clearance between machines, machines wire routing, workers’ ergonomics, and working environment hazards or risks can be visualized as well. A significant difference between VR simulation and other simulations is the feeling of “being there” in the immersive environment. Users literally feel that they are in the virtual environment; hence, they can make a better judgment on the effectiveness of the designed process path.

Conclusion

A virtual robotic workcell simulation has been designed and developed by integrating 3D modeling software and VR software. It is specifically targeted to solve process planning and scheduling issues. This simulation allows real-time interaction with the virtual objects and real-time modification on the virtual objects. Changes can be made as the simulation continuously runs and the impact can be observed in the virtual environment. This allows the user to evaluate the part’s traveling paths. By simulating the alternative traveling paths in the virtual environment, users can decide which path is more suitable by considering the total traveled distance of the material, as well as the working environmental issues and the future impact of the selection.

With the virtual simulation and the real-time interaction, users can directly see the impact of the proper process planning, hence making a better decision through the VR platform. VR system is a good decision-supporting tool for process planning, products’ traveling path planning, and job sequences planning. From the study, the main advantage of using a virtual robotic workcell is the repeatability factor, where the user can test the various situations and combinations without having real hardware or disturbance. Thus, the user is able to test and compare the result in the virtual environment. Furthermore, this simulation will cut down the cost and time of process planning. The real mock-ups can be avoided, and production still can be continued during the planning and testing stage for the process changes on an existing layout. As in the conducted experiment, the shortest process planning is the M3–M1–M5 process planning, which is only 1282.580 mm. In contrast, the process planning that a part goes through the same machine but in a different order which is M5–M1–M3 process had the highest travel distance which is 6943.949 mm, equal to 441.96% of the M3–M1–M5 process planning.