Abstract
A novel three-dimensional interface using immersive augmented reality to perform real-time visual analysis of structural models is presented. The interface integrates and builds on the functionalities of two commercial tools: ‘Leonar3Do’, for visual inspection in a fully three-dimensional immersive environment and ‘SAP 2000’, for structural analysis and simulation. The resulting interface allows the user to visualize the structural design model in three-dimensions, apply forces/loads directly with a three-dimensional physical pointer to indicate their magnitudes and directions and meanwhile observe the behavior of the structure under this action in fully perceived three-dimension. It integrates traditional structural analysis software, three-dimensional viewing and immersive virtual reality environment. The interface facilitates understanding of the different interactions between the structural components, detection of possible structural design weaknesses and improvement of the structural model in order to quickly develop better virtual prototypes.
Introduction
In the engineering design field, computer-aided engineering (CAE) tools are widely used for structural analysis, such as stress analysis with SAP 2000 software. These tools facilitate a design process with structural models, allowing the development of virtual prototypes capable of simulating such physical properties as material, weight and appearance. The models are then developed as structural analysis models and used to simulate their structural behavior under specific forces/loads. For analysis, force vectors and their positions are applied to the structure. After performing a simulation, the results are then given in the form of two-dimensional (2D) graphical and numerical data. Designers/engineers can then take corrective decisions afterward in order to improve the original design intent and optimize the structure.
However, the creation of a structural analysis model from a computer-aided design (CAD) model, even a simple wireframe model, requires a tedious process to add forces/loads and material properties. To test and optimize a design idea with different forces/load applied is a time-consuming task. On the other hand, the visualization of a structural analysis result on 2D screen lacks three-dimensional (3D) immersive feeling to quickly understand real structural model behaviors.
We propose utilizing current advances in the area of human computer interaction and augmented reality (AR) to significantly improve the interface for engineers and designers to interact with CAE tools in an immersive 3D environment for structural design. The user can not only interact in 3D with a virtual/physical design model, but also can view analysis results in a fully perceived 3D environment. In this paper, we have developed such an interface with the Leonar3Do toolkit incorporated with the stress analysis software SAP 2000. This interface provides a full 3D real-world-like visual and gesture interaction when simulating the effect of different types of forces applied to structural models.
Related work
According to the Oxford dictionary, 1 virtual reality (VR) is ‘a computer-generated simulation of a three-dimensional environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors’. In a VR environment the user wearing a head mounted display (HMD) device is concentrated on the virtual representation of a real world object. One drawback of the VR environment is that, providing realistic haptic feedback is circumvented, it is not very good for interactive design. The immersive AR (IAR) technology has overcome this drawback. 2 In an IAR environment, the user can see the real and virtual world mixed together. To do so, 3D virtual objects are integrated into a 3D real environment, perceived by the user through 3D visualization technology, like shutting glasses, in real time. 3
IAR has been used in applications in several fields like surgery, 4 manufacturing simulation, 5 product design 6 and education. 7 It has also been used for computational fluid dynamics (CFD) visualization 8 and aesthetic industrial design. 9 But IAR systems have not been fully explored for structural design and analysis.
The new interface
To implement a 3D IAR interface for structural analysis we take functional advantages of two commercial software tools: the Leonar3Do kit and SAP 2000. Leonar3Do is a VR–AR kit that allows full 3D interaction. By using a specially designed, fully 3D airborne device with six degrees of freedom called the ‘bird’ and stereo viewing shutter glasses, Leonar3Do allows 3D immersive interaction with 3D models by moving the bird freely in the 3D space between the user and the computer screen, as shown in Figure 1. SAP 2000 is structural analysis software that allows simulation of the effect of forces applied to a structural model.
Leonar3Do kit being used together with a 120 Hz computer screen.
By integrating the functionalities of both SAP 2000 and Leonar3Do kit, we built a new interface for designing and testing a structure by applying 3D forces directly on a structural model and visualization of the structural model in a fully 3D immersive environment. In this way, the user can freely explore the 3D virtual model and inspect for possible design weakness by moving the structure in the real 3D space or changing the viewpoint by moving his/her head. When the tip of the airborne is on the structural model in 3D, haptic vibration feedback is sensed by the user thus indicating touching/collision, and complementing 3D visual feedback. In this way the user can also apply a force, by simply touching the structure with the airborne. Because Leonar3Do is a six degrees of freedom device and the airborne position is constantly tracked, the positional and directional information of the airborne can be obtained in a 3D environment when touching the structure. When a force position and its direction are supplied with the airborne, the force magnitude is then entered through keyboard inputs because structural analysis usually requires applying a precise force magnitude parameter owing to security factors or specific structural requirements. The force data, including the force vector (direction and size) and its position, is then sent to the structural analysis software SAP 2000. This software carries out the simulation of the effect of the force on the structure and sends the resulting 3D deformed geometry back to Leonar3Do, where it can be visualized in a fully perceived 3D environment.
The method behind the interface
The proposed interface works on the basis of a six-step method, shown in Figure 2. In the figure, each box shows the task carried out in a given step (text inside a rectangle) and the software tool used to do so (text below the rectangle).
The method behind the interface.
First (step 1), a draft 3D structural design is modelled in SAP 2000 by using traditional interfaces (2D screen, keyboard and mouse). Second (step 2), the resulting structural model is transferred into a 3D IAR environment, to carry out visual inspection. Once in the IAR environment, the user can apply forces in an interactive manner using the Leonar3Do tool kit (step 3). In this step, the 3D model transfer from SAP 2000 to Leonar3Do is carried out by an ad-hoc application that reads the number of nodes of the structural model, their geometric relation and 3D coordinates. The user can now specify the forces applied to the structure based on the stereo image of the virtual object and the real image of the airborne. The end point (tip) of the airborne indicates the position of the force to be applied and the body position of the 3D physical airborne provides a direction. The force application point and orientation are, therefore, read from the airborne, and then added to the information entered by keyboard (force magnitude). Next (step 4), the force data is sent back to the SAP 2000 model to carry out the structural simulation of the effect of the force applied to the model. After the simulation, the new geometry is taken back to the 3D IAR environment (step 5), to visually analyse the results, inspect for possible design weakness and improve it accordingly. Note that the interface allows the user to rotate and move the 3D model. Also the position of the user is constantly tracked through the shutter glasses, which allows them to change their point of view of the 3D model. Finally, the model can be improved and transferred back to the structural analysis software for further developments (step 6). The process can be carried out in a cyclic manner until the structural design is deemed ready.
First implementation of the interface
In order to test the method just described, a first prototype of the interface was built with the equipment and software listed in Table 1.
Hardware and software used to build the 3D interface.
The software application of the interface was programmed by using Visual C++, Leonar3Do API and Open GL library for the graphic environment. The application can read the airborne and shutter glasses position and synchronize them with the deployed stereo image. The functionality of the interface was tested by carrying out the six steps of the method described to modify a simple structural model. The steps were performed as follows.
First, as shown in Figure 3(a), a basic orthogonal structure composed by frames representing beams and pillars was modelled in SAP 2000. Then, by using the developed application, the data from the SAP 2000 model was read and transferred into the 3D IAR. Figure 3(b) shows a user interacting with the structural model in the 3D IAR environment of Leonar3Do. The image seen in the screen corresponds to the 3D stereo image that a user with no shutter glasses would see. In the stereo image it is possible to see the structural model and a sphere, representing the airborne. Because this image is built by superimposing right- and left-eye projections in one image, a double and partially blurry image is observed. Figure 3(c) shows a zoomed image of the 3D stereo image shown by Leonar3Do. Next, the 3D position of the airborne is read when pressing its button (pressing the button represents the application of a force). Because the airborne is a six degrees of freedom device, the point and direction of application of the force are also read and stored. The force data (force magnitude entered by keyboard, and its position and angle read from the airborne) is then transferred into SAP 2000 format and transferred back to the structural software for simulation. Figure 3(d) shows the resulting deformed geometry, visualized in SAP 2000 after the structural simulation. Finally, the model deformed by the applied force is sent back into the IAR environment, as shown in Figure 3(e) and (f).
(a) A basic structure modelled in SAP 2000. (b) The user interacting with the model in an IAR environment. (c) The 3D stereo image of the model. (d) The resulting representation in SAP 2000 after the structural simulation. (e) The resulting representation transferred into the IAR environment. (f) The resulting model in a 3D stereo image.
Summary and conclusions
In this paper a novel 3D IAR interface for structural design was presented. The interface can significantly improve the design interaction between a structural designer and its analysis tools by means of a six-step process. With the proposed interface and the improved design process, an example has been used by the authors to test the basic functional principles of the interface on a prototype system. From the testing, it was observed that the new interface can potentially support a new more user-friendly design method to integrate structural design and analysis tools with direct 3D interaction on virtual/physical models and fully perceived 3D visualization for design evaluation.
Although the interface is still in its early development process for structural optimization design, the authors have shown its usability and effectiveness for structural design applications.
Footnotes
This work was supported by the Chilean National Commission for Scientific & Technological Research [FONDECYT 1109252] and the Universidad Técnica Federico Santa María [281025].