Implementation of Grid-computing Framework for Simulation in Multi-scale Structural Analysis Data

A new grid-computing framework for simulation in multi-scale structural analysis is presented. Two levels of parallel processing will be involved in this framework: multiple local distributed computing environments connected by local network to form a grid-based cluster-to-cluster distributed computing environment. To successfully perform the simulation, a large-scale structural system task is decomposed into the simulations of a simplified global model and several detailed component models using various scales. These correlated multi-scale structural system tasks are distributed among clusters and connected together in a multi-level hierarchy and then coordinated over the internet. The software framework for supporting the multi-scale structural simulation approach is also presented. The program architecture design allows the integration of several multi-scale models as clients and servers under a single platform. To check its feasibility, a prototype software system has been designed and implemented to perform the proposed concept. The simulation results show that the software framework can increase the speedup performance of the structural analysis. Based on this result, the proposed grid-computing framework is suitable to perform the simulation of the multi-scale structural analysis. Keywordsgrid computing, internet, multi-scale structural analysis, simplified model, detailed model, computer simulation


I. INTRODUCTION
dvances in computer technology and numerical methods have allowed the simulation of engineering problems that traditionally have been addressed only through experimentation and theoretical models.Some industries have been able to design sophisticated engineered systems based solely on computer simulation.In addition, many complex phenomena, such as airplane crashes and car accidents, can be analyzed through computer simulations.In structural engineering, using a computer simulation to realistically represent the detailed behavior of structural systems in various situations, such as the global response and the detailed damage to a structure during a major earthquake, is a goal which remains to be achieved by engineers.
The structural engineering systems, including bridges and buildings, are usually large-scale and contain the effects of various structural components and materials at many scales.Therefore, to successfully create a realistic simulation of a structural system, the global model must be able to capture the true behavior of the global system, including the detailed local mechanisms.Modeling the whole structural system in every detail, using very fine meshes, is one effective approach to simulate the responses of structural components.However, the resulting models become enormous and are difficult to process using current computational power.
High-performance computing appears to be the key to problems that are computationally intensive, such as in realistic simulations of structural engineering systems.At the dawn of parallel computing, shared memory machines dominated.In this kind of hardware architecture, the communication between processes was irrelevant.Thus, very few modifications of the finite element analysis algorithms have been necessitated by the computational hardware.As distributed memory computers and clusters of networked workstations were introduced, communication times between processes became significant.This brought about innovations in the finite element analysis, using domain decomposition algorithms, by dividing tasks into a few loosely coupled subtasks in order to minimize the communication penalty.These well-known domain decomposition methods include sub-structuring (static condensation, dynamic reduction), the parallel central difference algorithm by [1], the Iterative Group Implicit (IGI) algorithm by [2], Finite Element Tearing and Interconnecting (FETI) by [3], and the nonlinear substructuring algorithm by [4].
Presently, grid computing is being perceived as the most promising avenue to achieve further computational power.Virtually an infinite number of machines can be connected via the internet, theoretically leading to unlimited computational capabilities.However, unless each subtask is highly independent, internet communication can render any possible gain in efficiency from a grid-based distributed application.Depending on the time of the day, the traffic on the internet can be such that no timely communication is possible.In addition, for all the above domain decomposition methods, there is a limitation to the number of processors that can cooperate efficiently.Therefore, directly extending those methods to run on a platform consisting of massive machines connected by comparatively slow, over-utilized communication channels to efficiently perform a complex simulation is not a trivial task.New methodologies are now necessary to minimize the quantity and the frequency of data communication.
Since decomposing a complex model and distributing the decomposed tasks to all available machines will not work in a grid-based computing environment due to the internet-imposed communication time obstacle, a new grid-based simulation method for the realistic simulation of structural engineering systems is presented.Two levels of parallel processing will be involved in this framework: (1) multiple locally distributed computing Many researchers have developed simplified models that can generally achieve the goal for various structural components, such as the beam component by [5], and beam-column component by [6].However, in some occasions, the original simplified component model may become invalid to represent the behavior of its corresponding rigorous component model.For this reason, a rigorous model must check the state of its corresponding simplified model periodically, and calibrate or update it when necessary.For illustration, the modeling of a structural component by coupling of a simplified component model (SCM) and a rigorous component model (RCM) is shown in Fig. 1.These two separated numerical columns, which are modeled on two scales, actually represent the same structural columns in the real structural system.The integration of these scaled models according to their respective frames of reference is shown conceptually in Fig. 2. As the figure indicates, an SCM resides in a macro/global system to obtain the actions applied by the rest of the system.An RCM is analyzed in isolation to obtain its detailed responses and behavior.The details of the synchronization of these two models will be discussed in "the multi-scale hierarchical modeling and simulation" section of this paper.
Since the synchronic strategy of the multi-scale models of various scales has been developed, a method to systematically perform these simulation tasks in the proposed grid-base computing environment is introduced.These correlated multi-scale simulation tasks are distributed among clusters connected together by the internet to form a multi-level modeling hierarchy.These simulations, in separated clusters, coordinate with each other through the internet to complete a realistic simulation of a whole structural system.This paper also presents a software framework for supporting the proposed realistic simulation approach in a grid-based, cluster-to-cluster distributed computing environment.The architectural design of the program allows the integration of several multi-scale models as clients and servers under a single platform.Such integration will facilitate a more realistic simulation of a structural system.

II. THEORY
The present enterprise, using grid-based technology to produce a realistic simulation of a large-scale structural system, has one critical weakness, which is the low efficiency and reliability of internet communication.To solve this problem, the cluster-to-cluster computing environment and the multi-level hierarchical modeling and simulation method are proposed and combined in this study to form a grid-based framework for large-scale structural simulation.

A. The Cluster-to-Cluster Distributed Computing Environment
A simple one-level grid computing environment can be described as a massive collection of heterogeneous machines connected by comparatively slow, overutilized communication channels.The computer applications running in this kind of environment must assume that any communication with other machines is done over the internet, no matter whether the machines are local or remote.Thus, this kind of framework is only suitable for distributed computing problems for which each subtask is more or less independent, such that the time required for internet communication among massive machines is thereby rendered insignificant.Here, please note that for existing distributed finite element analysis algorithms, the communication time is significant and there is a limitation to the number of processors which can yield an efficient computational process.Therefore, there is no way to extend existing algorithms to be applicable in a simple one-level grid computing environment.
The internet communication bottleneck can be significantly bypassed by organizing the hardware configuration of a grid-based environment into a clusterto-cluster distributed computing framework, as shown conceptually in Fig. 3.This framework involves two levels of parallel processing: (1) multiple locally distributed computing environments, which are connected by local network to form a (2) grid-based cluster-to-cluster distributed computing environment.In fact, each of the clusters in the framework could be comprised of a number of different kinds of processing equipment, including, but not limited to, a distributed memory supercomputer, a shared memory supercomputer, or just a personal computer.The bulk of communication takes place between the computing nodes within a cluster, similar to a traditional cluster computing environment.Only those messages to be exchanged between clusters are required to communicate between the interface computers on the internet.

B. The Multi-Scale Hierarchical Modeling and Simulation
In simulating structural engineering systems, finite elements analyses are often used to represent the behavior of the system as a whole.In this type of analysis, simplistic nonlinear models, which consist of one or a few nonlinear elements, can be used for representing the global behaviors of various structural components, such as beams, columns, walls, and connections.Although the global responses of structural systems can be simulated in this way, the detailed responses of their components cannot be obtained using simplistic models.On the other hand, much research has been conducted on the analysis of individual structural components.In this type of analysis, the structural components are modeled using very fine meshes to produce detailed responses.However, these models are isolated without consideration of the relationship between their behaviors and those of the rest of the system.Therefore, these two levels of knowledge should be integrated under a single platform to produce a more realistic simulation of structural engineering systems.
Modeling the whole structural system in every detail using very fine meshes as the way to simulate the responses of structural components is one approach.However, the resulting models would be enormous, awkward and inefficient using current computational power and analysis techniques.Another possible simulation approach for grid computing environment is proposed in this study.The proposed approach uses independent models of various scales for simulating the global system of a structure and its detailed components.In the beginning, a structural system is analyzed in the ways traditional structural analysis is conducted.Once done, the simulations of its multi-scale component models are integrated and correlated with the analysis result of the structural system to achieve area listic simulation of the whole structure.In this way, a detailed simulation of the whole system can be decomposed into several detailed simulations of its individual components.
Fig. 4 shows that the proposed multi-scale hierarchical modeling and simulation method can fit into the proposed cluster-to-cluster computing environment nicely.Here, a global structural and a beam as well as a column component are both separately modeled at two scales.For example, the beam component of the SCM resides in the global structural system, analyzed by Cluster #1, and then it applies the nodal displacement obtained from the rest of the system.Through the global grid-based network, Cluster #3 receives the effects of the global system on the beam component of the SCM, and these effects are then applied to the beam of the RCM, analyzed by Cluster #3, to obtain the detailed responses and behaviors.A study for verifying the accuracy and demonstrating the applications of the proposed multi-scale hierarchical modeling and simulation method has been presented by [7].In their research, the two-level model consists of a global frame model in line elements and a detailed component model in plane stress elements which references to the beam portion of the global frame model.It is then analyzed as the RCM using the proposed simulation method for detailed response.The results indicate that the peak and average percentage of error of strain in the beam component are less than 1%.With this level of accuracy, the proposed simulation method should be appropriate to be used for simulating the responses of real structural systems.On the other hand, the comparison of RCM simulation with the real experimental results shows that the RCM predicts the deformed shape and stress results as they are observed in the experimental test.

C. The Integration of Commercial Analysis Software
There are many popular commercial structural analysis software products available, such as ETABS [8] and ABAQUS [9].Because of their outstanding performances and the many different features that they offer, many engineers and researchers from various disciplines regularly use them as analyzing tools.In the past decade, with increasing demands for computing power, these software vendors have used the parallelization technology to improve the computational capacity and efficiency of their products.Therefore, considering the benefits of the software analysis performance, and its general familiarity to the public, ETABS and ABAQUS have been chosen for each cluster of the proposed grid-based computing environment.In other words, if a cluster system has the software program installed, it can easily become one of the analysis servers of the proposed platform, simply by installing the serverside program of the core analysis module, which will be introduced in the later section.Therefore, those who use the proposed platform can create new system or component models through the pre-process interface of the software.After the model has been analyzed, the post-process interface can then show the visualized results.
Different commercial analysis software programs have their own unique and specific input file format.Therefore, some rules should be established for identifying the objects with the proposed software system, in order to enable the platform to recognize the data format of different analysis software programs in a dynamic and automatic fashion, without needing to modify the code of the system.
There are two interface requirements that need to be established before the commercial analysis software can be integrated into the proposed analysis platform.First, the system should be able to recognize the content of the input file of the commercial analysis software and convert it to the corresponding object types of the proposed software system.These objects types include nodes, elements, forces, boundary conditions, etc.This step is necessary because the global model and the rigorous component models of the whole simulation problem will be displayed on the graphical user interface of this system.The second requirement is that the system should be able to read the output from each of the models studied by the commercial analysis software.Therefore, after the global model is completed, the nodal displacements in the global model can then be extracted from the SCM.These extracted values of the displacements are then translated into the initial boundary conditions of the corresponding RCM.

III. METHOD
The proposed cluster-to-cluster distributed computing environment will require a modified client-server module with a formatted architecture to perform the proposed multi-level hierarchical modeling and simulation.The software framework consists of two modules, as shown in Fig. 5.They are the core analysis module and the graphical user interface module.Both modules are implemented using Java, a platform-independent, net infrastructure language, so that the system can run on any (interface) computer connected to the internet.
The hardware configuration is shown in Fig. 6.In the same way, the hardware configuration is divided into two parts.The graphical user interface module is installed on a personal computer (PC), and the core analysis module is orchestrated by one client cluster and at least one server cluster.As the figure shows, in the core analysis module, each cluster uses an interface computer to communicate, and the analyzing work is thereby administered to its own cluster system.

A. The Core Analysis Module
The core analysis module is designed as a clientserver-based distribution system, and its architecture is shown in Fig. 7.The interactions between the client and server are archived by the Command Channel objects, which are created by the Client Communicator class and Server Communicator class.On the Client side, the Coordinator class has three subclasses to handle the simulation processes.The Deployer class reads the simulation configurations and stores the data of the digital models into the Model Holder class.The Model Holder class stores a global model, simplified component models and rigorous component models.At this point, the stored object of the rigorous component model provides several functions to enable other objects to access their information.The Analysis Task Distributor class submits the analysis tasks to the assigned clusters and waits for their analyzed results to return from the server side.
Basically, a client can distribute the analysis tasks to more than one server.On the other hand, a server is able to receive more then one analysis task at a time.Therefore, on the server side, the RCM Analysis Server class can create several RCM Analysis Task objects, which control the analysis program to analyze the task, and queue specific tasks for execution.The Analyzer classes, which handle the integrated analysis programs, have a synchronization property allowing them to establish the queuing function of the RCM Analysis Server class.

B. The Graphical User Interface Module
The software architecture of the graphical user interface module is presented in Fig. 8

IV. RESULT AND DISCUSSION
The simulation of a simple two-level hierarchical model of a frame structure is presented to demonstrate the proposed system.As shown in Fig. 9, the global structural model is a 4-story structural system comprised of beams, columns and beam-column joint elements.The lateral loading representing earthquake loading is applied to the structural system.Two structural components have been selected to be analyzed in detail using the rigorous models.
For the hardware configurations, there are three clusters and one personal computer participating in the simulation.As shown in Fig. 10, the overall computational abilities and the assigned rigorous component models are shown beside these clusters.The state of each cluster is also monitored by the client in order to provide information to the user on the progress of each process, and also to reveal the source of any bottleneck in the process that might occur.For the analysis tools, first, the global model is analyzed using ETABS software at cluster A. This cluster is equipped with two computers, all having a 2.4 GHz Intel processor and a 2GB memory.Then, the column component, located on the second floor, is analyzed as RCM using ABAQUS software at cluster B. Again, this cluster is equipped with four computers, each having a 2.4 GHz Intel processor and a 2GB memory.Finally, the beam component, located on the second floor, is also picked and analyzed as RCM using ABAQUS software at cluster C. Again, this cluster is also equipped with four computers, with 2.4 GHz Intel processor and 2GB memory.
The nodal displacements of the simplified model in the global system are extracted and translated into the nodal displacements on the corresponding planes of the rigorous model, shown as known boundary conditions, to obtain the detailed responses of each component.The simulated results of this two-level hierarchical frame model, using the proposed system, are shown in Fig. 11.
This simulation using the proposed cluster-to-cluster grid computing environment took 18 seconds to complete.For comparison, the same simulation is also run using a single computer, and it took 39 seconds to complete.As can be seen, the simulation time can be reduced by using the proposed framework.However, the main goal of this study is not to maximize the speedup performance.Instead, the goal is to integrate the available computing resources which are geographically distributed by grid technology.Since the computing units in the proposed computing environment are expected to be highly heterogeneous, the distributed computational tasks of the proposed simulation method are also expected to be unbalanced.Therefore, the traditional speedup plot for evaluating the performance of parallel processing was not employed and not to be presented for comparison and presented for comparison.

V. CONCLUSIONS
This paper has presented a grid-computing framework for simulation in multi-scale structural analysis by integrating a cluster-to-cluster distributed computing environment.The purpose of this study was to utilize the idle and available computational resources on the internet for providing the computing power needed for processing large-scale structural simulations.However, slow internet communication is expected to be a significant bottleneck.To solve this internet-imposed obstacle, the grid computing environment is first organized as a two-level parallel platform, which first utilizes local cluster computing, and then remote, clusterto-cluster computing.A hierarchical modeling approach and computational procedures for the proposed clusterto-cluster computing environment have been added to streamline the process and avoid excessive internet communication.To fulfill the proposed concept, a prototype software system has been designed and implemented to perform the proposed multi-scale modeling and simulation in a cluster-to-cluster distributed computing environment.Additionally, the simulation time can be reduced by using the proposed framework.However, the main goal of this study is not to maximize the speedup performance.Instead, the goal is to integrate the available computing resources which are geographically distributed by grid technology.
. The Visual Manager class has a Control Tree object.And it can create many Model Viewer objects depending on how many models need to be displayed.The Control Tree class accesses the Deployer class sub-program in the Coordinator class of the core analysis module.It obtains the simulation configurations stored in the Deployer class and shows them on the management window.The Model Viewer class uses the Model Drawer class, the Model Builder class and the Scene Handler class to create and interactively manipulate the visualized objects of the model.