Procedures for Verification and Validation of Computer Simulations Used for Roadside Safety Applications (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

viii LIST OF TABLES Table 1. Report 350 evaluation criteria for test 3-31 on the Plastic Safety System CrashGard Sand Barrel System – test (left) and simulation (right). (26) ................................................ 24 Table 2. EN 1317 evaluation criteria for test 3-31 on the Plastic Safety System CrashGard Sand Barrel System – test (left) and simulation (right). (26) ................................................ 26 Table 3. Results of a ROBUST round robin crash test activity involving a 900-kg car striking a vertical concrete wall. (43) ................................................................................... 50 Table 4. Comparison of the six tests for the Ford Festiva rigid pole test (35) ..................................... 51 Table 5. Metric componentsfor the three simulation curve of Figure 10 (32) ..................................... 53 Table 6. Qualitative comparison of simulation results to full-scale test results (75) ........................... 74 Table 7. EN 1317 domain-specific metric comparisons of a simulation and crash test of a ............... small car striking a concrete wall at 20 degrees and 100 km/hr. (78) .................................. 76 Table 8. NARD time-domain metrics comparing a 100 km/hr, 25 degrees impact between a ............. pickup truck and the weak-post w-beam guardrail. (95) ...................................................... 83 Table 9. Domain-specific TRAP metrics for TTI test 471470-26 and LSDYNA simulations of two strong-post w-beam guardrails. (11) .............................................................................. 87 Table 10. NARD and ANOVA metrics for TTI test 471470-26 and LSDYNA simulations of two strong-post w-beam guardrails. (11) ................................................................................. 88 Table 11. Summary of methods used to validate models in roadside safety publications.................... 95 Table 12. Average shell element dimensions used in simulation studies ........................................... 107 Table 13. Average solid element dimensions used in simulation studies ........................................... 107 Table 14. Definition of MPC metrics ................................................................................................ 134 Table 15. Definition of single-value metrics ..................................................................................... 135 Table 16. Comparison metrics for the analytical curves for (1) the magnitude test and (2) the phase test. ............................................................................................................ 139 Table 17. Comparison metrics for Set #1, Set #2 and the combination of both sets ......................... 146 Table 18. Values of the comparison metrics using velocity time histories for Set #1. ...................... 149 Table 19. Comparison metrics for four essentially identical crash tests of a strong-post w-beam guardrail with different blockouts. .................................................... 152 Table 20. Phenomena Importance Ranking Table (PIRT) for the G4(1S). ....................................... 158 Table 21. Partial PIRT for the NCAC C2500R pickup truck. (83) ................................................... 160 Table 22. Analysis solution verification table for test case 1. ........................................................... 163 Table 23. Roadside safety validation metrics rating table for test case 1– (single channel). ............ 166 Table 24. Roadside safety validation metrics rating table for test case 1–(multi-channel,Area II). .. 171 Table 25. Roadside safety validation metrics rating table for test case 1– (multi-channel, Inertial) . 172 Table 26. Evaluation criteria test applicability table for test case 1. ................................................. 174 Table 27. Structural adequacy phenomena for test case 1. ................................................................ 175 Table 28. Occupant risk phenomena for test case 1........................................................................... 176 Table 29. Vehicle trajectory phenomena for test case 1. ................................................................... 177 Table 30. Summary of the test and simulation impact conditions for Case 2. .................................. 180

ix Table 31. Summary of Global Energy Checks for Case 2. ................................................................ 180 Table 32. Roadside safety validation metrics rating table for Case 2– (single channel option). ....... 185 Table 33. Roadside safety validation metrics rating table for Case 2 – (multi-channel, Area II). .... 187 Table 34. Roadside safety validation metrics rating table for Case 2 – (multi-channel, Inertial). ... 191 Table 35. Structural Adequacy Phenomena for Case 2. .................................................................... 192 Table 36. Occupant Risk Phenomena for Case 2. .............................................................................. 193 Table 37. Vehicle Trajectory Phenomena for Case 2. ....................................................................... 195 Table 38. Partial PIRT for the Geo-Metro model. (43) ..................................................................... 129 Table 39. Vehicle type and impact conditions for the two tests in Case 3. ....................................... 200 Table 40. Analysis Solution Verification Table for Case 3. .............................................................. 202 Table 41. Roadside safety validation metrics rating table for Case 3 (single channel option). ......... 209 Table 42. Roadside safety validation metrics rating table for the Case 3 (multi-channel option). .... 212 Table 43. Structural Adequacy Phenomena for Case 3 with Test 1. ................................................. 214 Table 44. Occupant Risk Phenomena for the Case 3 with Test 1. ..................................................... 215 Table 45. Vehicle Trajectory Phenomena for the Case 3 with Test 1. .............................................. 216 Table 46. Phenomenon Importance Ranking Table for Tractor-Semitrailer Model (Case 4). .......... 218 Table 47. Metric Evaluation Table for Leaf Spring Response (Case 3 Vehicle PIRT). .................... 221 Table 48. Summary of the test and simulation impact conditions for Case 4. .................................. 222 Table 49. Summary of Global Energy Checks for Case 4. ................................................................ 227 Table 50. Roadside safety validation metrics rating table for Case 4 – (single channel option). ...... 229 Table 51. Roadside safety validation metrics rating table for the Case 4 – (multi-channel option). . 232 Table 52. Evaluation Criteria Test Applicability Table for Case 4. .................................................. 233 Table 53. Structural Adequacy Phenomena for Case 4. .................................................................... 235 Table 54. Occupant Risk Phenomena for Case 4. .............................................................................. 236 Table 55. Comparison of Phenomenological Events for Case 4. ...................................................... 237

1 CHAPTER 1 INTRODUCTION THE FINITE ELEMENT METHOD Obtaining accurate and reliable finite element simulations requires careful attention to detail and careful verification of material properties, energy management, numerical stability and a number of other important computational characteristics. Confidence in the results of computations depends on careful verification and validation. The purpose of this research is to develop potential verification and validation procedures, quantifiable evaluation metrics and acceptance criteria for roadside safety research that maximize the accuracy and utility of using finite element simulations in much the same way NCHRP Report 350 and MASH ensure the accuracy and utility of full-scale crash tests. Developing procedures and evaluation metrics allows for results to be quickly checked and compared with other simulations. Before examining the details of verification and validation, it is useful to first review the development of the finite element method in general and, in particular, its application to roadside safety. The foundations for the finite element method were laid in the 1940s but it was the development of the digital computer in the 1950s that transformed a collection of obscure numerical methods into a powerful engineering design tool. Finite element methods and computers evolved together in a symbiotic relationship that began in earnest in the middle 1950s with pioneering work by engineers working for Boeing.(1) In a very real sense, applications and the needs of practicing designers have always been the fuel that ignited innovation in structural finite element software development. As is often the case, there were similar independent advances in Europe as, for example, by Argyris.(2) Many researchers followed the lead of these early analysts and the general finite element literature is now, of course, vast and highly differentiated. The Boeing group included Ray Clough, a civil engineering professor from the University of California at Berkeley, who brought the innovations and ideas from Boeing back to Berkeley. Berkeley became a center of research and a training ground for many researchers and academics who would make important contributions to finite element analysis.(3) Researchers such as Clough and Wilson mentored, trained and worked with a generation of researchers who took the finite element method with them as they took up teaching and research posts throughout the world. Belytschko points out that in addition to their ground breaking work in finite element analysis, the willingness of researchers like Clough and Wilson to freely and widely distribute their early finite element codes played an important role in bringing the method to a wider audience.(4) These first applications generally involved linear static solutions and it took nearly a decade before researchers began exploring solutions to nonlinear problems.(5) The names of the researchers and the codes they developed during this period are still familiar today: Marcal developed the program MARC, Swanson developed ANSYS, Hibbitt developed ABAQUS and Bathe developed ADINA.

2 While the codes and researchers mentioned above were exploring nonlinear materials and geometric nonlinearities, they were primarily concerned with static or steady-state dynamic solutions. None of these codes, at least at the time, used explicit time integration methods in their solution procedures and none really addressed the general contact and impact problems needed to model vehicle crashes. Interestingly, some insightful engineers recognized the potential for using finite element method to simulate vehicle impacts early in the 1970s. , Belytschko describes some of the very early explicit impact codes such as SAMSON and WRECKER and notes the great deal of work sponsored by the US Department of Transportation.(4) In particular, the USDOT was interested in design tools for automobiles and roadside barriers and was funding finite element development work. An often overlooked researcher from Berkeley who developed a very early explicit two- dimensional code for solving roadside barrier problems was Powell who developed the program BARRIER in 1971.(6) Powell’s program was relatively simple, using simple discreet elements such as springs, dash-pots, beams and truss elements to model components of guardrails and vehicles. The results proved so useful that the final version of the program, BARRIER VII, is still sometimes used in the roadside safety design community for quick calculations of barrier deformations.(6) Giavotto developed similar mixed-method codes called VEDYAC and later MUSIAC for evaluating roadside hardware designs.(7) The specialty codes GUARD and NARD were also developed during this time period but never really gained the confidence of roadside safety researchers. Significantly, however, NARD included a validation manual, which appears to be the first attempt of the roadside safety community to incorporate validation and verification techniques into computer-aided roadside design. These are probably just a few of what might be called specialty codes that were developed in the 1970s under DOT sponsorship to solve specific automotive and highway safety design problems. In the mid-1970s, the DOT stopped funding numerical methods and shifted more resources into full-scale testing both in the area of vehicle design (i.e., the National Highway Traffic Safety Administration (NHTSA)) and in barrier design (i.e., the Federal Highway Administration (FHWA)). It would be some twenty years before DOT came back to the use of finite element codes. The first true general-purpose impact finite element code was developed by Hallquist at Lawrence Livermore National Laboratories (LLNL) starting roughly in 1976.(8) The era of computational impact mechanics is, then, just over 30 years old and practical industrial solutions have only been widespread for little more than a decade. Given the importance and ubiquity of finite element calculations in the design of complex structures such as automobiles, aircraft, and occupant safety systems, it is sometimes difficult to remember that all these methods and tools have reached the commercial market place in a very short period. The combination of rapidly improving numerical methods coupled with an equally dramatic

3 increase in inexpensive computational power brought impact finite element codes out of the government research laboratories and into engineering design studios for a once unimaginable range of applications. Hallquist named his code DYNA3D, an acronym indicating nonlinear dynamic analysis of solids in three dimensions. DYNA3D resulted from the convergence of three ideas that Hallquist combined uniquely: • The use of explicit central difference time integration, • The use of under-integrated elements, and • The calculation of contact forces. Much of the FE methods research at the time involved developing more sophisticated elements and integration schemes. DYNA3D adopted nearly the opposite approach – very fine spacial discreetization and small time steps with a relatively straight-forward integration scheme. DYNA3D demonstrated that it would be possible to develop a general purpose three dimensional continuum mechanics contact-impact code using this approach. As Wilson freely distributed SAP a decade earlier, LLNL made DYNA3D available, with some security restrictions, to academics and engineers who in turn added features and improvements. The result was a code that was increasingly seen as a viable general-purpose code for dynamic impact and contact problems. All subsequent crashworthiness codes derive either directly or indirectly from the early LLNL codes. FINITE ELEMENT METHOD IN ROADSIDE SAFETY As discussed above, finite element computer simulations have been used in roadside safety research and design from a surprisingly early date. Programs like Barrier VII, NARD and Guard were used to gain insight into roadside hardware crashes at a time when finite element analysis in general was in its early development.(9) These early tools, however, had significant limitations and could really only be used to perform simple parametric evaluations that would later be more extensively explored using full-scale crash tests. In the early 1990s, the FHWA sponsored several projects to examine finite element simulation in roadside safety. A consensus eventually developed to focus on the use of DYNA3D and its commercial counterpart LSDYNA. By the early 1990s, LSDYNA had become an important design and evaluation tool in the automotive and aerospace industries for evaluating crash scenarios. Its importance is shown by the fact that each Transportation Research Board (TRB) Annual Meeting in the past several years has had one full session devoted to papers involving LSDYNA simulations in roadside safety. Numerous papers are also being published in a variety of journals. LSDYNA has become a “main stream” tool in roadside safety design in a relatively short period of time. Verification and validation have been a component of these developments from the very start. The NARD program contained a Validation Manual with a variety of validation metrics which will be discussed later. The first roadside safety researcher to seriously address the issue

4 of finite element verification and repeatability of full-scale crash tests was Ray, who in 1996 published a paper comparing six identical full-scale crash tests and an LSDYNA finite element simulation of the same impact scenario. Unsurprisingly, Ray discovered that the acceleration, velocity and displacement time histories obtained from crash tests were not identical but were subject to random experimental variations. Ray performed a DYNA3D simulation of the same impact scenario and the results generally remained inside the 90th percentile corridor until after the peak response. Considering the period during which this work was performed and the simplicity of the vehicle model, the results were very encouraging. There was an attempt by the FHWA to encourage users of LSDYNA simulation in roadside safety to calculate and report a variety of validation and verification metrics but the community resisted attempts to standardize procedures at the time. Instead, each research group developed its own ad hoc qualitative methods of assessing validity. At about this time, the FHWA established its “Center of Excellence” program to encourage roadside safety researchers to incorporate finite element analysis into roadside hardware design and testing. This effort used a small amount of funding to encourage the Centers to piggy-back finite element simulation efforts onto existing roadside hardware development projects. From that perspective, the program was quite successful; today, it is very common to see finite element simulations being used as an integral part of the roadside safety research and development process. In fact, roadside hardware development efforts that do not include finite element simulations are the exception rather than the rule. Early efforts using LSDYNA focused on replicating the results of crash tests that had already been performed but it was not long before simulation was used to predict the likely outcome of crash tests. Perhaps the first use of LSDYNA to explore roadside hardware performance and obtain an FHWA acceptance letter based only on LSDYNA analyses was performed by Plaxico and Ray for the Iowa DOT.(10) Iowa had used a much larger wood post than was typical and sought to obtain FHWA approval of its design. Instead of sponsoring crash tests, Iowa DOT asked Plaxico and Ray to evaluate the safety performance of the G4(2W) and the G4(1W) using finite element analysis. Since there were several crash tests available for the G4(2W), the research team first simulated those crash tests to verify that the finite element models produced accurate and reliable results. After this was established, the performance of the similar G4(1W) was explored with an LSDYNA model that was constructed in a similar way and used many of the same components (e.g., same vehicle model, same material models, and similar mesh densities). The simulation results for the never-tested G4(1W) showed that it was very likely to perform well in a Test 3-11 crash test. The results were forwarded to the FHWA which was confident enough in the results to issue an acceptance letter (FHWA).We believe that this was the first instance of an FHWA approval letter being written solely based on the results of finite element simulation. This process has accelerated such that there are now numerous examples of LSDYNA

5 analyses being used in the roadside safety design process. Many of these projects will be described later in the literature review section. Many researchers in Europe were also using finite element analyses to design roadside hardware and some of the EU countries even recommended accepting designs based only on finite element results. Beginning in 1998, the European Union sponsored the “Road barrier upgrade of standards” (i.e., the ROBUST Project) to improve the scientific and technical knowledge that form the basis of the EN1317 European crash test standards.(11, 43) One of the primary goals of the project is to “improve the accuracy and ease of use of computational mechanics as a complement to full-scale tests.” ROBUST Working Group Six in particular is charged with exploring computational mechanics (i.e., computer simulation programs) in aiding the design, evaluation and approval process. Verification and validation of finite element models is a major portion of the work of the ROBUST consortium. To-date the group has tackled several very important issues relating both to the repeatability of full-scale crash tests and the validation of finite element models. The ROBUST project has produced a number of interesting studies relevant to this research including the following: • A verification study of a modified Geo Metro finite element model developed by Anghileri striking a Norwegian standard guardrail, • A verification study to assess further revisions of the Geo Metro model including a finite element dummy and two seats, • A verification study to assess further revision of the Geo Metro model to include steering and suspension effects and • A study on how to best record vehicle acceleration data in finite element simulations so that they can be compared directly to full-scale tests. Generally, the ROBUST validation procedures involve careful documentation of changes from one version of a model to the next and comparison of crash test measures like the ASI, THIV and ORA. An example of typical ROBUST procedures is included in the validation report for the Geo Metro model.(12) One of the interesting features of the ROBUST reports is that a standard method of reporting is evolving such that finding and comparing information is much easier. OTHER AREAS OF COMPUTATIONAL MECHANICS The integration of computer aided engineering techniques into roadside safety design has been paralleled in many other fields including aerospace, medicine and vehicle design. The automotive industry in particular has been a leader in integrating computer aided engineering into the routine design process. Automotive companies are generally under great pressure to

6 develop and test new product lines quickly and efficiently. Codes like LSDYNA have become increasingly sophisticated in large part due to the needs of automotive manufacturers and their suppliers to perform rapid proto-typing of new designs. Computer simulation methods have become so main-stream in recent years, however, that the need for verification and validation procedures has shifted to the public sector as well. Today, several governmental bodies have developed validation and verification procedures in order to evaluate designs from the engineering community. For example, today the Federal Aviation Administration (FAA) allows aircraft seats to be designed and approved for use based solely on finite element analysis.(13) There is also a computation technique approved by the FAA to simulate bird strike impacts with engine turbines. Similarly, the Federal Food and Drug Administration FDA is developing validation procedures for use in evaluating human structural implants (e.g., artificial joints). In-vivo testing of new joint designs is not possible so using computational methods is one of the few ways to evaluate new biomedical devices prior to the start of human trials. The European Standard for rail car crashworthiness also allows for the use of computer simulations in the approval process of new rail-car vehicles.(14) The National Aeronautics and Space Administration (NASA), the Department of Defense (DoD) and many of the national laboratories have developed high-level managerial procedures for validating computer models. These approval procedures will be discussed at greater length in the Literature Review section of this report but they are mentioned here to illustrate that the use of computer simulation methods have become widely used throughout the engineering design community and are being used in the approval and acceptance process by several governmental organizations. SUMMARY There is a long history of using finite element methods to design and evaluate mechanical devices in general and roadside safety hardware in particular. While computer simulations have become widespread the issue of verification and validation has only recently begun to be addressed. The objective of this research is to develop guidelines for verification and validation of detailed finite element analyses for crash simulations of roadside safety features. The focus of these guidelines is establishing accuracy, credibility, and confidence in the results of crash test simulations intended (i) to support policy decisions and (ii) to be used for approval of design modifications to roadside safety devices that were originally approved through full-scale crash testing. The following chapters in this report include an in-depth review of the literature, a “best practices” guide to modeling roadside safety hardware, the research team’s recommended procedures for verifying and validating finite element models used in roadside safety applications, and an assessment of the procedures with the perspective of using the guidelines. The literature review chapter provides a review of the methods that have been used in verification and validation efforts – both in the roadside safety area as well as computational mechanics in general. Gaps in the literature were identified, which were later addressed in the

7 development of the research team’s recommended procedures for verification and validation. Chapter 3 identifies model building best practices in easily retrieved form so that both new and experienced users can develop models that are highly likely to run without errors. The best practices information was garnered from a survey of practitioners that was conducted to determine modeling techniques and the range of acceptable variation when performing typical roadside safety simulations. Chapter 4 presents recommendations for verifying solutions and validating computer simulations in roadside safety, and Chapter 5 presents several example cases to demonstrate the process of applying the procedures.

8