River Rapids Conference Proceedings of the Society for Experimental Mechanics Series Model Validation and Uncertainty Quantification, Volume 3 Robert Barthorpe Proceedings of the 37th IMAC, A Conference and Exposition on Structural Dynamics 2019 River Publishers
Conference Proceedings of the Society for Experimental Mechanics Series Series Editor Kristin B. Zimmerman, Ph.D. Society for Experimental Mechanics, Inc., Bethel, CT, USA
The Conference Proceedings of the Society for Experimental Mechanics present early findings and case studies from a wide range of fundamental and applied work across a broad range of fields that comprise experimental solid mechanics and structural dynamics. This series volume represents a collection of early findings and case studies on fundamental and applied aspects of Model Validation and Uncertainty Quantification.
River Publishers Model Validation and Uncertainty Quantification, Volume 3 Proceedings of the 37th IMAC, A Conference and Exposition on Structural Dynamics 2019 Robert Barthorpe Editor
Published, sold and distributed by: River Publishers Broagervej 10 9260 Gistrup Denmark www.riverpublishers.com ISBN 978-87-7004-984-9 (eBook) Conference Proceedings of the Society for Experimental Mechanics An imprint of River Publishers © The Society for Experimental Mechanics, Inc. 2020 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, or reproduction in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Preface Model Validation and Uncertainty Quantification represents one of eight volumes of technical papers presented at the 37th IMAC, A Conference and Exposition on Structural Dynamics, organized by the Society for Experimental Mechanics and held in Orlando, Florida, January 28–31, 2019. The full proceedings also include volumes on Nonlinear Structures & Systems; Dynamics of Civil Structures; Dynamics of Coupled Structures; Special Topics in Structural Dynamics & Experimental Techniques; Rotating Machinery, Optical Methods & Scanning LDV Methods; Sensors and Instrumentation, Aircraft/Aerospace, Energy Harvesting & Dynamic Environments Testing; and Topics in Modal Analysis & Testing. Each collection presents early findings from experimental and computational investigations on an important area within structural dynamics. Model Validation and Uncertainty Quantification (MVUQ) is one of these areas. Modeling and simulation are routinely implemented to predict the behavior of complex dynamical systems. These tools powerfully unite theoretical foundations, numerical models, and experimental data, which include associated uncertainties and errors. The field of MVUQ research entails the development of methods and metrics to test model prediction accuracy and robustness while considering all relevant sources of uncertainties and errors through systematic comparisons against experimental observations. The organizers would like to thank the authors, presenters, session organizers, and session chairs for their participation in this track. Sheffield, UK Robert Barthorpe v
Contents 1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry..................................................................................................... 1 M. Martarelli, P. Castellini, and A. Annessi 2 The Need for Credibility Guidance for Analyses Quantifying Margin and Uncertainty........................ 11 Benjamin B. Schroeder, Lauren Hund, and Robert S. Kittinger 3 Failure Behaviour of Composites Under Both Vibration Loading and Environmental Conditions............ 25 Georgios Voudouris, Dario Di Maio, and Ibrahim Sever 4 Verification and Validation for a Finite Element Model of a Hyperloop Pod Space Frame..................... 33 Vignesh Jayakumar, T. S. Indraneel, Rohan Chawla, Sudeshna Mohanty, Shishir Shetty, Dhaval Shiyani, and Shabaan Abdallah 5 Investigating Nonlinearities in a Demo Aircraft Structure Under Sine Excitation............................... 41 S. B. Cooper, S. Manzato, A. Borzacchiello, L. Bregant, and B. Peeters 6 Sensor Placement for Multi-Fidelity Dynamics Model Calibration ............................................... 59 G. N. Absi and S. Mahadevan 7 Application of Cumulative Prospect Theory to Optimal Inspection Decision-Making for Ship Structures... 65 Changqing Gong, Dan M. Frangopol, and Minghui Cheng 8 Establishing an RMS von Mises Stress Error Bound for Random Vibration Analysis.......................... 75 David Day, Moheimin Khan, Michael Ross, and Brian Stevens 9 A Neural Network Surrogate Model for Structural Health Monitoring of Miter Gates in Navigation Locks ..................................................................................................... 93 Manuel Vega, Ramin Madarshahian, and Michael D. Todd 10 Model Validation Strategy and Estimation of Response Uncertainty for a Bolted Structure with Model-Form Errors ............................................................................................... 99 Huijie Li, Qintao Guo, Ming Zhan, and Yanhe Tao 11 Characteristic Analysis of Modified Dolly Test: A Sensitivity Study of Initial Conditions on Rollover Outcomes ................................................................................................................. 107 Mohammad Reza Seyedi, Sungmoon Jung, and Jerzy Wekezer 12 Input Estimation of a Full-Scale Concrete Frame Structure with Experimental Measurements .............. 117 Xi Liu and Yang Wang 13 Bayesian Estimation of Acoustic Emission Arrival Times for Source Localization.............................. 127 Ramin Madarshahian, Paul Ziehl, and Michael D. Todd 14 Quantification and Evaluation of Parameter and Model Uncertainty for Passive and Active Vibration Isolation................................................................................................................... 135 Jonathan Lenz and Roland Platz vii
viii Contents 15 Bayesian Model Updating of a Five-Story Building Using Zero-Variance Sampling Method .................. 149 Mehdi M. Akhlaghi, Supratik Bose, Peter L. Green, Babak Moaveni, and Andreas Stavridis 16 Input Estimation and Dimension Reduction for Material Models ................................................. 153 Sam Myren, Emilio Herrera, Andrew Shoats, Earl Lawrence, Emily Casleton, D. J. Luscher, and Saryu Fensin 17 Augmented Sequential Bayesian Filtering for Parameter and Modeling Error Estimation of Linear Dynamic Systems ........................................................................................................ 163 Mingming Song, Hamed Ebrahimian, and Babak Moaveni 18 On-Board Monitoring of Rail Roughness via Axle Box Accelerations of Revenue Trains with Uncertain Dynamics..................................................................................................... 167 V. K. Dertimanis, M. Zimmermann, F. Corman, and E. N. Chatzi 19 Bayesian Identification of a Nonlinear Energy Sink Device: Method Comparison .............................. 173 Alana Lund, Shirley J. Dyke, Wei Song, and Ilias Bilionis 20 Calibration of a Large Nonlinear Finite Element Model of a Highway Bridge with Many Uncertain Parameters ............................................................................................................... 177 Rodrigo Astroza, Nicolás Barrientos, Yong Li, and Erick Saavedra Flores 21 Deep Unsupervised Learning for Condition Monitoring and Prediction of High Dimensional Data with Application on Windfarm SCADA Data........................................................................ 189 C. Mylonas, I. Abdallah, and E. N. Chatzi 22 Influence of Furniture on the Modal Properties of Wooden Floors................................................ 197 Lars Vabbersgaard Andersen, Christian Frier, Lars Pedersen, and Peter Persson 23 Optimal Sensor Placement for Response Reconstruction in Structural Dynamics .............................. 205 Costas Papadimitriou 24 Finite Element Model Updating Accounting for Modeling Uncertainty .......................................... 211 Rodrigo Astroza, Andres Alessandri, and Joel P. Conte 25 Model-Based Decision Support Methods Applied to the Conservation of Musical Instruments: Application to an Antique Cello ....................................................................................... 223 R. Viala, V. Placet, S. Le Conte, S. Vaiedelich, and S. Cogan 26 Optimal Sensor Placement for Response Predictions Using Local and Global Methods........................ 229 Costas Argyris, Costas Papadimitriou, and Geert Lombaert 27 Incorporating Uncertainty in the Physical Substructure During Hybrid Substructuring ...................... 237 Connor Ligeikis and Richard Christenson 28 Applying Uncertainty Quantification to Structural Systems: Parameter Reduction for Evaluating Model Complexity ....................................................................................................... 241 Robert Locke, Shyla Kupis, Christopher M. Gehb, Roland Platz, and Sez Atamturktur 29 Non-unique Estimates in Material Parameter Identification of Nonlinear FE Models Governed by Multiaxial Material Models Using Unscented Kalman Filtering ............................................... 257 Mukesh Kumar Ramancha, Ramin Madarshahian, Rodrigo Astroza, and Joel P. Conte 30 On Key Technologies for Realising Digital Twins for Structural Dynamics Applications....................... 267 D. J. Wagg, P. Gardner, R. J. Barthorpe, and K. Worden 31 Hygro-mechanical Modelling of Wood and Glutin-based Bond Lines of Wooden Cultural Heritage Objects ......................................................................................................... 273 Michael Kaliske and Daniel Konopka 32 Modelling of Sympathetic String Vibrations in the Clavichord Using a Modal Udwadia-Kalaba Formulation.............................................................................................................. 277 J.-T. Jiolat, J.-L. Le Carrou, J. Antunes, and C. d’Alessandro
Contents ix 33 Modeling and Stochastic Dynamic Analysis of a Piezoelectric Shunted Rotating Beam........................ 281 Zhenguo Zhang, Ningyuan Duan, Jiajin Tian, and Hongxing Hua 34 On Digital Twins, Mirrors and Virtualisations....................................................................... 285 K. Worden, E. J. Cross, P. Gardner, R. J. Barthorpe, and D. J. Wagg 35 Applications of Reduced Order and Surrogate Modeling in Structural Dynamics .............................. 297 Alexandros A. Taflanidis, Jize Zhang, and Dimitris Patsialis
Chapter 1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry M. Martarelli, P. Castellini, and A. Annessi Abstract This paper presents a procedure for the evaluation of the conservation state and the restoration efficiency of nineteenth century camorcanna vaults based on the analysis of objective features extrapolated from nondestructive vibration testing data. As example of application has been chosen the camorcanna vault of the “salone grande” in the nineteenth century Villa Greppi in Monticello Brianza near to Milan, Italy. Non-contact scanning laser Doppler vibrometry has been exploited for the evaluation of the dynamic behavior of the vault before and after rehabilitation. At the first, the structure where frescoes are attached, a cannulated loft spread with mortar, and related aging problem, e.g. painting detachment, are explained. Thus, usual and innovative non- invasive diagnostic techniques are listed, focusing attention on Laser Doppler Vibrometry. Then, Villa Greppi case study is considered, reporting on site equipment and how measurements were taken. Therefore, processed data results are shown, and objective feature indices defined. Keywords Camorcanna · Frescoes · Restoration · Scanning laser Doppler vibrometry · Modal analysis 1.1 Introduction Nowadays, Cultural Heritage (e.g. artwork and historical buildings) di-agnostics and conservation state assessment using various non-contact techniques is more and more conventional. Several techniques have been successfully exploited to evidence typical defects in artworks, as delamination, detachment of frescoes, wooden icons or mosaics, i.e. Scanning Laser Doppler vibrometry (SLDV [1, 2]), Electronic Speckle Pattern Interferometry (ESPI, [3, 4]), Infrared Thermography (IRT, [5–7]), acoustic and ultrasound imaging techniques ([8, 9]). In the present paper, the conservation state and the restoration efficiency of thin camorcanna vaults made with wooden beams, mats of reeds and mortar is assessed. Studies regarding camorcanna vaults show that differential settlements of light vaults may cause cracking on the lower surface, damaging stuccoes and frescoes. Thus, it is necessary to understand the causes for this to happen and find interventions to prevent cracks [10, 11]. To investigate the conservation state on camorcanna vaults, traditionally restorer use visual inspection and percussion techniques. Nowadays, Non-Destructive Testing (NDT) could be adopted using LDV and IRT [12]. The former seems to be one of the most promising diagnostic techniques, especially due to its nondestructive nature, high spatial resolution and frequency range [13]. It allows to measure vibrational frequency response of the structure in terms of mobility functions, which allow to evidence areas of detachment between frescoes and wooden structure. The latter is capable of identifying and characterizing imperfections over a large area. This can be achieved only if there is a sufficient temperature gradient between the flawed and sound area that is appreciable by the IRT sensor [14]. Also ultrasonic techniques are successful in evaluating camorcanna detachments. Defect localization and dimensions could be estimated [15]. M. Martarelli Faculty of Engineering, Università degli Studi eCAMPUS, Novedrate (CO), Italy P. Castellini ( ) · A. Annessi Industrial Engineering and Mathematical Sciences Department, Università Politecnica delle Marche, Ancona, Italy e-mail: p.castellini@univpm.it © Society for Experimental Mechanics, Inc. 2020 R. Barthorpe (ed.), Model Validation and Uncertainty Quantification, Volume 3, Conference Proceedings of the Society for Experimental Mechanics Series, https://doi.org/10.1007/978-3-030-12075-7_1 1
2 M. Martarelli et al. Fig. 1.1 Carmorcanna vault portion with intertwined reeds [16] 1.2 Camorcanna Vaults Structure In many theaters, churches and historical buildings between the sixteenth and nineteenth century, vaults were made by mats consisting of stitched thin cane and mortar fixed on a wooden framework. This kind of structure is usually called camorcanna. These vaults were cheap, lightweight and easy to built up, thus really popular. They were often decorated with precious frescoes and stuccoes that give them historical, artistic and architectural value. Camorcanna is composted by mats of reeds, constructed linking together thin canes with strings (see Fig. 1.1). Reed mats are fixed on a wooden supporting structure and mortar is spread on them. When it is dry, the surface is ready to be decorated with frescos and stuccoes. Nowadays, many camorcanna vaults are left in deterioration due to many factors linked with constructive system problems or external issues. The former may be errors in ribs sizing, inadequate link between the elements or wood inner defects. The latter are caused by accidental events e.g. subsidence, earthquakes, water infiltration, variation in thermo-hygrometric loads or attacks from fungi and insects [16]. The mortar detachment is one of the usual defect in paintings on described structures due to improper installation. During construction, the mortar has to be quite liquid so it can wrap straws properly. In fact, it as to penetrate the straw layer to create a mechanical connection (by interference and adhesion). If the mortar is too solid when given, the connection between it and the straws is only by adhesion. Hence, the joint is not resistant enough and the painting falls under his own weight. Another usual issue of cannulated vaults is straws detachment from the wooden support they are linked with. It can yield to a coat brake and a consequent fall of a vault portion. Both the indicated faults are not visible from the outside and are detectable only afterwards. 1.3 Case Study: Paintings in Greppi’s Manor Vault Villa Greppi is a nineteenth century manor located in Monticello (LC), Italy. The plan detail of Fig. 1.2a shows the frescoed coffered ceiling hall under study. In Villa Greppi’s case, the camorcanna panels were made as follow: 1. The first structure consists of thick chestnut wooden beams supporting the floor (about 40 by 40 cm in size). 2. Smaller transverse beams (about 8 by 10–15 cm in size), called ribs, are used to link together the latter beams.
1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry 3 Fig. 1.2 Frescoed hall scheme. (a) Plan detail of the frescoed hall. (b) Panels measurement labels scheme 3. Mat of reeds (5–10 mm in diameter) are fixed to ribs lower part by at head nails. 4. The mortar (made by plaster, lime or a mixture of both) is then spread on the straws. A painted panel and one missing the fresco and the mortar are shown in Fig. 1.3. The main hall has an area of 66 m2 and the coffered loft is 5 mhigh. It is divided in 15 painted panels (square or rectangular). A visualization scheme is shown in Fig. 1.2b. Panel names are arranged in a matrix form. 1.4 Testing Equipment and Measurement Set-Up The objective of this study is the assessment of the structural behavior improvement due to restoration process. The damage occurred to some parts of the ceiling were due to detachment of whole panels, and this suggest the need to verify the stiffness of the connection between the frescos structure and the supporting reeds, but also the behavior of the whole ceiling, which movements could stress the panels in a dangerous way.
4 M. Martarelli et al. Fig. 1.3 Villa Greppi’s painted loft. (a) Frescos on camorcanna. (b) Damaged camorcanna without mortar and painting For these reasons, vibration measurements were performed before and after the restoration following two approaches: 1. The vibration response measurement at local level, with a dense spatial sampling on each panel (where the fresco was present), that can be considered a panel level investigation. 2. The vibration response measurement at global level, for the entire vault where one measurement point for each panel was taken, that can be considered a ceiling level investigation. The vibration response of the structure was measured in both cases in terms of velocity by means of an LDV that gives mobility Frequency Response Function (FRF). The measurement was performed on the frescoes surface in the vertical direction with a frequency range of 0–512 Hz and a frequency resolution of 312.5 mHz. Measurement points were selected to highlight the local and the global behavior respectively. The panel analysis was performed on all ceiling coffers except to the number 11, 12 and 51, see scheme in Fig. 1.2b, since the frescoes were missing on those panels. Each coffer was forced into vibration by means of an electrodynamic shaker operating as an inertial vibrodyne. As shown in Fig. 1.4b, the shaker was leaning on the vault structural beam of the ceiling and it was pushing an inertial mass of 500 g. The reaction force was acting as excitation of the structure. A load cell was measuring the force applied on the mass, which corresponded to the sum of dynamic forces applied, as a reaction force, by the shaker on the beam. The panel vibration response was measured by the LDV pointing on a regular grid of points on the fresco surface. A grid of 6 ×7 points was sat for the square panel and 8 ×7 points grid for rectangular one. The measured data set is made of FRF functions, obtained as ratio between the vibration velocity response of the panel and the force imparted in input by the electrodynamic shaker. Each FRF is weighted by the transmission path from the shaker to each measurement point and indicates how the energy travels through the different supporting elements (beams, panel, fresco). Therefore, diagnostics is possible: loss of signal amplitude could be due to a structural problem, e.g. fresco detachment. The global behavior of the ceiling has been estimated by performing an additional test. Excitation was provided by an instrumented hammer on the ceiling beam while response was measured by LDV on the central point of each panel. The vibration velocity was measured by the LDV that was sequentially moved at the central position of all the panels.
1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry 5 Fig. 1.4 Test setup. (a) LDV during measurements. (b) Shaker position during measurements Table 1.1 Loft mode shapes Frequency (Hz) N. modes Type Before restoration After restoration Mode 1 2nd flexural 10.72 11.53 Mode 2 3rd flexural 12.96 15.00 Mode 3 1st torsional 28.80 32.01 1.5 Experimental FRF Analysis In order to assess the effectiveness of the restoration on the health status of the structure, an analysis of the measured FRFs has been performed at both ceiling and panel level. First, a modal analysis has been done on the FRFs set in order to identify the dynamic behavior of the structure before and after the restoration in terms of resonance frequencies, damping loss factor and mode shapes. Then FRFs measured at each panel have been processed in order to extract quantitative features correlated to the structural modification produced by the restoration. Finally, signal features were extracted from FRFs in order to obtain synthetic set of scores that could assess effectiveness of restoration accomplished. 1.6 Modal Analysis The modal analysis carried out on the impact testing FRFs set allowed estimating the resonance frequencies before and after the restoration at global level (listed in Table 1.1). Structure stiffening and frequency shift could be noticed in Fig. 1.5, particularly in the phase graph. The corresponding mode shapes are shown in Fig. 1.6, only for the configuration before the restoration, the shape of the mode is not experiencing variation after the intervention. The modal analysis carried out on the FRFs computed on panel level dataset allowed to estimate the resonance frequencies before and after the restoration of each panel. The resonance frequencies before and after restoration are reported in Table 1.2, specifically for panel 42. The corresponding mode shapes calculated for the configuration after the restoration are shown in Fig. 1.7. Panel irregularities (concerning materials, structure) and non-linear behavior increase the analyses complexity. The restoration intervention
6 M. Martarelli et al. Fig. 1.5 Comparison between FRFs before (blue curves) and after (red curves) therestoration on a loft representative point Fig. 1.6 Loft mode shapes. (a) Loft 2nd flexural mode. (b) Loft 3rd flexural mode. (c) Loft 1st torsional mode Table 1.2 Single panel mode shape Frequency (Hz) N. modes Type Before restoration After restoration Mode 1 Rigid mode 11.34 12.64 Mode 2 Rigid mode 13.76 15.90 Mode 4 1st flexural 31.84 42.50 Mode 6 2nd flexural 43.28 64.51 added mass and stiffened the panel structure changing the boundary conditions; thus, the disposition of the nodal lines, which previously coincided with possible cracks, has changed. Furthermore, the first panel modes, that looks like rigid body motion, at panel level, correspond to global modes of the loft.
1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry 7 Fig. 1.7 Loft mode shapes. (a) 1st flexural mode. (b) 2nd flexural mode. (c) 12nd flexural mode 1.7 FRF Analysis Modal analysis does not provide correct and stable information due to the marked variability of the structure under test (between different panels and before/after restoration). In order to catch the effects of the restoration on each panel, obtaining a “score scale” related to the efficiency of the intervention, synthetic parameters that could condense all the collected measures were searched. This can be tackled by observing the averaged FRF obtained from the mean of all the FRFs measured on each panel. The averaged FRF for each panel, before and after restauration are reported in Fig. 1.8; it is evident a stiffening effect after the intervention. By observing the average FRF plots it has been deducted that a significate feature representing the effect of the restauration is the stiffening of the structure, i.e. the shift at the highest frequency range of the panel resonances. This frequency shift can be straightforwardly identified by cross-correlating corresponding average FRFs before and after intervention. Crosscorrelation functions are reported in Fig. 1.9; they show a frequency lag (on the x-axis) that is always positive, thus a frequency shift towards high frequencies is present and is consistent for all panels. 1.8 Conclusions In the present paper, camorcanna ceilings consolidation is assessed using LDV. The aim of this work is to method to evaluate the restoration efficiency of frescoed panels (by means of its vibrational characteristics). Villa Greppi’s case study is presented. From the modal analysis of global ceiling and the single coffer, it has been possible to demonstrate that the restauration increases the stiffness of the entire ceiling that is evident from frequency resonance shift in the high range and from mode shapes modification. By analyzing the FRF data sets for each panel this frequency shift is evident at global level for the entire FRF. It was possible therefore to extract a feature representing this stiffening effect, which is the consequence of the restauration. Even if the stiffening is consistent on all the panels (there is an average frequency increase of 14 Hz) it is very variable between one panel to another: it goes from values around 30 Hz to values of about 3 Hz (with a standard deviation of 10 Hz). This demonstrates the inhomogeneity of the intervention: observe the map of the frequency shift according to the arrangement of the coffers, Fig. 1.10.
8 M. Martarelli et al. Fig. 1.8 Averaged FRF for each panel. (a) Averaged FRF panel 12. (b) Averaged FRF panel 13. (c) Averaged FRF panel 22. (d) Averaged FRF panel 23. (e) Averaged FRF panel 31. (f) Averaged FRF panel 32. (g) Averaged FRF panel 33. (h) Averaged FRF panel 41. (i) Averaged FRF panel 42. (j) Averaged FRF panel 43. (k) Averaged FRF panel 52. (l) Averaged FRF panel 53
1 Nondestructive Consolidation Assessment of Historical Camorcanna Ceilings by Scanning Laser Doppler Vibrometry 9 Fig. 1.9 Cross-correlation curves for each panel. (a) Panel 12. (b) Panel 13. (c) Panel 22. (d) Panel 23. (e) Panel 31. (f) Panel 32. (g) Panel 33. (h) Panel 41. (i) Panel 42. (j) Panel 43. (k) Panel 52. (l) Panel 53
10 M. Martarelli et al. Fig. 1.10 Frequency increase map over the coffers disposition References 1. Castellini, P., Revel, G.: Damage detection and characterization by processing of laser vibrometer measurement results: application on composite materials. 3411, 458–468 (1998). https://doi.org/10.1117/12.307732 2. Castellini, P., Esposito, E., Legoux, V., Paone, N., Stefanaggi, M., Tomasini, E.: On field validation of non-invasive laser scanning vibrometer measurement of damaged frescoes: Experiments on large walls artificially aged. J. Cult. Herit. 1(2), S349–S356,. cited By 16 (2000). https://doi.org/10.1016/S1296-2074(00)00145-X 3. Castellini, P., Abaskin, V., Achimova, E.: Portable electronic speckle interferometry device for the damages measurements in veneered wood artworks. J. Cult. Herit. 9(3), 225–233, cited By 16 (2008). https://doi.org/10.1016/j.culher.2008.05.002 4. Sfarra, S., Ibarra-Castanedo, C., Tortora, M., Arrizza, L., Cerichelli, G., Nardi, I., Maldague, X.: Diagnostics of wall paintings: a smart and reliable approach. J. Cult. Herit. 18, 229–241 (2016). https://doi.org/10.1016/j.culher.2015.07.011 5. Meola, C., Carlomagno, G.M.: Recent advances in the use of infrared thermography. Meas. Sci. Technol. 15(9), R27–R58 (2004). https://doi.org/10.1088/0957-0233/15/9/r01 6. Pucci, M., Cicero, C., Orazi, N., Mercuri, F., Zammit, U., Paoloni, S., Marinelli, M.: Active infrared thermography applied to the study of a painting on paper representing the Chigi’s family tree. Stud. Conserv. 60(2), 88–96 (2013). https://doi.org/10.1179/2047058413y.0000000117 7. Cadelano, G., Bison, P., Bortolin, A., Ferrarini, G., Peron, F., Girotto, M., Volinia, M.: Monitoring of historical frescoes by timed infrared imaging analysis. Opto-Electron. Rev. 23(1), 102–108 (2015). https://doi.org/10.1515/ oere-2015-0012 8. Calicchia, P., Cannelli, G.B.: Detecting and mapping detachments in mural paintings by non-invasive acoustic technique: measurements in antique sites in Rome and Florence. J. Cult. Herit. 6(2), 115–124 (2005). https://doi.org/10.1016/j.culher.2004.11.001 9. Kloiber, M., Reinprecht, L., Hrivnák, J., Tippner, J.: Comparative evaluation of acoustic techniques for detection of damages in historical wood. J. Cult. Herit. 20, 622–631 (2016). https://doi.org/10.1016/ j.culher.2016.02.009 10. Quagliarini, E., Lenci, S., Seri, E.: On the damage of frescoes and stuccoes on the lower surface of historical at suspended light vaults. J. Cult. Herit. 13(3), 293–303 (2012). https://doi.org/10.1016/j.culher.2011.11.008 11. Quagliarini, E., D’Orazio, M., Stazi, A.: Rehabilitation and consolidation of high-value camorcanna vaults with FRP. J. Cult. Herit. 7(1), 13–22 (2006). https://doi.org/10.1016/j.culher.2005.09.002 12. Quagliarini, E., Esposito, E., del Conte, A.: The combined use of IRT and LDV for the investigation of historical thin vaults. J. Cult. Herit. 14(2), 122–128 (2013). https://doi.org/10.1016/j.culher.2012.01.004 13. Martarelli, M., Castellini, P., Quagliarini, E., Seri, E., Lenci, S., Tomasini, E.P.: Nondestructive evaluation of plasters on historical thin vaults by scanning laser Doppler vibrometers. Res. Nondestruct. Eval. 25(4), 218–234 (2014). https://doi.org/10.1080/09349847.2014.896964 14. Tavares, S., Agnani, A., Esposito, E., Feligiotti, M., Rocchi, S., de Andrade, R.: Comparative study between infrared thermography and laser Doppler vibrometry applied to frescoes diagnostic. In: Proceedings of the 2006 International Conference on Quantitative InfraRed Thermography, QIRT Council, 2006. https://doi.org/10.21611/qirt.2006.039 15. Quagliarini, E., Revel, G.M., Lenci, S., Seri, E., Cavuto, A., Pandarese, G.: Historical plasters on light thin vaults: state of conservation assessment by a hybrid ultrasonic method. J. Cult. Herit. 15(2), 104–111 (2014). https://doi.org/10.1016/j.culher.2013.04.008 16. Quagliarini, M.D.E.: Recupero e Conservazione di volte in “Camorcanna”, Alinea Editrice, 2005
Chapter 2 The Need for Credibility Guidance for Analyses Quantifying Margin and Uncertainty Benjamin B. Schroeder, Lauren Hund, and Robert S. Kittinger Abstract Current quantification of margin and uncertainty (QMU) guidance lacks a consistent framework for communicating the credibility of analysis results. Recent efforts at providing QMU guidance have pushed for broadening the analyses supporting QMU results beyond extrapolative statistical models to include a more holistic picture of risk, including information garnered from both experimental campaigns and computational simulations. Credibility guidance would assist in the consideration of belief-based aspects of an analysis. Such guidance exists for presenting computational simulationbased analyses and is under development for the integration of experimental data into computational simulations (calibration or validation), but is absent for the ultimate QMU product resulting from experimental or computational analyses. A QMU credibility assessment framework comprised of five elements is proposed: requirement definitions and quantity of interest selection, data quality, model uncertainty, calibration/parameter estimation, and validation. Through considering and reporting on these elements during a QMU analysis, the decision-maker will receive a more complete description of the analysis and be better positioned to understand the risks involved with using the analysis to support a decision. A molten salt battery application is used to demonstrate the proposed QMU credibility framework. Keywords Credibility · Margin · Uncertainty · QMU · Guidance 2.1 Introduction The purpose of this paper is to describe the need for credibility guidance in quantification of margins and uncertainty (QMU) analyses and provide a potential structure for such guidance. Credibility is defined as “the quality or power of inspiring belief” [1], so credibility guidance should assist in the consideration of belief-based aspects of an analysis. A QMU credibility assessment framework comprised of five elements is proposed: requirement definitions and quantity of interest (QoI) selection, data quality, model uncertainty, calibration/parameter estimation, and validation. Through considering and reporting relevant aspects of these elements during a QMU analysis, the decision-maker will receive a more complete description of the analysis and be better positioned to understand the risks involved with using the analysis to support a decision. This paper will be structured as follows. The remained of this section will provide a history of QMU, motivation for why a credibility assessment framework is needed, and highlight similar efforts in the CompSim domain. Next will be a section outlining the proposed framework for gathering and organizing QMU credibility evidence. How to use the evidence to evaluate analysis credibility is then discussed. A demonstration of the process applied to a molten salt battery example problem is provided in the next section. Lastly, a summary of the paper is provided. 2.1.1 What Is QMU QMU originated at the national laboratories in the early 2000s to address risk in nuclear weapon stockpile stewardship in the absence of full system testing [2]. QMU was originally posed as a risk assessment framework for nuclear weapons, addressing the three elements of the risk triplet (what can occur? how likely is it? and what are the consequences?) [3]; this B. B. Schroeder ( ) · L. Hund · R. S. Kittinger Sandia National Laboratories, Albuquerque, NM, USA e-mail: bbschro@sandia.gov © Society for Experimental Mechanics, Inc. 2020 R. Barthorpe (ed.), Model Validation and Uncertainty Quantification, Volume 3, Conference Proceedings of the Society for Experimental Mechanics Series, https://doi.org/10.1007/978-3-030-12075-7_2 11
12 B. B. Schroeder et al. QMU formulation also included a fourth element, credibility, defined as the answer to the question ‘how much confidence do we have in our risk assessment?’ [4]. Historically at Sandia National Laboratories (Sandia), QMU was largely applied to experimental data-based problems, but it appears likely that an integration of computational simulation (CompSim) results and experimental data will be the paradigm of the future. While processes for conducting QMU have developed over time (e.g., [5, 6]), there are still no formal processes for evaluating the credibility of a QMU analysis. QMU entails comparing a performance measure to a performance requirement to determine the likelihood of functioning as intended, considering all relevant uncertainties. Implementing a QMU analysis requires building a team with the relevant expertise; identifying performance measures and requirements; assimilating relevant data; running an analysis; and communicating the results. Considering these steps of a QMU analysis, a corresponding QMU credibility assessment should address many of the inherent aspects of the analysis such as relevance of the performance measure and requirement, data quality, and analysis limitations. 2.1.2 Why Measure Credibility? There is currently a gap in guidance within Sandia National Laboratories (Sandia) for assessing the credibility of QMU analyses. New guidance for QMU was recently released as internal documents within Sandia in two sections: (1) an overview of high-level QMU concepts and processes and (2) descriptions of statistical tools that can be used to derive QMU results, with a focus on QMU for experimental data. This new guidance pushed for broadening the analyses supporting QMU results beyond extrapolative statistical models and advocated for a more holistic picture of risk, including information garnered from both experimental and CompSim campaigns. Although this new guidance improves the informational basis of QMU analyses, it does not provide a consistent framework for communicating the credibility of analysis results. Credibility assessment guidance for QMU is needed because: • Decision-makers are increasingly asking for credibility assessments when being provided analysis results. Decisionmakers are learning that they must understand the level of confidence they should invest in the results to better utilize the analysis that they commissioned. • Failing to provide guidance for communicating credibility may lead to overconfidence in results. A question that should be posed to QMU analysts is, “What is the credibility of your results?” Without asking this question, the decision-maker may believe results are more reliable than is warranted and make ill-informed decisions. • A unified QMU credibility framework would result in greater consistency in information presentation. When credibility results are analyst-specific and/or analysis-specific, decision-makers will interpret results differently depending on who conducted the analysis. • Streamlined documentation of important auxiliary information (e.g., metadata, methods) is integral to understanding and reproducing QMU results. Summary QMU results (for example, margin over uncertainty ratios) always rely on auxiliary supporting information about the QMU process and supporting experimental data. Without a consistent credibility assessment framework, decision-makers must rely on source credibility, or their belief in the source of the information. Although not specific to the reception of QMU results, psychological research has explored the role of source credibility in other information distribution areas. Across the psychology literature, source credibility is typically attributed to a person providing a message. Key aspects of source credibility include the source’s trustworthiness and expertise [7]; to a lesser degree, composure, dynamism, sociability [8] and even accents of voices [9]. Chaiken and Maheswaran found source credibility can affect decisions in two ways: (1) by serving as a peripheral cue for simple acceptance or rejection of an argument, and (2) by biasing the strength of the decision-maker’s argument processing [10]. While biasing the belief in results based on the source is potentially problematic in itself, Heesacker et al. found that as source credibility increases, persuasion also increases [11]. They attribute this phenomenon to more credible sources eliciting greater thinking about the message (improved information presentation, not informational content). Across psychological research a theme persists: human judgment is persuaded and biased by a variety of minute factors. As humans participate in high stakes decision making, it is important to understand how small changes in presentation of the message (or data) can unintentionally bias the decision-maker. To mitigate such bias, credibility frameworks may help through providing consistency, transparency, and structure.
2 The Need for Credibility Guidance for Analyses Quantifying Margin and Uncertainty 13 2.1.3 History of Credibility in CompSim The concepts of credibility continue to be developed for presenting CompSim results as evidence to support a decision as well as for the incorporation of experimental data into CompSim analyses. Reviewing the progress of credibility guidance for these fields provides a starting point for the analogous guidance for QMU analyses. For institutions that utilize CompSim to support decisions regarding complex engineering questions such as national laboratories, the aerospace and defense industries, and space agencies, the credibility associated with CompSim predictions must be understood. Methods for assessing and communicating the credibility of CompSim based evidence are being developed by many organizations [12]. As an example, the Predictive Capability Model Maturity (PCMM) [13] has been developed at Sandia over the last decade to provide a consistent framework for evaluating CompSim credibility. PCMM was developed as a method of directing discussion about and communication of the many assumptions, errors, biases, and uncertainties ever present in CompSim predictions. A broad spectrum of CompSim activities are covered by elements of PCMM including code verification, physics and material model fidelity, representation and geometric fidelity, solution verification, validation, and uncertainty quantification. Those elements are perceived to encompass the majority of error/uncertainty sources that may impact a CompSim analysis. An approach for grading a simulation’s performance in the different elements is also provided, which includes guidance describing the expected attributes needed to achieve a specific maturity level for each element. This grading is meant to foster gap identification and resource allocation. PCMM can be to be used as a results credibility communication tool as well as an initial analysis planning aid. Applications using PCMM as a prediction credibility assessment tool have been demonstrated [14, 15]. In the CompSim community, experimental credibility is currently being developed from the perspective of using experimental data for model validation and calibration [16, 17]. Through providing structure for the assessment of experiments used for CompSim and experimental integration activities, consistency between modeling activities can be increased. A common difficulty when comparing experimental and CompSim results is ensuring that the scenarios captured by each are similar enough to not be the cause of significant discrepancy. When such discrepancies occur, it may be difficult to determine the source. Through capturing information about the experimental setup from the perspective of how that information will be used in CompSim analyses, more information can be gained from the comparisons. This same framework can be used to increase an experimental campaign’s value through incorporating knowledge about how the data will be utilized into the test planning process. Outcomes of these experimental credibility processes include characterization of experimental uncertainties, assessment of model validation or calibration quality, and assessment of experimental alignment with modeling goals. 2.2 Important Elements for QMU Credibility Following the strategy for developing a credibility framework laid out by the CompSim community, potential sources of error, uncertainty, bias, or assumptions that could impact a QMU analysis are categorized into elements. It is proposed that QMU credibility can be assessed using the following five elements.
14 B. B. Schroeder et al. QMU Credibility Elements 1. Requirement Definition and QoI Selection Defining the requirement against which performance is compared and selecting the appropriate quantity of interest that can be used to represent performance 2. Data Quality Evaluating the available data and its attributes 3. Model Uncertainty Describing any models used to analyze the data and associated assumptions 4. Calibration/Parameter Estimation Considering how the model is fit or calibrated 5. Validation Determining if the model is a sufficient representation of the data with respect to making the prediction of interest The five elements are described in more detail in the subsections below. At the end of each element-specific section, suggested documentation is provided that would support credibility statements for each element. 2.2.1 Requirement Definition and QoI Selection Requirements may sometimes be clearly specified and the mapping from available data to that requirement may be simple, but this is not strictly true. Requirements may need interpretation that comes from consultation with a subject matter expert or simply from the QMU analyst. Available data often requires additional assumptions and/or processing to be comparable with the requirement. The quantity compared against the requirement is referred to as the QoI. QoIs are typically physical quantities, while requirements may be functions of these physical quantities. Determining how the requirement and QoI definitions will be compared is a necessary step of a QMU analysis. Suggested Documentation What is the requirement? Are there any perceived ambiguities in the requirement definitions? What is the QoI? What is the relevance of the QoI relative to the requirement? 2.2.2 Data Quality A great deal of qualitative information lives with the dataset that may impact the value of the dataset. Specifically, metadata about a dataset should be documented and preserved, so that important information about the data-generating mechanism can be evaluated when the data are analyzed. Metadata may include: • When was the data gathered? • What method was used to capture the measurements? • How well developed was the measurement/experimental method? • Where was the test conducted? • Who conducted the test? • What tester(s) was used? • How well characterized are the experimental conditions? Transparently evaluating metadata reduces the risk of omitting information that may impact the conclusion of the analysis. The following four categories are common categories of such auxiliary data (but should not be considered allencompassing).
2 The Need for Credibility Guidance for Analyses Quantifying Margin and Uncertainty 15 1. Sparsity The amount of data available impacts how much sampling uncertainty will exist in an estimate. Further, some estimates require more data than others to avoid extrapolative inferences; for instance, estimating a mean typically requires much less data than estimating an extreme percentile or rare exceedance probability to avoid extrapolation. Issues with presenting distributional tail extrapolation have been highlighted in [18]. Suggested Documentation How much data is available? Is the data sufficient to empirically validate any estimates being made? 2. Representativeness The QoI often cannot be directly measured given the available data. Therefore, the analyst must consider how the available data map onto the QoI. For example, are we interested in environment A, but only have data tested in a similar, but less stressing environment B? Suggested Documentation What is the representativeness of the data relative to the application space (including tested environments, age, etc.), as defined by the QoI? 3. Noisiness/Measurement uncertainty Most measurements contain some error. This error can arise from many different sources. A common source of error is the tester or instrument’s measurement error. In addition, errors can be introduced during data processing steps to convert a signal captured by a measurement device to a physical quantity. Uncertainty in the measurement can also be injected into the data through uncertainty about what is truly being measured. For example, measurement devices may be placed in orientations and exposed to boundary conditions that deviate from those specified for the experiment. Suggested Documentation What are the magnitudes and hypothesized sources of the measurement errors? 4. Bad data/Outliers Rejection of bad (inaccurate) data or non-physical outliers is an aspect of data analysis. Omitting outliers is often acceptable, but only when the root cause of the outlier is known. Understanding the root cause of impactful outliers often requires investigation into manufacturing and/or measurement process. Suggested Documentation How much data was rejected (not included in the final analysis)? Why it was rejected? 2.2.3 Model Uncertainty Models, whether physics-based or statistical, are an important aspect of QMU analyses, particularly when data are sparse or are not representative. Information about the types of models, underlying assumptions, and additional uncertainties associated with modeling activities must be considered and communicated. If the model is purely physics-based, then existing predictive maturity methods like PCCM [13] can be used to assess the model credibility. If the model is empirical or statistical, then the credibility for these types of models should be evaluated, though we are not aware of any formal frameworks for evaluating model credibility. Goodness-of-fit methods are not sufficient metrics for evaluating model credibility [19], due to only testing if the distribution form hypothesis can be rejected. A typical means of assessing a statistical model’s prediction capabilities is to demonstrate the model’s ability to predict data not used to train the model. While such activities may be used to support model validation (as will be address later in the validation subsection), this does not probe the underlying model uncertainties we deem to be essential to model credibility. We recommend assessing two components of model credibility: the causal structural and functional assumptions of the model. 1. Causal structural assumptions The inability to accurately represent the collected data in the empirical model will introduce bias in QoI estimates. Causal structural assumptions concern whether causal or physics-based relationships can be learned from the available data by comparing how the data were generated to an underlying causal model for the data. Specifically, causal analysis concerns establishing underlying causal relations between variables and then determining if the collected data are sufficient to infer the QoI under these causal relations [20]. Common sources of bias include [21]: • Omitted variable bias: important variables were not measured in the dataset that should be included in the model to accurately capture the physics in the empirical model. • Selection bias: the data are not a random sample from the population, but the model assumes a random sample. Suggested Documentation Was the causal structure of the model considered? Is the fitted model consistent with an underlying causal model for the data? Is selection or omitted variable bias present? To what fidelity is the causal structure understood?
RkJQdWJsaXNoZXIy MTMzNzEzMQ==