30 S. Atamturktur and G. Stevens at high temperatures, then the analyst can infer that the temperature dependency of the system is not resolved with sufficient accuracy. This non-uniformity of discrepancy for each constituent model as well as for the coupled model can be exploited in prioritizing validation experiments [3, 48]. Specifically, the extent to which a physical experiment can improve the predictive maturity of the coupled model depends on: (1) the domain of the experiment, i.e. separate- versus integral-effect experiment and (2) the settings at which that particular experiment is conducted. The purpose of any design of experiments should therefore be to most efficiently improve the predictive maturity of a model by intelligently selecting both the domain and settings for future experiments from the pool of possible experimental campaigns. Selecting future experiments depends upon the separate- and integral-effect experiments available at the time of the selection, where the optimality condition (i.e., objective function) is defined by an appropriate design criterion. Hemez et al. [19] articulates, quantitatively, the diminishing returns in improving the predictive capability by conducting an increasing number of experiments (also see [1]), which ultimately leads to a stabilization of the model predictions as discussed earlier in Sect. 3.3. Building upon this principle, the optimality condition can be defined as achieving stability in model predictions as new batches of experiments become available. In determining stability, predictive density of the unobserved predictions supplies a suitable feature that fully accounts for all relevant sources of uncertainty. Stability of predictive density can be quantified through various information-based metrics, such as Expected Improvement for Predictive Stability (EIPS), Expected Improvement for Global Fit (EIGF), the Expected Maximum Entropy (ENT), and the (Integrated and Maximum) Mean Square Error (MSE) [29]. The EIPS criterion, also known as the expected Kullback–Leibler information, can be expressed as the loss in entropy between the initial predictive density and the predictive density obtained if new experiments are indeed performed. The EIGF criterion selects experiments which efficiently train surrogate models and defines the optimal experimental design as one that balances the trade-off between reducing variance and defining bias [25]. The ENT criterion operates by selecting experiments that produce maximum information gain regarding discrepancy at untested settings by minimizing entropy in predictions [38]. The MSE criterion selects experiments based on minimization of variance in the posterior discrepancy. Specifically, integrated MSE minimizes the integrated discrepancy variance and maximum MSE minimizes the largest discrepancy variance, both throughout the entire input domain [36, 37]. With this optimality condition, the selected experiments will be those that, once conducted, minimize the quantity of information which could be gained through further experimentation. 3.5 Conclusions This paper presented a resource allocation framework devised specifically for the validation of strongly coupled models (shown earlier in Fig. 3.2). This framework closely associates numerical modeling efforts with the design of physical experiments, where those physical experiments are used to improve the predictive capabilities of numerical models while the associated model predictions are in turn used to design future experiments [17]. This approach naturally leads to an iterative, sequential augmentation in designing future experiments [34] (Thompson 2010) where the model developer is allowed to refine model predictions between each batch resulting in a more efficient use of resources. The model validation framework would then continue with code improvement and the selection of batches of experiments until the MVUQ budget is consumed or a threshold gain in stability fails to be met. Implementation of the framework discussed herein is demonstrated to yield to efficient allocation of resources focusing on a controlled case study application, a scaled steel frame built and tested in the laboratory (see [17, 39]), focusing on the plasticity of polycrystals in Atamturktur et al. [3] and Stevens et al. [40]. The framework discussed herein, however omits discussions on numerical uncertainties [33], which needs to be evaluated in future research (as discussed in a companion paper [13]). Acknowledgements This research is being performed using funding received from the DOE Office of Nuclear Energy’s Nuclear Energy University Programs (Contract Number: 00101999). References 1. Atamturktur S, Hemez F, Williams B, Tome C, Unal C (2011) A forecasting metric for predictive modeling. Comput Struct 98:2377–2387 2. Atamturktur S, Hemez F, Laman J (2012) Verification and validation applied to finite element models of historic masonry monuments. Eng Struct 43:221–234
RkJQdWJsaXNoZXIy MTMzNzEzMQ==