the best cases are only available in limited numbers. Moreover, obtaining the measurements at the desired experimental settings may be prohibitive due to policy regulations or infeasible due to technical limitations. The inevitable experimental scarcity is the primary reason behind the increased reliance on modeling and simulation in various scientific and engineering fields. The availability of simulation models incorporating sound physics or engineering principles can significantly reduce the number of required experiments for certification. In the most extreme example, if the model can perfectly reproduce reality, the dependency on experimentation can be entirely eliminated. However, simulation models are naturally impaired by imprecise model parameters (known unknowns) and inaccuracies in the interpretation of the underlying physics or engineering principles (unknown unknowns). Therefore, experimental evidence is routinely required to improve the simulation model fidelity. Hence, in model-based certification, experiments are used to calibrate and bias-correct the physics-based simulation models instead of training an arbitrary best-fitted function. Model calibration is achieved through the comparison of a multitude of model predictions with a family of experimental measurements. This comparison has two main objectives: (1) to reduce the uncertainty in the imprecise input parameters (known unknowns) and (2) to quantitatively estimate the errors due to inadequate or missing physics (unknown unknowns). The manner in which we distinguish between these two interrelated objectives, parameter calibration versus discrepancy bias, is explained in Section 2, where we discuss our approach to quantify discrepancy. The number of experiments needed to successfully achieve these objectives of model calibration is heavily dependent upon the fidelity of the modeled (physics or engineering) principles to reality. If the simulation model lacks a vital principle, parameter, or interaction between principles or parameters (i.e., large unknown unknowns), the fundamental ability of this model to capture the phenomena of interest is compromised. If the model is overly crude resulting in too large unknown unknowns, attempts to improve the fidelity through calibration or bias-correction will inevitably be unsatisfactory due to compensating effects. Hence, a crucial first step in certification involves assessing the suitability of a simulation model for use in forecasting. In Section 3, we outline three important assertions that we contend play a foundational role in determining the suitability of simulation models for forecasting. Because the purpose of simulation models is to predict in lieu of experiments, such models calibrated against a reduced number of experimental measurements are routinely applied to forecast at untested 432
RkJQdWJsaXNoZXIy MTMzNzEzMQ==