3 Validation of Strongly Coupled Models: A Framework for Resource Allocation 29 x1 ind, θ 1 ind, v 1 ind Constituent 1 Constituent 2 x 2 ind, θ 2 ind, v 2 ind Y1 ind Y2 ind Y1 dep Y2 dep Yint x2 dep, θ 2 dep, v 2 dep x1 dep, θ 1 dep, v 1 dep Fig. 3.3 Schematic representation of independent and dependent parameters interfaced between constituent models for a two-model strong coupling problem Although not explicitly shown in Fig. 3.3, discrepancy discussed in the previous section is part of Ydep i and thus is propagated between constituent models during coupling iterations. In this step, several different approaches can be taken. For example, each constituent can be calibrated using the experiments available in their respective domains (traditional treatment of model calibration or single-solvers models using separate-effect experiments, for instance as discussed in [2]). Alternatively, all available experiments (both separate and integral) can be utilized in the calibration of both the constituent models and the coupled model (for instance through an approach discussed in [13]). In either approach, the need of any given constituent for improvement can be quantified through this identified discrepancy suitably normalized for comparative evaluation between constituents. Of course, a constituent with a higher relative discrepancy would have a higher priority compared to a constituent with a lower relative discrepancy. Furthermore, the discrepancy identified for each of the constituents can be used to bias-correct the constituent models prior to coupling and the calibration of constituent models can be repeated at each iteration of coupling algorithm. This treatment would lead to a nested problem, where the inner loop tackles model calibration and bias-correction and the outer loop handles the coupling iterations. Provided that discrepancies of the constituent models are represented with sufficient accuracy, this approach would help reduce the discrepancy of the coupled model. A demonstration of this principle can be seen in Stevens et al. [40]. It is important to note that a functional relationship exists between the discrepancy of the resulting coupled model and the discrepancies of individual constituents. This functional relationship is unknown, but can be evaluated numerically through sensitivity analysis. Sensitivity analysis would also explain the relative importance of a constituent model in improving the predictive ability of the coupled system. Various forms of sensitivity analysis can be implemented, such as through the statistical concept known as coefficient of determination [5, 21], which determines the variance of one variable that is predictable from the other variable [6]. From a resource allocation standpoint, the principle is that a constituent with negligible sensitivity would have a small influence on the discrepancy of the coupled system, therefore, dedicating resources for improvement of this constituent would be ineffective in improving the predictive ability of the coupled model. Hence, a constituent with larger values of coefficient of determination should have a higher priority in code improvement activities. Lastly, for each constituent, the demands of code development on resources must also be considered. This is perhaps the most challenging aspect in code prioritization due to hard-to-control effects of human and organization related factors [30]. A lower cost indicates a higher priority for that constituent model as a lower amount of resources are sufficient to achieve improvement. The overall goal of the code prioritization then entails achieving the most improvement in the predictive capability of the coupled system with a minimum cost. Hence, the aforementioned three attributes: (1) need for improvement, (2) importance of improvement and (3) demands on resources can be combined to rank the constituents for future code development activities. 3.4.2 Experiment Prioritization It should be intuitive that discrepancy identified for each constituent model would be non-uniform throughout the domain of applicability, exhibiting a functional dependency on the control parameter settings. Evaluation of this dependency can allow the model developer to unearth the deficiencies of a simulation model. For instance, if the discrepancy of the model is greater
RkJQdWJsaXNoZXIy MTMzNzEzMQ==