The European Centre for
Mathematics and Statistics in Metrology

Uncertainty evaluation for computationally expensive models


Stacks Image 19003
Many important applications in metrology are described by model equations whose numerical solutions are computationally expensive e.g., the Navier-Stokes equations for fluid flows, and other transport equations such as the heat equation. Most of these computationally expensive systems are strongly nonlinear. Linear approximations for uncertainty evaluation, as suggested by the GUM, can be applied, but since the results may be invalid, simulations using a Monte Carlo method are required, as is recommended by Supplement 1 to the GUM (GUM-S1). However, current Monte Carlo methods are based on a large number of model evaluations and consequently cannot be applied practically to computationally expensive systems due to the time involved or the computational power requirements. As a result, uncertainty statements of measurements in such systems are usually either missing or are based on crude approximations.
Stacks Image 113307

Finite element methods for the analysis of flow measurements
is an example for computationally expensive models

In the analysis of computationally expensive problems the model function $$\mathbf{Y}=F(\mathbf{X})$$ is typically considered as a 'black box', so that evaluation of $F$ is possible, but evaluation of its derivatives with respect to the inputs is not. In computationally expensive models the evaluation of $F$ requires a significant amount of computer calculation time. This makes a straightforward application of Monte Carlo sampling for uncertainty evaluation impractical.

To this end, many approaches to uncertainty evaluation for expensive models consider alternative sampling methods. Examples are stratified sampling, importance sampling or Latin hypercube sampling, where the selection of realisations drawn from the input quantities follows a certain rule. The goal is to reduce the number of samples that have to be propagated by selecting representative samples.

A different approach is that of so called 'polynomial chaos', which was first used by Wiener in 1938 in the context of statistical mechanics. The main idea is that every stochastic quantity with finite second moment can be expanded as a sum of a set of orthogonal polynomials in an appropriate set of random variables. This set of of orthogonal polynomials of a random variable is called polynomial chaos. Its main advantage is that one can obtain a good representation of the complete stochastic behaviour of the outputs for fewer samples than are required by other methods. More information on polynomial chaos can be found here.

Instead of using smarter sampling methods one may replace the complex model function by a so called surrogate model. Examples are nearest neighbour interpolation and Gaussian process emulation and polynomial chaos.


Smart sampling

Smart sampling methods (such as polynomial chaos and Latin hypercube sampling) can drastically reduce the number of function evaluations and the resulting computational cost required for uncertainty evaluation compared to random sampling. Many sampling methods have specific assumptions (such as the inputs must be independent, the output quantity can be locally approximated as a polynomial function of the input quantities), which can limit their applicability. Thus, smart sampling methods have to be adapted and tested for their suitability for application in metrology.

Surrogate models

While the smart sampling methods try to minimise the computational expense by making the number of function evaluations needed for determination of uncertainties as small as possible, surrogate models replace the computationally expensive model by an approximate computationally cheap model such as polynomial or spline models. This involves a computationally expensive pre-processing step that samples the parameter space of the original physical model in order to derive the surrogate model.

Related journal papers

L. WrightParameter Estimation from Laser Flash Experiment DataChapter 8 of Computational optimisation and applications in engineering and industry, S. Koziel and X. S. Yang (eds.), Springer2011
Bloembergen, P., Battuello, M., Girard, F., Machin, G., Wright, L. On the Influence of the Furnace and Cell Conditions on the Phase Transition of the Eutectic Co–CInternational Journal of Thermophysics2015
L. Wright, S. P. Robinson and V. F. HumphreyPrediction of acoustic radiation from axisymmetric surfaces with arbitrary boundary conditions using the boundary element method on a distributed computing systemJ. Acoust. Soc. Am. 125, 1374-13832009
S. Alonso, K. John, and M. BärComplex wave patterns in an effective reaction-diffusion model for chemical reactions in microemulsionsJ. Chem. Phys. 134, 0941172011
H. Groß, J. Richter, A. Rathsfeld and M. BärInvestigations of a robust profile model for the reconstruction of 2D periodic absorber lines in scatterometryJ. Eur. Opt. Soc. – Rapid 52010
G. Lindner, S. Schmelter, R. Model, A. Nowak, V. Ebert und M. BärA Computational Fluid Dynamics Study on the Gas Mixing Capabilities of a Multiple Inlet SystemJ. Fluids Eng, 138(3), 0313022015
T. R. Emardson, P.O.J. Jarlemark, P. FlobergUncertainty evaluation in multivariate analysis - a test case study.J. Math. Model. Algorithm. 4, 289-3052005
R. Model, A. Rathsfeld, H. Groß, M. Wurm and B. BodermannA scatterometry inverse problem in optical mask technology.J. Phys. 135, 0120712008
A. Allard, N. Fischer, F. Didieux, E. Guillaume and B. IoossEvaluation of the most influent input variables on quantities of interest in a fire simulationJ. Soc. Fr. Stat. 152,103-117.2011
H. Gross, S. Heidenreich, M. A. Henn, F. Scholze and M. BärModelling line edge roughness in periodic-line space structures by Fourier optics to improve scatterometryJEOS, vol. 9, 14003 (10pp)2014
S. Demeyer, J.-L. Foulley, N. Fischer and G. SaportaBayesian analysis of structural equation models using parameter expansionLearning and data science, L. Bottou, F. Murtagh, M. Gettler-Summa, B. Goldfarb, C. Pardoux and M. Touati (eds.), Chapman & Hall2011
H. Gross, A. Rathsfeld, F. Scholze and M. BärProfile reconstruction in extreme ultraviolet (EUV) scatterometry: modeling and uncertainty estimatesMeas. Sci. Technol, vol. 20, 105102-1041122009
T. J. Esward and L. WrightEfficient updating of PDE models for metrologyMeasurement, vol. 792015
A. Allard, N. Fischer, G. Ebrard, B. Hay, P. M. Harris, L. Wright, D. Rochais, J. MattoutA multi-thermogram based Bayesian model for the determination of the thermal diffusivity of a materialMetrologia2016
GJP Kok, AMH van der Veen, PM Harris, IM Smith, C ElsterBayesian analysis of a flow meter calibration problemMetrologia 52, 392-3992015
S. Alonso, R. Kapral and M. BärEffective medium theory for reaction rates and diffusion constants in heterogeneous systemsPhys. Rev. Lett. 102, 2383022009
S. Heidenreich, H. Gross and M. BärAlternative methods for uncertainty evaluations in EUV scatterometryProc. SPIE: Modeling aspects in Optical metrology IV2013
G. W. A. M. van der Heijden and R. EmardsonMultivariate measurementsTheory and methods of measurements with persons, B. Berglund, G. B. Rossi, J. Townsend and L. R. Pendrill (eds.), Psychology Press, Taylor & Francis.2011
H. Groß and A. RathsfeldSensitivity analysis for indirect measurement in scatterometry and the reconstruction of periodic grating structuresWaves. Random. Complex. Media. 18, 129-1402008
This website uses cookies occasionally to provide you with the best web browsing experience. However, no web-analytics tracking based on cookies is employed here.