MATHMET

The European Centre for
Mathematics and Statistics in Metrology

Bayesian framework for proficiency tests using auxiliary information on laboratories

A recent paper proposes a Bayesian framework to analyse interlaboratory comparison data when auxiliary information regarding the quality of the practice of laboratories can be investigated or when reported uncertainties are modelled to account for the fact that they are estimates of the unknown variances.
Authors S. Demeyer and N. Fischer (LNE)

A recent paper proposes a Bayesian framework to analyse interlaboratory comparison data when auxiliary information regarding the quality of the practice of laboratories can be investigated or when reported uncertainties are modelled to account for the fact that they are estimates of the unknown variances.

When uncertainties are available, modelling uncertainties allows to take into account that reported uncertainties are only estimates of the unknown variances.
Figure 1 shows a comparison of this method (ICUWmean) with traditional uncertainty weighted mean (UWmean) and corrected for scale adjustment uncertainty weighted mean (CUWmean) on CCQM-K70 key comparison. Given the gravimetric KCRV, only the ICUW mean yields realistic uncertainty, where UWmean clearly produces too small uncertainty.

Pasted Graphic 2
Figure 1 Expanded uncertainties with a coverage factor k=2 obtained for the gravimetric KCRV and the three uncertainty weighted mean based methods


When uncertainties are not available
(e.g. when laboratories do not have to report uncertainties or when the uncertainties are not validated independently), auxiliary information on laboratories is investigated to produce more reliable estimates of the consensus value and its associated uncertainty. This approach relies on the fact that some laboratories share common features (e.g. same method), forming groups of laboratories, and most importantly, that experts can produce a ranking of the groups on an ordinal scale. Ordinal data are then classically converted into thresholded continuous latent variables. If these conditions are not met, we recommend to analyse data with traditional statistical methods like algorithm A from NF ISO 13528.

The methodology results in a distribution of weights for laboratories, over which is integrated the posterior distribution of the consensus value.
Figures 2 and 3 show the effect of the number of laboratories on the ability of the method to well discriminate between groups, by plotting the distributions of the thresholds associated to an ordinal variable with 4 categories. As the number of laboratories increases, distinction between groups is made clearer. Such graphs can help validate the methodology on a particular application.

Estimated densitities for various threshold values
Figure 2 Plot of the estimated densities of the thresholds $\gamma_1, \gamma_2, \gamma_3, \gamma_4$ from left to right, based on 16 laboratories.


Estimated densitities for 80 laboratories
Figure 3 Plot of the estimated densities of the thresholds $\gamma_1, \gamma_2, \gamma_3, \gamma_4$ from left to right, where the number of laboratories is multiplied by 5 in each category for a total of 80 laboratories


Reference
Demeyer, S. and Fischer, N., Bayesian framework for proficiency tests using auxiliary information on laboratories, Accred Qual Assur (2017) 22: 1. doi:10.1007/s00769-017-1247-y
blog comments powered by Disqus