Conformity assessment
Description

For conformity assessment the state of the art is embodied in documents such as ISO 10576 and draft documents including GUM Supplement JCGM 106 and OIML documents. Guides such as those from EURACHEM are already widely used by major stakeholder groups. However, others such as OIML D 30: Guide for the application of ISO/IEC 17025 to the assessment of Testing Laboratories involved in legal metrology do not reflect modern developments in statistical methods.
Measurement uncertainty can lead to a risk of incorrect decisions when attempting to answer questions about an entity, for example, whether the test result inside or outside specifications. Probability density functions (PDFs) that characterise the measurands and the methods to obtain them are also key information used in conformity assessment and decision-making. This is because the probability of the measurand exceeding a specification limit is calculated as an integral of the probability distribution. This probability is then used as a basis for a decision as to whether the object of interest conforms to the given requirements.
Decisions of conformity are currently made in many important application areas, such as environmental monitoring and product safety testing, without a clear and harmonised basis for sharing the risks that arise from measurement uncertainty between the consumer and the supplier. Measurements requiring multivariate approaches (e.g. in healthcare products) are commonly required in conformity assessment. In these cases, two or more quantities and the associated PDFs are used in conformity assessment and decision making. In many fields demands for conformity assessment based on computationally expensive modelling (e. g. in fire engineering) are also of increasing interest. Current guidelines do not address these issues.
Decisions of conformity are currently made in many important application areas, such as environmental monitoring and product safety testing, without a clear and harmonised basis for sharing the risks that arise from measurement uncertainty between the consumer and the supplier. Measurements requiring multivariate approaches (e.g. in healthcare products) are commonly required in conformity assessment. In these cases, two or more quantities and the associated PDFs are used in conformity assessment and decision making. In many fields demands for conformity assessment based on computationally expensive modelling (e. g. in fire engineering) are also of increasing interest. Current guidelines do not address these issues.
Research
Decision-making and conformity assessment in multivariate cases
Multivariate approaches to decision-making in these (in the broadest sense) conformity assessment contexts are commonly required. Such approaches are based on the knowledge of the PDFs (or joint PDFs) of two or more measured quantities.
A simple illustration of bivariate conformance regions and probability density functions (PDF) is shown in Figure 1 (taken from multivariate process control where entity variation was assumed much larger than measurement uncertainty [Abu Zahid & Sultana 2008]). A value drawn at random from this PDF that lies within the tolerance region is a conforming value (blue); otherwise it is judged as a case of non-conformity (red). For many draws from the PDF, the fraction of conforming values is the conformance probability
A simple illustration of bivariate conformance regions and probability density functions (PDF) is shown in Figure 1 (taken from multivariate process control where entity variation was assumed much larger than measurement uncertainty [Abu Zahid & Sultana 2008]). A value drawn at random from this PDF that lies within the tolerance region is a conforming value (blue); otherwise it is judged as a case of non-conformity (red). For many draws from the PDF, the fraction of conforming values is the conformance probability
Unfortunately there are very few guidelines which treat the multivariate case about handling the risks of incorrect decisions caused by measurement uncertainty when attempting to answer questions about an entity, for example, whether the test result is inside or outside specifications. Multivariate guidelines which specifically include cost modelling are even fewer. Hence, there is a need to extend existing approaches to multivariate cases and develop a harmonised treatment of multivariate conformity assessment.
Related journal papers
Authors
|
Title
|
Journal
|
Year
|
---|---|---|---|
H. Källgren, M. Lauwaars, B. Magnusson, L. Pendrill and P. Taylor | Role of measurement uncertainty in conformity assessment in legal metrology and trade | Accred. Qual. Assur. 8, 541-547 | 2003 |
L. R. Pendrill | Optimised measurement uncertainty and decision-making in conformity assessment | NCSLi Measure, vol 2, nr. 2, 76-86 | 2007 |
L. R. Pendrill, R. Emardson, B. Berglund, M. Gröning, A. Höglund, A. Cancedda, G. Quinti, F. Crenna, G. B. Rossi, J. Drnovsek, G. Gersak, T. Goodman, S. Harris, G. van der Heijden and K. Kallinen | Measurement with persons: a European network | NCSLi Measure 5, 42-54 | 2010 |
L. R. Pendrill | Uncertainty & risks in decision-making in qualitative measurement | AMCTM 2011 International Conference on Advanced Mathematical and Computational Tools in Metrology and Testing, Göteborg, Sweden | 2011 |
L. R. Pendrill | Risk assessment and decision-making | Theory and methods of measurements with persons, B. Berglund, G. B. Rossi, J. Townsend and L. R. Pendrill (eds.), Psychology Press, Taylor & Francis. | 2011 |
(BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML | Evaluation of measurement data – The role of measurement uncertainty in conformity assessment | Joint Commitee for Guides in Metrology, Bureau International des Poids et Mesures, JCGM 106 | 2012 |
L R Pendrill and W P Fisher Jr | Quantifiying Human Response: Linking metrological and psychometric characterisations of Man as a Measurement Instrument | Journal of Physics Conference Series 459 (2013) 012057 | 2013 |
L. R. Pendrill | Using measurement uncertainty in decision-making & conformity assessment | Metrologia, Vol 51, S206 - S218 | 2014 |
L. R. Pendrill and W. P. Fisher | Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number | Measurement 71, pp. 46-55 | 2015 |
S. Demeyer, N. Fischer, F. Didieux and M. Binacchi | Statistical methods for conformity assessment when dealing with computationally expensive systems : application to a fire engineering case study | Advanced Mathematical and Computational Tools in Metrology and Testing X, World Scientific vol 86 | 2015 |
L. R. Pendrill and N Petersson | Metrology of human-based and other qualitative measurements | Measurement Science & Technology, 27, 094003 | 2016 |
S. Demeyer, N. Fischer, D. Marquis | Surrogate model based sequential sampling estimation of conformance probability for computationally expensive systems: application to fire safety science | Journal de la Société Française de Statistique, Vol. 158, No. 1, p.111-138 | 2017 |
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva and D. Brynn Hibbert | Conformity assessment of multicomponent materials or objects: Risk of false decisions due to measurement uncertainty | Talanta 164, 189-195 | 2017 |
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva and D. Brynn Hibbert | Risk of false decision on conformity of a multicomponent material when test results of the components' content are correlated | Talanta 174, 789-796 | 2017 |
F. Pennecchi, I. Kuselman, R. J. N. B. da Silva and D. Brynn Hibbert | Risk of a false decision on conformity of an environmental compartment due to measurement uncertainty of concentrations of two or more pollutants | Chemosphere 202, 165-176 | 2018 |
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva, D. Brynn Hibbert and Elena Anchutina | Total risk of a false decision on conformity of an alloy due to measurement uncertainty | Talanta 189, 666-674 | 2018 |