Conformity assessment
Description
For conformity assessment the state of the art is embodied in documents such as ISO 10576 and draft documents including GUM Supplement JCGM 106 and OIML documents. Guides such as those from EURACHEM are already widely used by major stakeholder groups. However, others such as OIML D 30: Guide for the application of ISO/IEC 17025 to the assessment of Testing Laboratories involved in legal metrology do not reflect modern developments in statistical methods.
Measurement uncertainty can lead to a risk of incorrect decisions when attempting to answer questions about an entity, for example, whether the test result inside or outside specifications. Probability density functions (PDFs) that characterise the measurands and the methods to obtain them are also key information used in conformity assessment and decisionmaking. This is because the probability of the measurand exceeding a specification limit is calculated as an integral of the probability distribution. This probability is then used as a basis for a decision as to whether the object of interest conforms to the given requirements.
Decisions of conformity are currently made in many important application areas, such as environmental monitoring and product safety testing, without a clear and harmonised basis for sharing the risks that arise from measurement uncertainty between the consumer and the supplier. Measurements requiring multivariate approaches (e.g. in healthcare products) are commonly required in conformity assessment. In these cases, two or more quantities and the associated PDFs are used in conformity assessment and decision making. In many fields demands for conformity assessment based on computationally expensive modelling (e. g. in fire engineering) are also of increasing interest. Current guidelines do not address these issues.
Decisions of conformity are currently made in many important application areas, such as environmental monitoring and product safety testing, without a clear and harmonised basis for sharing the risks that arise from measurement uncertainty between the consumer and the supplier. Measurements requiring multivariate approaches (e.g. in healthcare products) are commonly required in conformity assessment. In these cases, two or more quantities and the associated PDFs are used in conformity assessment and decision making. In many fields demands for conformity assessment based on computationally expensive modelling (e. g. in fire engineering) are also of increasing interest. Current guidelines do not address these issues.
Research
Decisionmaking and conformity assessment in multivariate cases
Multivariate approaches to decisionmaking in these (in the broadest sense) conformity assessment contexts are commonly required. Such approaches are based on the knowledge of the PDFs (or joint PDFs) of two or more measured quantities.
A simple illustration of bivariate conformance regions and probability density functions (PDF) is shown in Figure 1 (taken from multivariate process control where entity variation was assumed much larger than measurement uncertainty [Abu Zahid & Sultana 2008]). A value drawn at random from this PDF that lies within the tolerance region is a conforming value (blue); otherwise it is judged as a case of nonconformity (red). For many draws from the PDF, the fraction of conforming values is the conformance probability
A simple illustration of bivariate conformance regions and probability density functions (PDF) is shown in Figure 1 (taken from multivariate process control where entity variation was assumed much larger than measurement uncertainty [Abu Zahid & Sultana 2008]). A value drawn at random from this PDF that lies within the tolerance region is a conforming value (blue); otherwise it is judged as a case of nonconformity (red). For many draws from the PDF, the fraction of conforming values is the conformance probability
Unfortunately there are very few guidelines which treat the multivariate case about handling the risks of incorrect decisions caused by measurement uncertainty when attempting to answer questions about an entity, for example, whether the test result is inside or outside specifications. Multivariate guidelines which specifically include cost modelling are even fewer. Hence, there is a need to extend existing approaches to multivariate cases and develop a harmonised treatment of multivariate conformity assessment.
Related journal papers
Authors

Title

Journal

Year


S. Demeyer, N. Fischer, D. Marquis  Surrogate model based sequential sampling estimation of conformance probability for computationally expensive systems: application to fire safety science  Journal de la Société Française de Statistique, Vol. 158, No. 1, p.111138  2017 
L. R. Pendrill and N Petersson  Metrology of humanbased and other qualitative measurements  Measurement Science & Technology, 27, 094003  2016 
L. R. Pendrill and W. P. Fisher  Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number  Measurement 71, pp. 4655  2015 
S. Demeyer, N. Fischer, F. Didieux and M. Binacchi  Statistical methods for conformity assessment when dealing with computationally expensive systems : application to a fire engineering case study  Advanced Mathematical and Computational Tools in Metrology and Testing X, World Scientific vol 86  2015 
L. R. Pendrill  Using measurement uncertainty in decisionmaking & conformity assessment  Metrologia, Vol 51, S206  S218  2014 
L R Pendrill and W P Fisher Jr  Quantifiying Human Response: Linking metrological and psychometric characterisations of Man as a Measurement Instrument  Journal of Physics Conference Series 459 (2013) 012057  2013 
(BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML  Evaluation of measurement data – The role of measurement uncertainty in conformity assessment  Joint Commitee for Guides in Metrology, Bureau International des Poids et Mesures, JCGM 106  2012 
L. R. Pendrill  Uncertainty & risks in decisionmaking in qualitative measurement  AMCTM 2011 International Conference on Advanced Mathematical and Computational Tools in Metrology and Testing, Göteborg, Sweden  2011 
L. R. Pendrill  Risk assessment and decisionmaking  Theory and methods of measurements with persons, B. Berglund, G. B. Rossi, J. Townsend and L. R. Pendrill (eds.), Psychology Press, Taylor & Francis.  2011 
L. R. Pendrill, R. Emardson, B. Berglund, M. Gröning, A. Höglund, A. Cancedda, G. Quinti, F. Crenna, G. B. Rossi, J. Drnovsek, G. Gersak, T. Goodman, S. Harris, G. van der Heijden and K. Kallinen  Measurement with persons: a European network  NCSLi Measure 5, 4254  2010 
L. R. Pendrill  Optimised measurement uncertainty and decisionmaking in conformity assessment  NCSLi Measure, vol 2, nr. 2, 7686  2007 
H. Källgren, M. Lauwaars, B. Magnusson, L. Pendrill and P. Taylor  Role of measurement uncertainty in conformity assessment in legal metrology and trade  Accred. Qual. Assur. 8, 541547  2003 