The European Centre for
Mathematics and Statistics in Metrology

Conformity assessment


Stacks Image 20112
Many measurements are made to provide an objective basis for decisions about a product or a process, for example compliance with a regulation related to environmental emissions, decisions as to whether or not a person is infected with a disease, assessment of compliance of a product with manufacturing tolerances or the water content in oil for legal metrology and fiscal purposes. The inevitable presence of measurement uncertainty leads to the risk of incorrect conformity decisions for consumers, suppliers and the public at large. There is therefore the need to make reliable decisions of conformity given relevant measurement results with associated uncertainty statements and to ensure the consistent application of decision-making techniques, in particular for multivariate cases, which are not currently addressed in guidelines and standards. One example of a multivariate case of conformity assessment is the decision as to whether a specimen meets its design, when this design is not described in terms of a single parameter, but in terms of several parameters (e.g. the several critical dimensions of periodic nanostructures).

For conformity assessment the state of the art is embodied in documents such as ISO 10576 and draft documents including GUM Supplement JCGM 106 and OIML documents. Guides such as those from EURACHEM are already widely used by major stakeholder groups. However, others such as OIML D 30: Guide for the application of ISO/IEC 17025 to the assessment of Testing Laboratories involved in legal metrology do not reflect modern developments in statistical methods.
Measurement uncertainty can lead to a risk of incorrect decisions when attempting to answer questions about an entity, for example, whether the test result inside or outside specifications. Probability density functions (PDFs) that characterise the measurands and the methods to obtain them are also key information used in conformity assessment and decision-making. This is because the probability of the measurand exceeding a specification limit is calculated as an integral of the probability distribution. This probability is then used as a basis for a decision as to whether the object of interest conforms to the given requirements.

Decisions of conformity are currently made in many important application areas, such as environmental monitoring and product safety testing, without a clear and harmonised basis for sharing the risks that arise from measurement uncertainty between the consumer and the supplier. Measurements requiring multivariate approaches (e.g. in healthcare products) are commonly required in conformity assessment. In these cases, two or more quantities and the associated PDFs are used in conformity assessment and decision making. In many fields demands for conformity assessment based on computationally expensive modelling (e. g. in fire engineering) are also of increasing interest. Current guidelines do not address these issues.


Decision-making and conformity assessment in multivariate cases

Multivariate approaches to decision-making in these (in the broadest sense) conformity assessment contexts are commonly required. Such approaches are based on the knowledge of the PDFs (or joint PDFs) of two or more measured quantities.
A simple illustration of bivariate conformance regions and probability density functions (PDF) is shown in Figure 1 (taken from multivariate process control where entity variation was assumed much larger than measurement uncertainty [Abu Zahid & Sultana 2008]). A value drawn at random from this PDF that lies within the tolerance region is a conforming value (blue); otherwise it is judged as a case of non-conformity (red). For many draws from the PDF, the fraction of conforming values is the conformance probability
Unfortunately there are very few guidelines which treat the multivariate case about handling the risks of incorrect decisions caused by measurement uncertainty when attempting to answer questions about an entity, for example, whether the test result is inside or outside specifications. Multivariate guidelines which specifically include cost modelling are even fewer. Hence, there is a need to extend existing approaches to multivariate cases and develop a harmonised treatment of multivariate conformity assessment.

Related journal papers

H. Källgren, M. Lauwaars, B. Magnusson, L. Pendrill and P. TaylorRole of measurement uncertainty in conformity assessment in legal metrology and tradeAccred. Qual. Assur. 8, 541-5472003
L. R. PendrillOptimised measurement uncertainty and decision-making in conformity assessmentNCSLi Measure, vol 2, nr. 2, 76-862007
L. R. Pendrill, R. Emardson, B. Berglund, M. Gröning, A. Höglund, A. Cancedda, G. Quinti, F. Crenna, G. B. Rossi, J. Drnovsek, G. Gersak, T. Goodman, S. Harris, G. van der Heijden and K. KallinenMeasurement with persons: a European networkNCSLi Measure 5, 42-542010
L. R. PendrillUncertainty & risks in decision-making in qualitative measurementAMCTM 2011 International Conference on Advanced Mathematical and Computational Tools in Metrology and Testing, Göteborg, Sweden2011
L. R. PendrillRisk assessment and decision-makingTheory and methods of measurements with persons, B. Berglund, G. B. Rossi, J. Townsend and L. R. Pendrill (eds.), Psychology Press, Taylor & Francis.2011
(BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIMLEvaluation of measurement data – The role of measurement uncertainty in conformity assessment Joint Commitee for Guides in Metrology, Bureau International des Poids et Mesures, JCGM 1062012
L R Pendrill and W P Fisher JrQuantifiying Human Response: Linking metrological and psychometric characterisations of Man as a Measurement InstrumentJournal of Physics Conference Series 459 (2013) 0120572013
L. R. PendrillUsing measurement uncertainty in decision-making & conformity assessmentMetrologia, Vol 51, S206 - S2182014
L. R. Pendrill and W. P. FisherCounting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of numberMeasurement 71, pp. 46-552015
S. Demeyer, N. Fischer, F. Didieux and M. BinacchiStatistical methods for conformity assessment when dealing with computationally expensive systems : application to a fire engineering case studyAdvanced Mathematical and Computational Tools in Metrology and Testing X, World Scientific vol 86 2015
L. R. Pendrill and N PeterssonMetrology of human-based and other qualitative measurementsMeasurement Science & Technology, 27, 0940032016
S. Demeyer, N. Fischer, D. MarquisSurrogate model based sequential sampling estimation of conformance probability for computationally expensive systems: application to fire safety scienceJournal de la Société Française de Statistique, Vol. 158, No. 1, p.111-1382017
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva and D. Brynn HibbertConformity assessment of multicomponent materials or objects: Risk of false decisions due to measurement uncertaintyTalanta 164, 189-1952017
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva and D. Brynn HibbertRisk of false decision on conformity of a multicomponent material when test results of the components' content are correlatedTalanta 174, 789-7962017
F. Pennecchi, I. Kuselman, R. J. N. B. da Silva and D. Brynn HibbertRisk of a false decision on conformity of an environmental compartment due to measurement uncertainty of concentrations of two or more pollutantsChemosphere 202, 165-1762018
I. Kuselman, F. Pennecchi, R. J. N. B. da Silva, D. Brynn Hibbert and Elena AnchutinaTotal risk of a false decision on conformity of an alloy due to measurement uncertaintyTalanta 189, 666-6742018
This website uses cookies occasionally to provide you with the best web browsing experience. However, no web-analytics tracking based on cookies is employed here.