A Guide to the Evaluation of Measurement Accuracy
Semyon G. Rabinovich
Introduction
This Guide includes a collection of concise descriptions of methods for calculating measurement accuracy, starting from simple single measurements and ending with complex multiple indirect measurements. This collection aims at everyone who needs to know or apply these methods but may not want to understand all the theory behind them. Therefore, the justification of the formulas used in the included methods is not presented here; we refer the reader tomonograph [6] for this information.
The Guide presents state of the art methods reflecting physical characteristics of measurements and measuring instruments. First, it covers measurement tasks that occur commonly in practice but are often left without grounded solutions in metrological literature. For example, the Guide includes a universal method for summation of systematic and random errors – a thelong-standing problemthat was at some point consideredunsolvable [3]. Another gap includes single measurements, which are the most common measurement types in practice: the Guide includes methods for estimating accuracy of these measurements while traditional literature leaves them out, concentrating instead on statistical methods applicable for multiple measurements only.
Second, in many cases, for measurement tasks with traditionalsolutions, the Guide includes methods representing the latest advances in theory of measurements that provide substantially better solutions. These include the method of reduction for experimental data processing in dependent indirect measurements (especially nonlinear ones) and the methods of transformation and enumeration for independent indirect measurements. The method of reduction avoids the need for the linearization of the measurement equation (which immediately introduces inaccuracy into the data processing procedure and often leads to a bias in the estimate of the measurand) and rids the theory of measurements of the correlation coefficient (a known thorny issue). This method also allows one to characterize the inaccuracy of these measurements by a confidence interval. Traditionally inaccuracy in these measurements was characterized by the standard deviation, which is an imprecise measure that only reflects the reproducibility of measurement but not the accuracy of the measurement (see [6], Section 9.2).. The method of enumeration, and to a large extent the method of transformation, removed the need in linearization of the measurement equation in independent indirect measurements and for this reason, similar to the method of reduction, also improved the accuracy of measurement data processing. However, because these two methods have not yet been appropriately tested in practice, we provide them in the Appendix, retaining the traditional linearization method in the main text of this Guide.
The only previous relevant recommendation, "Guide to the expression of uncertainty in measurement"(GUM)[1] is old and has significant limitations(see [4,8] and Section 9.4 of [6]). In particular, it does not offer methods for assessing accuracy of single measurements (instead providing only discussion that is too general to deduce concrete practical methods), or a general method for computing confidence intervals (called “expanded uncertainty” in GUM). In fact, GUM’s method is simply wrong when combining systematic and random errors[1].
The need for GUM’srevision has long been understood and pointed out in [4,6, 8]. However, the new revision, which started as early as in 2006[4], has never come out. Instead, in 2008 and 2010, an updated version of GUM, which incorporated what it referred to as “minor corrections”, and a Supplement 1 to GUM appeared; neither addressed GUM’s shortcomings mentioned above.The present Guide addresses these drawbacks and reflects the latest advances in metrology. We thus believe it can serve as the replacement of GUM and the basis of a better future version of GUM that needs to be developed.
Acknowledgement: Many thanks to my son, Dr. Michael Rabinovich, for translating and editing this document.
Keywords: Measurement, single measurement, multiple measurement, measurement error, measurement uncertainty
- Key terminology
Note that some of the terminology listed here deviates from “International Vocabularly of Metrology” (VIM)[3]. See [6], Section 9.3 for rationale behind these deviations and criticism of VIM.
1.1. Measurement– A set of experimental operations, involving at least one measuring instrument, performed for the purpose of obtaining the value of a quantity.
1.2. Measuring instrument -- A technical product that is created for the purpose of being used in a measurement and which has known metrological characteristics.
1.3Direct measurement -- A measurement in which the value of the measurand is read from the indication of a measuring instrument; the latter can be multiplied by some factor or adjusted by applying certain corrections
1.4. Indirect measurement -- A measurement in which the estimate of the measurand is calculated using measurements of other quantities related to the measurand by known function. These other quantities are called measurement arguments (or briefly, arguments), and the function is called the measurement equation.
1.5. Single measurement --A measurement that is carried out by a single contact of the measuring instrument with the object whose characteristic is being measured. A single measurement utilizes a single indication of the measuring instrument.
1.6. Multiple measurement -- A measurement in which the same quantity is measured multiple times. Repeated measurements forming a multiple measurement are called observations.
1.7. True value of the measurand -- The value of a quantity that being known would ideally reflect the property of an object with respect to the purpose of the measurement.
Note: True value is an abstract concept that is used in the theory of measurements to refer to an ideal that one tries to approach but which – as any ideal -- can never be found. However, based on the measurement data, one can find an interval that covers the true value of the measurement.
Example:Consider a simple measurement problem – themeasurement of the diameter of a metal washer (disk). Diameter is a mathematical concept, which is a characteristic of a geometric figure, a circle. Thefact that the diameter of a disk is to be measured means that we have assumed that the model of the washer is taken to be a circle. The measurand -- the diameter of the circle -- is a parameter of the model. Thus, in this example, the true value of the measurand is the exact value of the diameter of the circle. To find this true value is obviously impossible, not least because the washer is never an ideal circle and thus its “diameter” would be different in different places. Nonetheless, it is possible to find an estimate of the diameter and the interval covering both the estimate and the true value. For more details see [6], Section 1.4.
1.8.Measurement inaccuracy --A quantitative characteristic of the degree of deviation of a measurement result from the true value of the measurand. Inaccuracy of a measurement may be expressed as limits of measurement error or as measurement uncertainty. Either alternative can be represented in the absolute or relative form. Measurement inaccuracy is expressed by no more than two significant figures. (More detailed rules are often used but we will adhere to the above rule in this guide for simplicity.)
1.9. Measurement result --An estimate of the true value of the measurand obtained by the means of a measurement. The measurement result is expressed as a dimensional value (i.e., the product of a number and proper measurement unit, such as 10cm) whose last digit must have the same significance as the last digit of the measurement inaccuracy in absolute form. Extra digits are dropped according to the rules of rounding.
1.10. Limits of measurement error --A measure of measurement inaccuracy specifying the limits of the deviation of the measurement result from the true value of the measurand, which are obtained with deterministic methods (i.e., without applying mathematical statistics). The limits of measurement error are expressed as offsets from the measurement result.
1.11.Uncertainty of measurement --A measure of measurement inaccuracy specifying the probabilistic limits of the deviation of the measurement result from the true value of the measurand, which are obtained with statistical methods. It is an interval within which the true value of a measurand lies with given confidence probability. Uncertainty is expressed by its limits, which are listed as offsets from the result of the measurement. The uncertainty of measurement is sometimes referred to as the confidence interval, in which case its limits are called confidence limits.
2. Conventions for expressing accuracy of measuring instruments
Measuring instruments are manufactured with a given pre-defined accuracy.
An appropriate classification of instruments based on their accuracy into “accuracy classes”is established by national standards and recommendations separately for each area of measurements (e.g., differently for voltmeters, hygrometers, etc). General rules for establishing these classes are specified in International Recommendation [7].
The limits of permissible errors of instruments are listed in instruments’ documentation during production. This documentation also specifies the conditions of usage of the device, which can be referenceconditions (the conditions under which the instrument is verified and calibrated) orrated conditions (a wider range of conditions under which the characteristics of the instrument remain within certain limits and the instrument can be used as intended). The error of an instrument under reference conditions is called the intrinsic error. Under rated conditions the instrument acquiresadditional errors.
Instruments in exploitation are periodically verified or calibrated in calibrating laboratories. During calibration, corrections to the instrument’s indications are established, which can increase the precision of the instrument, theoretically up to the accuracy of the calibration.
2.1. Analog Instruments
For analog instruments (i.e., instruments with an analog indicator moving within a certain scale), in which the limits of the permissible absolute error Δare constant across the entire scale, the permissible error is usually listed in the form of the limits offiducial errorϒ. The fiducial error is the ratio (expressed as the percentage) of the permissible absolute error to some standardized value (fiducial value) xf :
ϒ =100Δ/xf. (2.1)
The fiducialvalue depends on the scale type. For uniform scales, xf is usually taken to be the upper measurement limit of the instrument. In this case, the limits of fiducial error are the same as the limits of the intrinsic error expressed in relative form. For non-uniform scales (e.g., when the scale’s graduations narrow towards the upper range of the scale), xf is usually taken to be the length of the scale expressed in units of length (typically, cm). Accordingly, the limits of error Δ are also expressed in units of length (typically, mm). Finally, sometimes fiducial value is taken to be the nominal value of the measurand if it is defined (e.g., 60Hz in cymometers).
Note that the instrument error often has a random component, and this component is included in the intrinsic error. The parameters of this component – the variation of indications or standard deviation -- are sometimes listed in addition to the intrinsic error.
2.2. Digital Instruments
The accuracy of digital instruments isconventionally expressed in the form
±(b + q), (2.2)
where b and q represent the limits of components of the instrument error that remain constant over a givenmeasurement range of the instrument (we say “a given range” since some instruments can switch between multiple ranges). The first term represents the component that is constant when expressed in the form of relative error; the second is a certain number of units of the least significantdigit of the digital readout device (expressed as a non-dimensional count), which represents the constant error of the instrument for the given measurement range in absolute form. To obtain the value of permissible absolute error at a given indication, the first term in (2.2) is multiplied by the indication and added with the second term transformed into the measurement units of, the latter giving directly the error component in the absolute form once the value (in measurement units) of the least significant digit of the readout device is determined.
Sometimes term q is expressed in percent or in ppm relative to some normalizing value, typically the upper limit of the measurement range. This might create an impression that q represents a relative error but in fact it is still an absolute error albeit expressed in a normalized form.
A more convenient and less error-prone way of expressing instrument accuracy is given in recommendation [7], where permissible error is specified as
(2.3)
where c is the limit of permissible error in relative form at the top of the measurement range and d is the value indicating how the relative error grows for lower readouts.
The relationship between the alternatives (2.2) and (2.3) is as follows:
c = b +/ and d =/,
where is the value of a unit of the least significant digit in the readout device of the instrument and is the upper limit of the measurement range (note that it is often but not always the same as fiducial value ).
As instruments age, their errors usually increase. Therefore, for high-precision digital instruments, the limits of permissible errors are usually rated both right after the calibration and for a certain time since the last calibration. An example of such specification is given in Table 1, where the last row specifies the permissible error limits that grow with time after calibration. The second row lists the reference conditions of using the instrument, which also may become less stringent with time as the permissible error grows.
Time after calibration / 24 hours / 3 months / 1 year / 2 yearsTemperature / 23 ± 1C / 23 ± 5 C / 23 ± 5C / 23 ± 5 C
Limits of error / (0.01% + 1) / (0.015% + 1) / (0.02% + 1) / (0.03% + 2)
Table 1. Limits of permissible error of an instrument depending on time from calibration
The last row of Table 1 lists the limits of permissible error of the instrument depending on the time since calibration; these limits are specified in the form (2.2). The second term in each limit specification is given in units of the least significant digit of the readout device and must be recalculated into the measurement units of the measurand. For instance, with the measurement range with the upper value of 15V and with 5 digits in the readout, the value of one unit of the least significant digit is 1mV.
3. Single Measurements
Single measurements are carried out by a single contact of the measuring instrument with the object whose characteristic isbeing measured. The obtained indication becomes the estimate of the value of the measured quantity. The accuracy of this estimate is determined by the accuracy of the instrument used and the measurement conditions.
Measurement instruments are most commonly intended for single measurements. Some instruments are so simple that accuracy of measurements where these instruments are employed is estimated without any calculations. For instance, measurement of distance with a ruler, when the indication is obtained with accuracy of up to one graduation, has measurement error due to rounding and it does not exceed half the graduation.
Single measurements are ubiquitous in manufacturing, quality control, trade, and other societal activities. Usually their accuracy is not estimated explicitly, since it has been estimated during the design of the activity in question (e.g., the development of the manufacturing process) and by the selection of an appropriate measuring instrument. In general, the accuracy of measurement need not be calculated if it is known a-priori to be “sufficient” for its purpose. In all other cases one has to estimate the measurement accuracy.
Note that the selection of appropriate instrument is in itself an important separate task. However, it is outside the scope of the present guide and is not considered here.
3.1 Direct single measurementunder reference conditions
Direct single measurements under reference conditions of the measuring instrument involved are conducted according to the following procedure.
Step 1. Prior to conducting the measurement, check to make sure that the measurement conditions correspond to reference conditions of the measuring instrument used, and that the limits of the intrinsic error of the instrument are known.
Step 2. Perform the measurement, i.e., bring the measuring instrument into contact with the object of measurement and read out the instrument’s indication.
Step 3. Obtain the estimate of the quantity being measured (the measurand). If the instrument has the scale in the units of the measurand, then the instrument indication gives the measurand estimate directly. If the instrument indication is in the number of graduations on the scale, then the indication must be recalculated into the units of the measurand. For this, one needs to know the value of a graduationin the units of the measurand.
Step 4. If the instrument has a certificate from a calibrating laboratory, and the certificate listsactual values of instrument’s indications or corrections to these indications, apply a correction to the measurand estimateobtained in Step 3. Note that the certificate lists corrections only for the numbered points of the scale (i.e., points with corresponding numbers depicted on the scale). If the indication falls in-between numbered points, the correction is computed as an approximation between the corrections of the two neighboring numbered points from either side of the indication (“enclosing numbered points”). Specifically, if the distance of the indication to the closest numbered point is within the quarter of the distance between the enclosing numbered points, use the correction of the closest numbered point. Otherwise, use the average between the corrections of the enclosing numbered points as the correction to the indication.
Note: the above assumes that the corrections are listed in terms of the dimensional values in measurement unit of the measurand. If the corrections are listed in fractions of a graduation of the scale one needs to first translate it into a dimensional value.
Step 5. Estimate the accuracy of the measurement, which in this case is determined by the intrinsic error of the measuring instrument. Because the intrinsic error is usually given in the form of fiducial error, it must be converted into the limit of error. When using an analog instrument, the limit of absolute error Δ is computed as follows: