Measurements and Error Analysis (2024)

"It is better to be roughly right than precisely wrong." — Alan Greenspan

The Uncertainty of Measurements

Some numerical statements are exact: Mary has 3 brothers, and 2 + 2 = 4. However, all measurements have some degree of uncertainty that may come from a variety of sources. The process of evaluating the uncertainty associated with a measurement result is often called uncertainty analysis or error analysis.The complete statement of a measured value should include an estimate of the level ofconfidence associated with the value. Properly reporting an experimental result alongwith its uncertainty allows other people to make judgments about the quality of theexperiment, and it facilitates meaningful comparisons with other similar values or atheoretical prediction. Without an uncertainty estimate, it is impossible to answer thebasic scientific question: "Does my result agree with a theoretical prediction or resultsfrom other experiments?" This question is fundamental for deciding if a scientifichypothesis is confirmed or refuted.When we make a measurement, we generally assume that some exact or true value exists based on how we define what is being measured. While we may never know this true value exactly, we attempt to find this ideal quantity to the best of our ability with thetime and resources available. As we make measurements by different methods, or even when making multiple measurements using the same method, we may obtain slightly different results. So how do we report our findings for our best estimate of this elusive true value? The most common way to show the range of values that we believe includesthe true value is:

Let's take an example. Suppose you want to find the mass of a gold ring that youwould like to sell to a friend. You do not want to jeopardize your friendship, so you wantto get an accurate mass of the ring in order to charge a fair market price. You estimate themass to be between 10 and 20 grams from how heavy it feels in your hand, but this is nota very precise estimate. After some searching, you find an electronic balance that gives amass reading of 17.43 grams. While this measurement is much more precise than theoriginal estimate, how do you know that it is accurate, and how confident are you thatthis measurement represents the true value of the ring's mass? Since the digital display ofthe balance is limited to 2 decimal places, you could report the mass as

m = 17.43 ± 0.01 g.

Suppose you use the same electronic balance and obtain several more readings: 17.46g, 17.42 g, 17.44 g, so that the average mass appears to be in the range of

17.44 ± 0.02 g.

By now you may feel confident that you know the mass of this ring to the nearesthundredth of a gram, but how do you know that the true value definitely lies between17.43 g and 17.45 g? Since you want to be honest, you decide to use another balance thatgives a reading of 17.22 g. This value is clearly below the range of values found on thefirst balance, and under normal circ*mstances, you might not care, but you want to be fairto your friend. So what do you do now? The answer lies in knowing something about theaccuracy of each instrument.To help answer these questions, we should first define the terms accuracy and precision:

Accuracy is the closeness of agreement between a measured value and a true or accepted value. Measurement error is the amount of inaccuracy.

Precision is a measure of how well a result can be determined (without reference to a theoretical or true value). It is the degree of consistency and agreement among independent measurements of the same quantity; also the reliability or reproducibility of the result.

The uncertainty estimate associated with a measurement should account for both the accuracy and precision of the measurement.

Note: Unfortunately the terms error and uncertainty are often used interchangeably todescribe both imprecision and inaccuracy. This usage is so common that it is impossibleto avoid entirely. Whenever you encounter these terms, make sure you understandwhether they refer to accuracy or precision, or both.Notice that in order to determine the accuracy of a particular measurement, we haveto know the ideal, true value. Sometimes we have a "textbook" measured value, which iswell known, and we assume that this is our "ideal" value, and use it to estimate theaccuracy of our result. Other times we know a theoretical value, which is calculated frombasic principles, and this also may be taken as an "ideal" value. But physics is anempirical science, which means that the theory must be validated by experiment, and notthe other way around. We can escape these difficulties and retain a useful definition ofaccuracy by assuming that, even when we do not know the true value, we can rely on thebest available accepted value with which to compare our experimental value.For our example with the gold ring, there is no accepted value with which to compare,and both measured values have the same precision, so we have no reason to believe onemore than the other. We could look up the accuracy specifications for each balance asprovided by the manufacturer (the Appendix at the end of this lab manual contains accuracy data for most instruments you will use), but the best way to assess the accuracyof a measurement is to compare with a known standard. For this situation, it may bepossible to calibrate the balances with a standard mass that is accurate within a narrowtolerance and is traceable to a primary mass standard at the National Institute ofStandards and Technology (NIST). Calibrating the balances should eliminate thediscrepancy between the readings and provide a more accurate mass measurement.Precision is often reported quantitatively by using relative or fractional uncertainty:

Example:

m = 75.5 ± 0.5 g

has a fractional uncertainty of:

0.5 g

75.5 g

=0.006 = 0.7%.

Accuracy is often reported quantitatively by using relative error:

If the expected value for m is 80.0 g, then the relative error is:

75.5 − 80.0

80.0

=−0.056 = −5.6%

Note: The minus sign indicates that the measured value is less than the expectedvalue.

When analyzing experimental data, it is important that you understand the difference between precision and accuracy. Precision indicates the quality of the measurement, without any guarantee that the measurement is "correct." Accuracy, on the other hand, assumes that there is an ideal value, and tells how far your answer is from that ideal, "right" answer. These concepts are directly related to random and systematic measurement errors.

Types of Errors

Measurement errors may be classified as either random or systematic, depending on how the measurement was obtained (an instrument could cause a random error in one situation and a systematic error in another).

When making careful measurements, our goal is to reduce as many sources of error as possible and to keep track of those errors that we can not eliminate. It is useful to know the types of errors that may occur, so that we may recognize them when they arise. Common sources of error in physics laboratory experiments:

Estimating Experimental Uncertainty for a Single Measurement

Any measurement you make will have some uncertainty associated with it, no matter the precision of your measuring tool. So how do you determine and report this uncertainty?

For example, if you are trying to use a meter stick to measure the diameter of a tennis ball, the uncertainty might be

± 5 mm,

but if you used a Vernier caliper, the uncertainty could be reduced to maybe

± 2 mm.

The limiting factor with the meter stick is parallax,while the second case is limited by ambiguity in the definition of the tennis ball's diameter (it's fuzzy!). In both of these cases, the uncertainty is greater than the smallest divisions marked on the measuring tool (likely 1 mm and 0.05 mm respectively). Unfortunately, there is no general rule for determining the uncertainty in all measurements. The experimenter is the one who can best evaluate and quantify the uncertainty of a measurement based on all the possible factors that affect the result. Therefore, the person making the measurement has the obligation to make the best judgment possible and report the uncertainty in a way that clearly explains what the uncertainty represents:

where the ± standard uncertainty indicates approximately a 68% confidence interval (see sections on Standard Deviation and Reporting Uncertainties).
Example: Diameter of tennis ball =

6.7 ± 0.2 cm.

Estimating Uncertainty in Repeated Measurements

Suppose you time the period of oscillation of a pendulum using a digital instrument (that you assume is measuring accurately) and find: T = 0.44 seconds. This single measurement of the period suggests a precision of ±0.005 s, but this instrument precision may not give a complete sense of the uncertainty. If you repeat the measurement several times and examine the variation among the measured values, you can get a better idea of the uncertainty in the period. For example, here are the results of 5 measurements, in seconds: 0.46, 0.44, 0.45, 0.44, 0.41.

For this situation, the best estimate of the period is the average, or mean.

Consider, as another example, the measurement of the width of a piece of paper usinga meter stick. Being careful to keep the meter stick parallel to the edge of the paper (toavoid a systematic error which would cause the measured value to be consistently higherthan the correct value), the width of the paper is measured at a number of points on thesheet, and the values obtained are entered in a data table. Note that the last digit is only arough estimate, since it is difficult to read a meter stick to the nearest tenth of a millimeter(0.01 cm).

This average is the best available estimate of the width of the piece of paper, but it iscertainly not exact. We would have to average an infinite number of measurements toapproach the true mean value, and even then, we are not guaranteed that the mean value isaccurate because there is still some systematic error from the measuring tool, which cannever be calibrated perfectly. So how do we express the uncertainty in our average value?One way to express the variation among the measurements is to use the averagedeviation. This statistic tells us on average (with 50% confidence) how much theindividual measurements vary from the mean.

However, the standard deviation is the most common way to characterize the spreadof a data set. The standard deviation is always slightly greater than the average deviation, and is used because of its association with the normal distribution that is frequently encountered in statistical analyses.

Standard Deviation

To calculate the standard deviation for a sample of N measurements:

• 1

  Sum all the measurements and divide by N to get the average, or mean.

• 2

  Now, subtract this average from each of the N measurements to obtain N "deviations".

• 3

  Square each of these N deviations and add them all up.

• 4

  Divide this result by

  (N − 1)

  and take the square root.

We can write out the formula for the standard deviation as follows. Let the N measurements be called x₁, x₂, ..., x_N. Let the average of the N values be called

x.

Theneach deviation is given by

δx_i = x_i − x, for i = 1, 2, , N.

See Also

Physics Practical Skills Part 3: Systematic VS Random Errors...Gross Errors: Definition, Examples, How To Reduce It, and UsesStatistics in Analytical Chemistry - Stats (4)

The standard deviation is:

In our previous example, the average width

x

is 31.19 cm. The deviations are:The average deviation is:

d = 0.086 cm.

The standard deviation is:

s =

(0.14)2 + (0.04)2 + (0.07)2 + (0.17)2 + (0.01)2 5 − 1

= 0.12 cm.

The significance of the standard deviation is this: if you now make one more measurement using the same meter stick, you can reasonably expect (with about 68% confidence) that the new measurement will be within 0.12 cm of the estimated average of 31.19 cm. In fact, it is reasonable to use the standard deviation as the uncertainty associated with this single new measurement. However, the uncertainty of the average value is the standard deviation of the mean, which is always less than the standard deviation (see next section).Consider an example where 100 measurements of a quantity were made. The average or mean value was 10.5 and the standard deviation was s = 1.83. The figure below is a histogram of the 100 measurements, which shows how often a certain range of valueswas measured. For example, in 20 of the measurements, the value was in the range 9.5 to 10.5, and most of the readings were close to the mean value of 10.5. The standard deviation s for this set of measurements is roughly how far from the average value most of the readings fell. For a large enough sample, approximately 68% of the readings willbe within one standard deviation of the mean value, 95% of the readings will be in the interval

x ± 2 s,

and nearly all (99.7%) of readings will lie within 3 standard deviations from the mean. The smooth curve superimposed on the histogram is the gaussian or normal distribution predicted by theory for measurements involving random errors. As more and more measurements are made, the histogram will more closely follow the bellshaped gaussian curve, but the standard deviation of the distribution will remain approximately the same.

Standard Deviation of the Mean (Standard Error)

When we report the average value of N measurements, the uncertainty we shouldassociate with this average value is the standard deviation of the mean, often called thestandard error (SE).

The standard error is smaller than the standard deviation by a factor of

1/

N

.

This reflects the fact that we expect the uncertainty of the average value to get smaller whenwe use a larger number of measurements, N. In the previous example, we find the standard error is 0.05 cm, where we have divided the standard deviation of 0.12 by

5

.

The final result should then be reported as:

Average paper width = 31.19 ± 0.05 cm.

Anomalous Data

The first step you should take in analyzing data (and even while taking data) is to examine the data set as a whole to look for patterns and outliers. Anomalous data pointsthat lie outside the general trend of the data may suggest an interesting phenomenon thatcould lead to a new discovery, or they may simply be the result of a mistake or randomfluctuations. In any case, an outlier requires closer examination to determine the cause ofthe unexpected result. Extreme data should never be "thrown out" without clearjustification and explanation, because you may be discarding the most significant part ofthe investigation! However, if you can clearly justify omitting an inconsistent data point,then you should exclude the outlier from your analysis so that the average value is notskewed from the "true" mean.

Fractional Uncertainty Revisited

When a reported value is determined by taking the average of a set of independent readings, the fractional uncertainty is given by the ratio of the uncertainty divided by the average value. For this example,

Note that the fractional uncertainty is dimensionless but is often reported as a percentage or in parts per million (ppm) to emphasize the fractional nature of the value. A scientist might also make the statement that this measurement "is good to about 1 part in 500" or "precise to about 0.2%".The fractional uncertainty is also important because it is used in propagating uncertainty in calculations using the result of a measurement, as discussed in the next section.

Propagation of Uncertainty

Suppose we want to determine a quantity f, which depends on x and maybe several other variables y, z, etc. We want to know the error in f if we measure x, y, ... with errors σ_x, σ_y, ...Examples:

For a single-variable function f(x), the deviation in f can be related to the deviation inx using calculus:

Thus, taking the square and the average:

and using the definition of σ, we get:

Examples:(a)

f =

x

(b)

f = x²

(c)

f = cos θ

In the case where f depends on two or more variables, the derivation above can berepeated with minor modification. For two variables, f(x, y), we have:

The partial derivative

∂f

∂x

means differentiating f with respect to x holding the othervariables fixed. Taking the square and the average, we get the law of propagation ofuncertainty:

If the measurements of x and y are uncorrelated, then

δx δy = 0,

and we get:

Examples:(a)

f = x + y

(b)

f = xy

Dividing the previous equation by f = xy, we get:

(c)

f = x/y

Dividing the previous equation by

f = x/y,

we get:

Note that the relative uncertainty in f, as shown in (b) and (c) above, has the sameform for multiplication and division: the relative uncertainty in a product or quotientdepends on the relative uncertainty of each individual term.Example: Find uncertainty in v, where

v = at

with a = 9.8 ± 0.1 m/s², t = 1.2 ± 0.1 s

Notice that the relative uncertainty in t (2.9%) is significantly greater than the relativeuncertainty for a (1.0%), and therefore the relative uncertainty in v is essentially the sameas for t (about 3%).Graphically, the RSS is like the Pythagorean theorem:

The total uncertainty is the length of the hypotenuse of a right triangle with legs the length of each uncertainty component.

The Upper-Lower Bound Method of Uncertainty Propagation

An alternative, and sometimes simpler procedure, to the tedious propagation ofuncertainty law is the upper-lower bound method of uncertainty propagation. Thisalternative method does not yield a standard uncertainty estimate (with a 68% confidenceinterval), but it does give a reasonable estimate of the uncertainty for practically anysituation. The basic idea of this method is to use the uncertainty ranges of each variable tocalculate the maximum and minimum values of the function. You can also think of thisprocedure as examining the best and worst case scenarios. For example, suppose youmeasure an angle to be: θ = 25° ± 1° and you needed to find f = cos θ, then:

Note that even though θ was only measured to 2 significant figures, f is known to 3figures. By using the propagation of uncertainty law:

σ_f = |sin θ|σ_θ = (0.423)(π/180) = 0.0074

(same result as above).

The upper-lower bound method is especially useful when the functional relationshipis not clear or is incomplete. One practical application is forecasting the expected range inan expense budget. In this case, some expenses may be fixed, while others may beuncertain, and the range of these uncertain terms could be used to predict the upper andlower bounds on the total expense.

Significant Figures

The number of significant figures in a value can be defined as all the digits betweenand including the first non-zero digit from the left, through the last digit. For instance,0.44 has two significant figures, and the number 66.770 has 5 significant figures. Zeroesare significant except when used to locate the decimal point, as in the number 0.00030,which has 2 significant figures. Zeroes may or may not be significant for numbers like1200, where it is not clear whether two, three, or four significant figures are indicated. To avoid this ambiguity, such numbers should be expressed in scientific notation to (e.g.1.20 × 10³ clearly indicates three significant figures).When using a calculator, the display will often show many digits, only some of whichare meaningful (significant in a different sense). For example, if you want to estimate thearea of a circular playing field, you might pace off the radius to be 9 meters and use theformula: A = πr². When you compute this area, the calculator might report a value of254.4690049 m². It would be extremely misleading to report this number as the area ofthe field, because it would suggest that you know the area to an absurd degree ofprecision—to within a fraction of a square millimeter! Since the radius is only known toone significant figure, the final answer should also contain only one significant figure:Area = 3 × 10² m².From this example, we can see that the number of significant figures reported for avalue implies a certain degree of precision. In fact, the number of significant figuressuggests a rough estimate of the relative uncertainty:

To understand this connection more clearly, consider a value with 2 significantfigures, like 99, which suggests an uncertainty of ±1, or a relative uncertainty of ±1/99 = ±1%. (Actually some people might argue that the implied uncertainty in 99 is ±0.5 since the range of values that would round to 99 are 98.5 to 99.4. But since the uncertainty hereis only a rough estimate, there is not much point arguing about the factor of two.) Thesmallest 2-significant figure number, 10, also suggests an uncertainty of ±1, which in this case is a relative uncertainty of ±1/10 = ±10%. The ranges for other numbers of significant figures can be reasoned in a similar manner.

Use of Significant Figures for Simple Propagation of Uncertainty

By following a few simple rules, significant figures can be used to find theappropriate precision for a calculated result for the four most basic math functions, allwithout the use of complicated formulas for propagating uncertainties.

Example:

	6.6
×	7328.7
	48369.42	=	48 × 103

(2 significant figures)

(5 significant figures)

(2 significant figures)

Examples:

	223.64			5560.5
+	54		+	0.008
	278			5560.5

If a calculated number is to be used in further calculations, it is good practice to keepone extra digit to reduce rounding errors that may accumulate. Then the final answershould be rounded according to the above guidelines.

Uncertainty, Significant Figures, and Rounding

For the same reason that it is dishonest to report a result with more significant figuresthan are reliably known, the uncertainty value should also not be reported with excessiveprecision.For example, it would be unreasonable for a student to report a result like:

The uncertainty in the measurement cannot possibly be known so precisely! In mostexperimental work, the confidence in the uncertainty estimate is not much better thanabout ±50% because of all the various sources of error, none of which can be knownexactly. Therefore, uncertainty values should be stated to only one significant figure (orperhaps 2 sig. figs. if the first digit is a 1).

To help give a sense of the amount of confidence that can be placed in the standarddeviation, the following table indicates the relative uncertainty associated with thestandard deviation for various sample sizes. Note that in order for an uncertainty value tobe reported to 3 significant figures, more than 10,000 readings would be required tojustify this degree of precision!*The relative uncertainty is given by the approximate formula:

σσ

σ

=

1

2(N − 1)

When an explicit uncertainty estimate is made, the uncertainty term indicates how many significant figures should be reported in the measured value(not the other way around!). For example, the uncertainty in the density measurementabove is about 0.5 g/cm³, so this tells us that the digit in the tenths place is uncertain, andshould be the last one reported. The other digits in the hundredths place and beyond areinsignificant, and should not be reported:

measured density = 8.9 ± 0.5 g/cm³.

RIGHT!

In most instances, this practice of rounding an experimental result to be consistentwith the uncertainty estimate gives the same number of significant figures as the rulesdiscussed earlier for simple propagation of uncertainties for adding, subtracting,multiplying, and dividing.

Combining and Reporting Uncertainties

In 1993, the International Standards Organization (ISO) published the first officialworldwide Guide to the Expression of Uncertainty in Measurement. Before this time,uncertainty estimates were evaluated and reported according to different conventionsdepending on the context of the measurement or the scientific discipline. Here are a fewkey points from this 100-page guide, which can be found in modified form on the NIST website.When reporting a measurement, the measured value should be reported along with anestimate of the total combined standard uncertainty

U_c

of the value. The totaluncertainty is found by combining the uncertainty components based on the two types ofuncertainty analysis:The individual uncertainty components u_i should be combined using the law ofpropagation of uncertainties, commonly called the "root-sum-of-squares" or "RSS"method. When this is done, the combined standard uncertainty should be equivalent to thestandard deviation of the result, making this uncertainty value correspond with a 68% confidence interval. If a wider confidence interval is desired, the uncertainty can bemultiplied by a coverage factor (usually k = 2 or 3) to provide an uncertainty range thatis believed to include the true value with a confidence of 95% (for k = 2) or 99.7% (for k = 3). If a coverage factor is used, there should be a clear explanation of its meaning sothere is no confusion for readers interpreting the significance of the uncertainty value.You should be aware that the ± uncertainty notation may be used to indicate differentconfidence intervals, depending on the scientific discipline or context. For example, apublic opinion poll may report that the results have a margin of error of ±3%, which means that readers can be 95% confident (not 68% confident) that the reported results areaccurate within 3 percentage points. Similarly, a manufacturer's tolerance ratinggenerally assumes a 95% or 99% level of confidence.

Conclusion: "When do measurements agree with each other?"

We now have the resources to answer the fundamental scientific question that was asked at the beginning of this error analysis discussion: "Does my result agree with a theoretical prediction or results from other experiments?"Generally speaking, a measured result agrees with a theoretical prediction if the prediction lies within the range of experimental uncertainty. Similarly, if two measured values have standard uncertainty ranges that overlap, then the measurements are said to be consistent (they agree). If the uncertainty ranges do not overlap, then themeasurements are said to be discrepant (they do not agree). However, you should recognize that these overlap criteria can give two opposite answers depending on the evaluation and confidence level of the uncertainty. It would be unethical to arbitrarilyinflate the uncertainty range just to make a measurement agree with an expected value. Abetter procedure would be to discuss the size of the difference between the measured andexpected values within the context of the uncertainty, and try to discover the source of thediscrepancy if the difference is truly significant. To examine your own data, you areencouraged to use the Measurement Comparison tool available on the lab website.Here are some examples using this graphical analysis tool:

A = 1.2 ± 0.4

B = 1.8 ± 0.4

These measurements agree withintheir uncertainties, despite the fact thatthe percent difference between theircentral values is 40%.However, with half the uncertainty ± 0.2, these same measurements do not agree since theiruncertainties do not overlap. Further investigationwould be needed to determine the cause for thediscrepancy. Perhaps the uncertainties wereunderestimated, there may have been a systematicerror that was not considered, or there may be a true difference between these values.

An alternative method for determining agreement between values is to calculate thedifference between the values divided by their combined standard uncertainty. This ratiogives the number of standard deviations separating the two values. If this ratio is lessthan 1.0, then it is reasonable to conclude that the values agree. If the ratio is more than2.0, then it is highly unlikely (less than about 5% probability) that the values are thesame.Example from above with

u = 0.4:

|1.2 − 1.8|

0.57

= 1.1.

Therefore, A and B likely agree.Example from above with

u = 0.2:

|1.2 − 1.8|

0.28

= 2.1.

Therefore, it is unlikely that Aand B agree.

References

Baird, D.C. Experimentation: An Introduction to Measurement Theory and ExperimentDesign, 3^rd. ed. Prentice Hall: Englewood Cliffs, 1995.Bevington, Phillip and Robinson, D. Data Reduction and Error Analysis for the PhysicalSciences, 2^nd. ed. McGraw-Hill: New York, 1991.ISO. Guide to the Expression of Uncertainty in Measurement. International Organizationfor Standardization (ISO) and the International Committee on Weights and Measures(CIPM): Switzerland, 1993.Lichten, William. Data and Error Analysis., 2^nd. ed. Prentice Hall: Upper Saddle River,NJ, 1999.NIST. Essentials of Expressing Measurement Uncertainty. http://physics.nist.gov/cuu/Uncertainty/Taylor, John. An Introduction to Error Analysis, 2^nd. ed. University Science Books:Sausalito, 1997.

Copyright © 2011 Advanced Instructional Systems, Inc. and the University of North Carolina | Credits