Brigham Young University - Idaho
College of Physical Sciences and Engineering
Department of Mechanical Engineering
Class Prep Notes #4
Computer Precision & Round-Off Error
Introduction
Using computers to solve problems often times involves approximation, thus introducing error into our solutions. For many engineering problems, we cannot obtain analytical solutions and thus use numerical methods to find good approximations. Some questions to ponder concerning error and approximations include the following:
How is error introduced into our calculations?
How much error is present in our calculations?
Is the error present tolerable?
Please consider these questions as you review the following material.
Numerical Error
Numerical error in computer calculations can be classified into two major forms: round-off error and truncation error. Round-off error is due to the fact that computers can only represent quantities with a finite number of digits. Truncation error is introduced because numerical methods employ approximations to represent exact mathematical operations and quantities.
Significant Figures
Significant figures may be defined as those digits of a number that can be used with confidence. Many times in working problems by calculator, students will jot down intermediate answers to a couple of decimal places. Later they will reenter the intermediate answer manually in an additional calculation to arrive at a final answer. By not including all available significant figures, small errors are introduced. Even if intermediate answers are passed directly with all of their significant digits to the final calculations, small errors may be introduced due to the limited number of significant digits available in the calculator or spreadsheet.
Quantities such as pi, e, and the square root of many numbers are irrational numbers and cannot be expressed exactly by a limited number of digits.
The omission of significant figures is called round-off error.
Accuracy and Precision
A brief discussion of the terms accuracy and precision may be useful in understanding computer precision and round-off errors. Accuracy refers to how closely a computed or measured value agrees with the true value. Precision refers to how closely individual computed or measured values agree with each other.
The target analogy shown in Figure 1 illustrates that both accuracy and precision are desired. Suppose the bulls eye represents the true answer to a problem and the bullet holes represent the approximate answers as determined by a numerical calculation. It may be possible to have a numerical technique that produces highly repeatable, precise results that are not very accurate and miss the bulls eye by a substantial amount. Care must be taken to ensure that the chosen numerical techniques produce results that are both accurate and precise.
Figure 1. Accuracy vs. Precision.
Computer Representation of Numbers
A number system is a convention for representing quantities. By far the most common numbering system in use throughout the world is base-10. A base number is the number used as a reference for constructing the number system. Why base-10? I suppose it has a lot to do with the fact that we have 10 fingers and toes. Figure 2 illustrates briefly how the base-10 numbering system works.
Figure 2. Base-10 Numbering System
Digital computers, on the other hand, are on/off electronic devices. As such numbers on computers are represented internally with a binary or base-2 system (See Figure 3). The fundamental unit whereby information is stored is called a word. A word consists of several binary digits or bits. Numbers are typically stored in one or more words.
Figure 3. Base-2 Numbering System.
Integers provide a quick and convenient method for doing many simple calculations. The range of the integer is limited by the number of bits in the word:
8-bit -128 to 127
16-bit -32,768 to 32,767
32-bit -2.15 billion to 2.15 billion
64-bit -9.2E18 to 9.2E18
Most engineering calculations, however, require the use of real numbers. In computer terminology, these are typically referred to as float-point numbers. Float-point numbers are essentially a computer implementation of scientific notation. They consist of a mantissa and exponent. The mantissa is the fractional part of the number, where the exponent is the integer part.
The advantage of floating-point representation over integer representation in a computer is that it can support a much wider range of values. The range of floating-point numbers is shown below:
32-bit 1E-38 to 1E38 7 significant digits
64-bit 1E-308 to 1E308 15-16 significant digits
Floating-point representation also has some disadvantages. For example, floating-point numbers take more memory and longer processing time than integers. More significantly, however, their use introduces a source of error because the mantissa holds only a finite number of significant digits. Thus, round-off error is introduced. It should be noted, however, although round-off errors can be important in contexts such as testing convergence, the number of significant digits carried on most computers allows most engineering computations to be performed with more than acceptable precision.
Excel and Mathcad routinely use double precision (64-bit) to represent numerical quantities. The developers of these packages decided that mitigating round-off errors would take precedence over any loss of speed incurred by using extended precision. Others like Mathlab allow you to use extended precision, if you desire.
Losses of significant digits in floating-point calculations can occur by adding very large and very small numbers, resulting in chopping. Losses also occur during the subtraction of nearly equal numbers. This type of loss is among the greatest source of round-off error in numerical methods.
Even though an individual round-off error may be small, the cumulative effect over the course of a large computation can be significant. There are times when the foregoing examples can be circumvented by using a transformation. However, the only general remedy is to employ extended precision.