Numerical Methods for Eng [ENGR 391] [Lyes KADEM 2008]

CHAPTER II

Truncation Errors and the Taylor series

I. Definition

Truncation errors are those that result from using an approximation in place of an exact mathematical procedure.

Example Approximation of the derivative

One of the most important methods used in numerical methods to approximate mathematical functions is: Taylor series

II. Taylor series

The Taylor series provides a means to predict a function at one point in terms of the function value and its derivative at another point.

Therefore, any smooth function can be approximated as a polynomial.

Figure.2.1. Taylor series.

The expression of Taylor series for any function is:

A remainder term is included to account for high order terms neglected (from (n+1) to infinity):

This represents the remainder for the nth-order approximation. x is a value that lies somewhere between xi and xi+1.

It is very convenient to write the Taylor series under a compact form by replacing (xi+1- xi) by the step (h):

and

Although, theoretically an infinite number of terms are needed to yield to an exact estimate, in practice only few terms are sufficient.

The number of terms needed is dependent on the application, the precision needed and it is determined using the remainder term of the expansion.

However, the determination of the remainder term Rn is not straightforward, since we have:

-  To know x

-  To differentiate the function (n+1) times, and for that we need to know the function f !!!!

The only term that we can control in this expression is (h). Therefore, it is very convenient to express Rn as:

Which must be interpreted as a truncation error of order n+1. This means that the error is proportional to hn+1.

Therefore, if h is sufficiently small, an accurate estimate for can be reached using only few terms.

II.1. Using Taylor series to estimate truncation errors

Let us see how Taylor series may be used to estimate truncation errors. If we consider an approximation of the derivative:

How to determine the error introduced by using this formulation to compute the derivative instead of using the real mathematical definition?

Using a Taylor series truncated at the first-order:

Therefore,

We have now an estimate of the truncation error:

Truncation error =

So the order of the error due to the formulation used to compute the derivative is h.

III. Numerical differentiation

Another interesting point using the Taylor series is that we are able to compute the derivative of a function using the values of the function at xi+1 and xi and we know how to compute the error due to this approximation.

The formulation:

Is called first forward difference. Forward because we are using (i) and (i+1)

- First-backward difference approximation

gives:

- Centered difference approximation

and

Will give

Note that the truncation error is of the order of h2, so it is more accurate than the forward and backward formulation.

III.1. Finite difference approximations of higher derivatives

When you look at the Taylor series, you could notice that we can use it to derive any derivative of the function [second; third; …]

Example

Second order derivative:

Forward /
Backward /
Centered /
Note that for the centered formulation, it is a derivation of a derivative:

III.2. Functions of more than one variable

Taylor series can be applied to functions of more than one variable.

For example for a function of two variables:

IV. Error propagation

IV.1. Functions of single variable

In this part, the objective is to estimate the effect of a discrepancy between and on the value of a function, or mathematically speaking, to estimate:

But the problem is that we do not know because we do not know. But, thanks to Taylor series we can estimate using (if it is close to) and is continuous and differentiable:

keeping only the zero and first order:

thus, gives

This allows us to approximate the error in given the derivative of a function and an estimate of the error in the independent variable.

Figure.2.2. Estimation of the error in .

V. Stability and Condition

The condition of a mathematical problem relates to its sensitivity to changes in its input values. We say than a computation is numerically unstable if the uncertainty of the input values is grossly magnified by the numerical method.

We have:

We can compute the relative error on by:

The error on x is:

V.1. Condition number

It can be defined as the ratio between the error on and the error on [from above]:

Condition number (CN) =

The condition number provides a measure of the extent to which an uncertainty in is magnified by

CN = 1 / The relative error on is identical to the relative error on
CN > 1 / The relative error is amplified
CN < 1 / The relative error is attenuated
CN > 1 / Ill-conditioned

VI. Total numerical error

The total error is the sum of the truncation and round-off errors, however:

Round-off ¯ / ­ the number of figures
Round-off ­ / With subtractive cancellation and the increase in the number of computations.
Truncation errors ¯ / ¯ h [step], however this leads to subtractive cancellation or to the increase in computations.

Therefore, we are facing a dilemma: decreasing one component of the total errors increases the other.

However, with actual computers, the round-off errors can be minimized and therefore we will be able to decrease the truncation error by reducing (h).

Figure.2.3. Variation of total error as a function of the step size.

VII. Some advice to control numerical errors

-  Avoid subtracting two nearly equal numbers.

-  Avoid subtractive cancellation by reformulating your problem.

-  When you add or subtract numbers sort them and start with the smallest one.

8

Truncation Errors and the Taylor Series