Skip to main content

Section 12.2 Loss of significance

Loss of significance may occur when we subtract two numbers that are very close to each other. This can occur in any context, not just numeric differentiation. But since differentiation involves expressions like \(f(x+h)-f(x)\) with small \(h\text{,}\) it is particularly vulnerable to the loss of significance.

A simplified illustration of the loss of significance: suppose we can only have three decimal digits, any others have to be rounded. This can still be useful for approximate computations like \(9.23 + 3.45 = 12.7\) or \(3.45 + 100 = 103\) (note the rounding here). These approximate results are relatively close to what we would get with exact arithmetic. But when subtraction of nearly equal quantities are involved, the results can be less satisfactory:

\begin{equation} (3.45 + 100) - 102 = 103 - 102 = 1\label{eq-loss-sig1}\tag{12.2.1} \end{equation}

while the exact arithmetic gives \(1.45\text{.}\) The error is nearly \(50\%\text{.}\)

An actual illustration is the following attempt to compute the derivative of \(f(x) = \sqrt{x}\) at the point \(x = 10^6\text{.}\) Note that \(f(10^6) = 1000\) and \(f'(10^6) = 1/2000 = 5\cdot 10^{-4}\text{.}\) Both looks like reasonable numbers. Let us try to use the approximation (12.1.1) with the values \(h = 10^{-9}, 10^{-10}, 10^{-11}\text{,}\) in Matlab console:

>> x = 1e6;
>> h = 1e-9; (sqrt(x+h) - sqrt(x))/h
ans = 5.6843e-04

>> h = 1e-10; (sqrt(x+h) - sqrt(x))/h
ans = 0.0011

>> h = 1e-11; (sqrt(x+h) - sqrt(x))/h
ans = 0

The first result is off by about \(10\%\text{,}\) the second is more than twice the real answer, the third is even worse. All this is due to the loss of significant digits similar to what we see in (12.2.1).

Sometimes one can avoid the loss of significance by rearranging the expression algebraically. For example,

\begin{equation*} \frac{\sqrt{x+h} - \sqrt{x}}{h} = \frac{(\sqrt{x+h} - \sqrt{x})(\sqrt{x+h} + \sqrt{x})}{h(\sqrt{x+h} + \sqrt{x})} = \frac{x+h-x}{h(\sqrt{x+h} + \sqrt{x})} = \frac{1}{\sqrt{x+h} + \sqrt{x}} \end{equation*}

Algebraically these expressions are identical, but in Matlab, 1/(sqrt(x+h) + sqrt(x)) produces accurate answer 5.0000e-04 for any of the above values of \(h\text{.}\)

An important case of such rearrangement in the quadratic formula: its familiar form

\begin{equation} x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}\label{eq-quad-variant-1}\tag{12.2.2} \end{equation}

leads to loss of significance when \(b\) is much larger than \(ac\text{.}\) The following code erroneously returns 0 as a root of the quadratic equation \(x^2+10^9x+1=0\text{.}\)

>> a=1; b=1e9; c=1;
>> (-b+sqrt(b^2-4*a*c))/(2*a)
ans = 0

The alternative form of (12.2.2) is

\begin{equation} x = \frac{2c}{-b \mp \sqrt{b^2 - 4ac}}\label{eq-quad-variant-2}\tag{12.2.3} \end{equation}

where the signs \(\mp\) indicate that the root we would get with \(+\) in (12.2.2) is obtained with \(-\) in (12.2.3). To compute both roots without loss of significance, one uses both formulas (12.2.2) and (12.2.3) with the sign opposite to the sign of \(b\text{.}\)

>> (2*c)/(-b-sqrt(b^2-4*a*c))
ans = -1.0000e-09
>> r = roots([1 1e9 1]); r(2)
ans = -1.0000e-09

As this sample shows, Matlab takes the numerically safe approach in its roots function for finding roots of polynomials.

Matlab also has some built-in logic for avoiding loss of significance in two frequently encountered situations: \(e^{x} - 1\) and \(\log (1+x)\) when \(x\approx 0\text{.}\) These are the functions expm1 and log1p. To see the difference, compare the results of (exp(x)-1)/x and expm1(x)/x when x=1e-20.

Another situation where loss of significance may occur is the evaluation of a polynomial of high degree: Example 21.1.1.