Skip to main content

Section 6.4 The effect of round-off errors

The round-off errors made by a computer are usually very small: for example,

0.1 + 0.2 - 0.3

produces 5.5511e-17 instead of the expected 0. The underlying reason is that, for example, the number \(0.1\) is internally represented not as \(1/10\) but as a fraction \(p/q\) where \(q\) must be a power of 2 (because the computer arithmetic is binary in nature. Both \(p\) and \(q\) are chosen very large which makes the approximation \(p/q \approx 1/10\) very good for most purposes. But it cannot be perfect, because if \(p/q = 1/10\text{,}\) then \(10p = q\) which is a contradiction: the left hand side is divisible by 5, so the right hand side cannot be a power of 2.

For each of the numbers 0.1, 0.2, 0.3, find a fraction with denominator 32 which is the best approximation to it. Use these approximations, instead of the actual numbers, in the formula \(0.1 + 0.2 - 0.3\text{.}\) What is the result?

Solution

To find approximations, multiply each of the numbers by 32 and round the result to the nearest integer. Thus, \(0.1\approx 3/32\text{,}\) \(0.2\approx 6/32\text{,}\) and \(0.3 \approx 10/32\) (because \(0.3\cdot 32 = 9.6\) rounds to 10). Hence

\begin{equation*} 0.1 + 0.2 - 0.3 \approx \frac{3}{32} + \frac{6}{32} - \frac{10}{32} = -\frac{1}{32} \end{equation*}

The result is not zero due to round-off errors. This is exactly what happens in Matlab computations, except that they involve a much larger power of 2.

The tiny round-off errors have a major impact on the solution of linear systems when, as a result of some row operations, some entry should theoretically be zero but in practice it is not (like \(0.1 + 0.2 - 0.3\) above). So, the process of elimination that was supposed to eliminate a redundant equation, making it \(0=0\text{,}\) instead turns it into some “numerical garbage”. Matlab tries to detect such situations and warn the user about them, indicating that the solution it finds may be useless.