Floating Point Number Representation

Goals

This lab provides experience viewing the representation of floating-point real numbers on PC/Linux machines, and explores an application for which numerical round-off error has visible consequences.

Binary Representation of Floating-Point Numbers:

The first part of this lab asks you to review the bit-level storage of floating point numbers on PC/Linux computers.

Write the real numbers ± 1, ± 2, ± 3, ± 6, ± 9 using the IEEE Standard for 32-bit Floating Point Numbers..
Copy the program data-rep.c to your account and compile it. Then enter real numbers, and conduct experiments to determine:
- which bit is the sign bit,
- which bits are used for the mantissa,
- which bits are used for the exponent, and
- what bias or excess is used in the storage of exponents.
When the decimal number 0.1 (one tenth) is converted to binary, the resulting floating-point number is the repeating sequence 0.0001100110011 (just as the decimal representation of one third is the repeating sequence 0.333).

Use data-rep.c to determine the floating-point number that is actually stored for the decimal number 0.1 (one tenth). Note how this differs from the actual binary number.
Use your knowledge of the storage of real numbers to determine what real number comes "immediately after" 3.0 and 10.0 on this system. That is, look at the mantissa to determine what change would yield the smallest number above 3.0 and 10.0.

Hint: When running the data-rep program, toggle an appropriate bit and look at what results.

Floating-point Numbers and Loops

Inaccuracies in representing floating-point numbers with a limited number of digits of accuracy have an impact on how programs are written and how they run. This section of the lab explores some of these consequences.

Copy float-loop.c to your account. The idea of this program is to work through a loop, starting at 0.0, incrementing by 0.1 each time through the loop, and continuing while the number is not 1.0.
1. Read through this program. Write out what should be printed (including the expected value of sum to be printed each time).
2. Run this program, and describe what happens.
  Note: You can stop any running program by typing ctrl+c.
3. Review the first part of the output printed to determine why the program ran the way it did.
  
  Hint: After compiling the program, you might use the following line to run the program and look at the first several lines of output:
```
./float-loop | head -n 20
```
4. Change the loop condition to (val <= end). Again, explain what happens. What is the last value printed within the loop? What sum is actually computed?
Change float-loop.c so that the variables are declared as double rather than float, and repeat Steps 5a-d.
- What happens this time?
- Why?
The program float-loop.c illustrates that loops may or may not repeat the number of times expected, when the variables within the loop condition are floating-point numbers. One common way to resolve this problem is to change the loop control variables to an integer. For example, for float-loop.c, we could use an int variable iter to control the loop. Effectively, iter has the value of 10 times the value we intend for val. The main loop might be
```
for (int iter = 0; iter <=10; iter++)
{
   val = iter / 10.0;
   ...

}
```
Here, the int iter is always computed exactly, so the loop always runs exactly the desired number of times, and the value of val is recomputed from the exact number i each time so inaccuracies in the storage of 0.1 do not compound.

Rewrite float-loop.c to replace the while (val < end) loop with a for construction using an integer as the loop control variable. Then run the program to confirm it produces the desired output.

More Floating-point Numbers and Loops

Your experience with float-loop.c illustrates that some issues arising with float numbers may be resolved with double numbers. The extra digits of accuracy sometimes can make a substantial difference. This section explores this observation further.

Program double-loop.c is similar to float-loop.c, except that its variables are double, the range of numbers for the loop is 1000 to 1001, and the condition is val <= end as in Step 4c. Copy this program to your account.
1. Read through this program. Write what should be printed (including the expected value of sum to be printed each time).
2. Run this program and describe what happens.
3. What, if anything, happens if the variables are changed to float type?

Computing Area Under y = x²:

[The following is an edited version of Section 5.5 from Introduction to Computing and Computer Science with Pascal by Henry M. Walker, Little, Brown, and Company, 1986 and is used with permission of the copyright holder.]

Suppose we are given a function y = f(x), and we want to find the area under the graph between x = a and x = b.
(The following figure illustrates the area under the curve between x = 1 and x = 3 when f(x) = x².)

Area under f(x) = x^2 between x=1 and x=3

Using calculus, the exact size of this area is 8 2/3 or 8.666.

Discussion

In what follows, we will not try to compute the desired area exactly. Rather, we will consider a fairly simple approach, called the trapezoidal rule, which can give good approximations to the area. In this approach, we break down a large area into small pieces and approximate each of the small pieces by a trapezoid (as shown below).

From geometry, we we can compute the area of a trapezoid:

Then we can approximate the entire area under the curve by adding up the areas of the trapezoids.

More precisely, we first divide the interval [a, b] into n equal pieces a=x₀, x₁, x₂, . . ., x_n=b. Then we use the pieces to divide the overall areas into trapezoids. After we compute the area of each trapezoids, we add up these small areas. The final formula is

Approximate Area = h[f(x₀)/2 + f(x₁) + f(x₂) + . . . + f(x_n-1) + f(x_n)/2)]

where h = (b - a) / n and x_j = a + jh for j = 0, 1, 2, . ., n. This is the formula trapezoidal rule. (The interested reader should consult books in calculus or numerical methods for the details of this and other methods.)

To make this formula more concrete, we apply it to f(x) = x² between x = 1 and x = 3 (as shown in an earlier figure), and we divide the interval ]1, 3] into five pieces. This gives: n = 5; a = 1; b = 3. The overall interval [1, 3] has length 2; we divide it into five subintervals of length h = 2/5 = 0.4. The x values are x₀ = 1, x₁ = 1.4, x₂ = 1.8, x₃ = 2.2, x₄ = 2.6, x₅ = 3. The trapezoidal rule gives:

Approx. Area	= h[f(x₀)/2 + f(x₁) + f(x₂)+ f(x₃)+ f(x₄)+ f(x₅)/2)]
	= 0.4[f(1)/2 + f(1.4) + f(1.8) + f(2.2) + f(2.6) + f(3)/2]
	= 0.4]1²/2 + (1.4)² + (1.8)² + (2.2)² + (2.6)² + 3²/2]
	= 8.72

Theoretical Accuracy of the Trapezoidal Rule

While it is hard to predict the accuracy of approximations with the trapezoidal rule, we can make several useful observations.

The trapezoidal rule relies upon the actual area under the graph being close to the area under the trapezoid.
If the graph of the function is a straight line, then the trapezoids should give exact results. Otherwise the trapezoidal rule cannot be expected to be exactly correct.
If we divide the interval [a, b] into a large number of pieces, we can expect each trapezoid to be close to the actual area under the graph.
As n gets bigger, the approximation of area using the trapezoidal rule should get better.

Practical Implications of Floating Point Error

Since floating-point numbers are not stored exactly, work with any individual floating point number may involve a small amount of error. If these numbers are combined in many arithmetic operations, such small numerical errors sometimes can come together to significantly affect results.

Programming

This part of the lab asks you to run and expand program trap-rule.c that computes area using the trapezoidal rule. You then will experiment with this program to investigate the effect of numerical errors.

Copy trap-rule.c to your account, and then compile and run it.
1. Review the program and describe how it works. For example, how the table is produced? Why does the function area_l_to_r use the variable i? Why does the computation for xvalue give appropriate values for x values in the trapezoidal rule?
2. As noted above, the correct value of this area is 8 2/3 or 8.666 as determined with calculus. Discuss how the computed approximations compare to this exact value as the number of intervals increases.

The function area_l_to_r adds terms in the Trapezoidal Rule from first to last. For the function given, the terms get steadily larger as the function is increasing from left to right. A natural question arises regarding what might happen if the terms were added in the opposite order.

Modify the program to include another function area_r_to_l, which adds the terms in the Trapezoidal Rule from last to first (i.e., from the nth term toward the initial term). Then, in the main loop, add another column to the table, for "Computing from R to L".
1. Run the revised program, showing the results of both left-to-right and right-to-left computations.
2. Compare the results of the left-to-right and right-to-left computations. What patterns do you observe? What, if any, differences do you identify? Briefly explain what you see.
3. If this lab is to be turned in, include your program for this step as well as your explanations and other work.