Lab: Epipolar Geometry

CSC 295 - Computer Vision - Weinman

Summary:: You will estimate the fundamental matrix and use it to plot the epipoles and epipolar lines in a stereo image pair.
Due:: 4/19

Deliverables

The Matlab script used to make your comparisons and generate all figures
(5 points) Rank of the initial fundamental matrix estimate (C.4)
(10 points) Singular values of the initial fundamental matrix estimate and commentary (C.6)
(5 points) Adjusted fundamental matrix rank (C.8)
(10 points) Image of left point and right epipolar line (D.2)
(10 points) Image of right point and left epipolar line (D.4)
(10 points) Observations on matching along epipolar line (D.5)
(10 points) Image of all right epipolar lines (D.7)
(10 points) Observations of epipolar geometry (D.8)
(10 points) Conclusions about image acquisition (D.9)
(10 points) Professionalism of write-up

Extra Credit

(10 points) Automated point correspondences

Preparation

Load the stereo image pair for this lab from the MathLAN. You need not convert to grayscale or double.

: /home/weinman/courses/CSC295/images/left.jpg
/home/weinman/courses/CSC295/images/right.jpg

Exercises

A. Getting Correspondences

At the outset, we will get point correspondences manually. In the future you may want to see if your MOPS implementation can do it automatically.

Show the left and right images in two separate figures. Use figure(n) where n is the number 1 for the left image and 2 for the right image, so that you get predictable figure numbers.
According to the Matlab documentation, the command "ginput enables you to select points from the figure using the mouse for cursor positioning. The figure must have focus before ginput receives input." For example,

[xx,yy] = ginput(1);

places a cross-hairs on the current figure. Once you click, the coordinates of the click are returned, with the column in xx, and the row in yy. Write a for loop to gather eight points using the following steps:
1. Display the left and right images in their appointed figure windows.
2. Focus the figure containing the left image (use figure(n), where n is the figure number you wish to focus).
3. Allow the user to select a point in the left image.
4. Add the selected point to two vectors, one containing x-coordinates and the other containing y-coordinates of points from the left image.
5. Plot the selected point on the left image using a plus in a color you find easy to see.
6. Focus the figure containing the right image.
7. Allow the user to select a point in the right image.
8. Add the selected point to two similar vectors of corresponding points from the right image.
Use your loop to find eight point correspondences. I recommend you click on corners because they are easier to localize. Try to distribute your points throughout the image. To the extent possible, you should minimize the coplanarity of the points you select.
You may wish to save the points you created in a Matlab workspace so that you can repeat other aspects of the lab without having to do this again. Use save.

B. Constructing the System Matrix

Construct the 8×9 system matrix using the correspondences you found in part A. Recall that each row has the form

[

x_Lx_R,

x_Ly_R,

x_L,

y_Lx_R,

y_Ly_R,

y_L,

x_R,

y_R,

]

C. Estimating the Fundamental Matrix

Apply singular value decomposition (SVD) to your system matrix.
Extract the column from V corresponding to the smallest singular value. (Matlab orders the singular values from largest to smallest for you).
Use reshape to make the 9×1 column into a 3×3 matrix, which is your initial estimate of the fundamental matrix F.
What is the rank of this matrix? (Note: Use Matlab's rank function.)
Apply SVD to your initial estimate of F.
What are the singular values of this estimate? Are the relative magnitudes about what you'd expect?
Set the smallest singular value to zero and reconstitute a new estimate for F from the SVD by multiplying the resulting factors:

F=UWV^T
What is the rank of your adjusted estimate of the fundamental matrix F?

D. Visualizing Epipoles

Recall the equation for the right image epipolar line coefficients can be given by

u_R=F

for an augmented point from the left image. The epipolar line in the right image is then expressed by

T
R

u_R=0,

y_R=-(u₁x_R+u₃)/u₂.

You can also do the reverse: give the right point to provide coefficients (this time in a row vector) for a line in the left image.

Plot a new point (i.e., using a +) in the left image (not one you used to construct the system matrix) and its corresponding epipolar line in the right image.
Hint: Use the first and last column of the image as two x coordinates and calculate the corresponding y coordinates of these for plotting.
Save these two image figures.
Plot another new point in the right image and its epipolar line in the left image.
Save these two image figures.
Do the points you selected in one image appear along the epipolar line in the other image?
Using the same method as D.1, plot the epipolar lines for all eight points in the right image.
Save this image figure.
Do your lines intersect? What can you conclude about the epipolar geometry?
Can you determine whether these two images were taken at the same time? Why or why not?

Extra Credit

Use your MOPS-based feature matching code to automatically find at least 8 point correspondences (you can use more in constructing the system matrix. Include your existing kpfeat.m and kpdet.m and any other relevant functions in your submission and use these in a new procedure that takes two images as input and returns the fundamental matrix estimate.

Full extra credit will be given for those solutions that work correctly in a variety of situations. You may need to further tighten your detection/matching parameters so that you ensure you get at least 8 matches and that all are correct.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.