Lab: Feature Detection

CSC 295 - Computer Vision - Weinman

Summary:: You will implement an algorithm for detecting stable features (or keypoints) at a particular scale.
Due:: Mon 3/8

Deliverables

The Matlab script used to make your comparisons and generate all figures
(10 points) Determinant of feature matrix image (B.2)
(10 points) Observations of determinant (B.3)
(10 points) Trace of feature matrix image (B.5)
(10 points) Observations of trace (B.6)
(10 points) Feature strength image (B.8)
(10 points) Observations of feature strength (B.9)
(10 points) Detection function kpdet.m (D)
(10 points) Feature detection results (E.5)
(10 points) Feature detection observations (E.6)
(10 points) Professionalism of write-up

Extra Credit

(5 points) Feature orientations returned in kpdet.m

Preparation

Begin by loading your venerable cameraman and converting the image to doubles. While we will start by applying a feature detection algorithm to a specific image, you will end up transforming this into a function that detects keypoints. I therefore recommend that you give the image a rather generic name so that your converted result is both meaningful and painless to create.

Exercises

A. Filtering for Detection

The feature detection algorithm detailed in Szeliski Section 4.1 requires Gaussian filters at two different scales: one for calculating the derivatives, and another for averaging the responses used to calculate the features. In the first part we will establish these filters.

You will need to compute Gaussian partial derivatives for the matrix A. Using gkern, create a Gaussian and first derivative kernels of variance 1 (standard deviation is 1, too). Thus, the scale of features we will be isolating will be nearly as small as possible.
You also need a Gaussian kernel to average (blur) the quantities in the matrix A. Create a single Gaussian kernel of variance 1.5² (standard deviation is thus 1.5). We will thus be averaging responses over a slightly larger area for robustness.
Using conv2, create the partial derivatives of the image in each direction, I_x and I_y, with the separable filters you created in A.1. You should ask for responses that are the same size as the original image.
Using conv2, create blurred versions of the entries in A using the larger (separable) kernel you created in A.2. That is,
- calculate I_x² and then blur it with the kernel
- calculate I_y² and blur it with the kernel, and
- calculate I_xI_y and blur it with the kernel.
Remember that your multiplication should be component-wise! Likewise here, your convolution results should be the same size as the input.

You should now have the three entries for the (blurred) matrix

I_x²

I_xI_y

I_y².

B. Calculating Feature Strength

Recall that the determinant of a 2×2 matrix

is given by ad-bc and the trace of the matrix is the sum of the diagonal entries: a+d.

Calculate an image/matrix where each entry contains the determinant of A using the values computed in A.4.
Display this image. It may be easier to visualize with a hot-cool colormap, which you can add with the command colormap(jet). You can add a colorbar to help you visualize the magnitudes if you wish. Save the result.
Where are the responses the strongest (i.e., at what sorts of images features)? Can you explain this based on quantities used to calculate the determinant?
Calculate an image/matrix where each entry contains the trace of A using the values computed in A.4.
Display this image. It, too, may be easier to visualize with the jet colormap. You can add a colorbar to help you visualize the magnitudes, if you wish. Save the result.
Where are the responses the strongest (i.e., at what sorts of images features)? Can you explain this based on quantities used to calculate the trace?
Calculate an image/matrix containing the final measure of feature strength using Szeliski and Brown's quantity, the ratio of the determinant and the trace,

det
A

tr A
.
Display this image. Once again, it may be easier to visualize with the jet colormap. You can add a colorbar to help you visualize the magnitudes, if you wish. Save the result.
Where are the responses the strongest (i.e., at what sorts of images features)? Can you explain this based on the previous two images?

C. Finalizing Feature Detection

Selecting features are based on two criteria. First, they must be local maxima. That is, we want to chose the locally best responses of our interest measure. Second, they should be sufficiently strong. A local maxima that displays no response is probably not worth examining. In this section, we'll complete these two steps.

Based on the values from the image B.8, choose a value to threshold your feature detection. My suggestion would be that you keep it rather low, making it just high enough to eliminate the truly flat areas. After all, we're trying to match images and we'll want as many potential features as possible. If you're unsure of your choice, feel free to ask.
Next we need to find the locations that are local maxima. I have provided a Matlab function maxima that does this by finding locations in an image that are larger than all four of their neighbors

M = maxima(X);

where M is a binary (logical) matrix the same size as X. Use this to find the local maxima of the feature strength from B.7.
Combine your results from the previous two exercises into one binary (logical) matrix the same size as your image. The location of a non-zero index in the matrix should indicate where there is a maxima that is above the threshold.

D. Making a Detection Function

As promised, it is time to abstract your commands into a Matlab function that you will be using in subsequent labs.

Open a file called kpdet.m in your Matlab editor of choice.
Add a declaration line to your function so that it takes one parameter, an image, and returns one value, the logical detection image.
Copy/convert the commands you used in Parts A, B, and C to create the detection image using the function input, so that it can be returned by your function.

E. Estimating Repeatability

Load two different images of the same scene from the MathLAN:

/home/weinman/courses/CSC295/images/kpdet1.tif
/home/weinman/courses/CSC295/images/kpdet2.tif

Don't forget to convert them to doubles for processing.
Use your kpdet function to find the keypoints in each of these images.
Use find to locate the rows and columns of the keypoints detected in each of these images.
Display both of these images and use hold on to indicate that you want to add more to each figure.
Use plot to display plus symbols where your features are located in each image. (Don't forget that plot takes x and y coordinates, which are the reverse of rows and columns.) Save these image.
How do the feature locations look? Are they in the places you expect? Characterize the repeatability of the feature detection algorithm. (That is, how well does it find the same features in both images?)
Note: If you are unsatisfied with your results, you may want to tweak your threshold, though it is by no means required.

Extra Credit

Alter your method kpdet so that instead of returning a logical matrix, the values where the features are detected contain the gradient orientation of the image at that point. In order to do this robustly, the derivatives should be taken at a larger scale. Thus, you should blur the partial derivatives with a larger Gaussian of variance 4.5² before calculating the orientation. This would potentially allow us to orient a feature descriptor based on the local orientation of the image surface.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.