Lab: Image Filtering

CSC 295 - Computer Vision - Weinman

Summary:: We compare filtering using separable kernels along with different smoothing kernels and finally use filtering to detect image some interesting image features.
Due:: Fri 2/12

Deliverables

The Matlab script used to make your comparisons and generate all figures
(0 points; EXEMPT: Matlab seems to not be working for this part on the MathLAN) Plot of 2-D Gaussian, A.9
(5 points) Image of 2-D Gaussian, A.11
(10 points) Image and observations of box filtering result, B.2
(10 points) Image and observations of Gaussian filtering result, B.4
(5 points) Answer to B.5
(10 points) Comparison and analysis of separable filtering, B.7
(10 points) Image and observations of correlation result, C.7
(10 points) Correlation detections, C.14
(10 points) Answer to C.15
(15 points) Professionalism of write-up

Preparation

Load the mandrill demo image and convert it from an indexed color image to a grayscale image using the following commands:

load mandrill
X = im2double(ind2gray(X,map));
clear map;
Show the mandrill image so that you know what it looks like before filtering.

Exercises

A. Separability

In this part of the lab we will construct two different 2-D filter kernels using their separable components.

Create a 1×10 boxcar smoothing filter using the ones function. (Be sure it is normalized!)
Verify that the 2-D box filter kernel is separable by creating using Equation (3.20) of Szeliski,

K=vh^T,

to create a 10×10 box filter from your boxcar filter. (Note that he assumes that filters v and h are column vectors, rather than row vectors, so you'll have to adjust the equation ever so slightly.)
The single quote operator ' tells Matlab to take the tranpose of a matrix or vector.
Is the 2-D kernel also normalized?
Create a 1-D Gaussian kernel with a variance of 10 using the provided function gkern (this is not a standard Matlab function; it is in the toolbox you added to Matlab's search path in the first lab):

gauss = gkern(10);
Plot the kernel and verify that it has a 1-D Gaussian appearance. How big is the kernel? Is it normalized?
Create and plot a few Gaussian kernels with other variances to see how the shape changes.
Use the same transformation (called an "outer product") that you used for the box filter to generate a 2-D Gaussian kernel. Is this 2-D version also normalized?
It is worthwhile to visualize your kernel two different ways, as an image and as a 3-D surface plot. Use surf to create a simple 3-D plot of your kernel. Does it have the expected 2-D Gaussian appearance?
Save your plot using print. You do not need to label the axes.
Show your kernel as an image (you will likely need to tell imshow to ignore the conventional range of 1 as white, as you did in the image formation lab, in order to see the complete contrast).
Save this more easily visible image of your 2-D kernel using imwrite and imadjust (as you did in the image formation lab).

B. Smoothing Filters

Now that you have constructed some filters, let us put them to use. The procedure conv2 does the two-dimensional convolutional filtering. As we mentioned in class, convolution has some important caveats to consider when handling border cases. For simplicity, we will usually let the kernel overhang the edge of the image, using zeros. By default conv2 will give a result for every location where the kernel touches the image. It is often more worthwhile to restrict the answer to be the same size as the image. This leads to "incorrect" results at the borders, which we could eliminate by telling the procedure to give us back only "valid" regions where the kernel and the image completely overlap. However, it is often hard to reconcile this smaller result with the locations in the original image.

There are two ways to use conv2-with a regular 2-D kernel, say it is called K, applied to an image, say I. In this case, the filtered result would be given by

: Ifilt = conv2(I,K,'same');

where 'same' gives us back a result of the same size as I, but with a few "invalid" responses near the edges. The other common possibility we'll use is when we have vertical (across rows/along columns) and horizontal (across columns/along rows) filters , say vfilt and hfilt. Matlab only cares that these are vectors, rather than matrices, and not whether they are in fact row or column vectors. The usage then (note the different ordering of arguments) is

: Isepfilt = conv2(vfilt,hfilt,I,'same');

Use your 1-D boxcar filter as a separable filter to apply a 2-D (10×10) box filter to the mandrill image.
Display, inspect, and save this image. What artifacts can you find? (Hint, look near high-contrast fine structures.)
Use your 1-D Gaussian as a separable filter to apply a 2-D Gaussian to the mandrill.
Display, inspect, and save this image. What differences do you notice from the box-filtered image?
Which filter seems qualitatively better for blurring an image? Why?
The commands tic and toc can be used to measure wall-clock time for a (series of) Matlab command(s). Use these to calculate the time it takes to filter the mandrill image using
1. the two separable 2-D gaussians, versus
2. the (equivalent) fully 2-D Gaussian kernel.
Report your times. How do the times compare? Can you quantitively relate the times theoretically? Is separable filtering of any advantage?

C. Correlation for Detection

In this part of the lab, we will explore using correlation as a means of detecting instances of a "feature" in an image. In this case, we will use it to find the letter "t" in an image.

Load a scanned image of a legal document in

/home/weinman/courses/CSC295/images/legal.png
Convert this image to grayscale and from a uint8 image to a double (so that the largest value can be 1).
Create an inverted copy of the image so that the regions that are currently black (i.e., the letters) now becomes white, and vice-versa. The result should be white characters on a black background.
Next we will create a template to use for detection. While we could simply use array indexing to extract a region of the image/matrix, it is often helpful to do this graphically. The Matlab command imcrop does just that, i.e.,

template = imcrop(L);

will display the image L and a pair of crosshairs that you can use to select a square region of the image. Once you have placed a region it is also adjustable. Once you have are satisfied with the region selected, you can double-click to have it returned as the value of the function (assigned to tt in our example).
Use imcrop to extract a lower-case t from the inverted image. Be sure to include a small amount of space around the letter "t" (vertically at least), but not too much horizontally (since you don't want any neighboring characters in your examplar). My template was 15×11.
Rescale your template so that the range is -1 to 1, rather than 0 to 1. (Thus a zero becomes a negative 1 and one-half would become zero):

template = 2*template - 1;

Consider why your template is being adjusted this way. You'll be asked about it in a moment after you've used it.
Recall that the different between convolution and correlation is a "flipping" of the kernel. Since we want to use your template as an exemplar, we need to use correlation. By default, the Matlab function filter2 does this. The same border-handling issues are manifest, so we will stick to using 'same' as an option to return an image of the same size, as in

result = filter2(template,Image,'same');

Use this to calculate the correlation of your template on the inverted image.
Display, inspect, and save the result (note that the range of the image is not necessarily 0-1). Are you seeing strong responses where you expect to see them?
In addition to the visual inspection, one useful data analysis to perform is investigating the population of the responses more quantitatively. The sort command takes a vector and returns the same data sorted (ascending, by default). Use this to display a plot of a sorted version of all the responses.
Using the figure window's zoom capabilities, navigate to the region of the plot where the strongest responses are. Which responses do you think are due to actual "t"s?
Inspecting this region of the sorted responses, guess at a threshold on the response value that you think separates the responses to due actual "t"s from the responses of other image features.
Recall that the comparison operators (e.g., >, etc.) are vectorized in Matlab. Vectorized operations work over arrays (and images) just the same as over 1-D vectors.
Additionally, in the same way that find returns the indices of non-zero elements of a vector, it also allows you to find all the row, column pairs of non-zero elements of an array. For instance,

[rows,cols] = find(A);

will have rows and cols as vectors whose corresponding entries (e.g., row(5) and col(5)) are the row and column of each point found in A.
Use the find command to locate the row and column of all the points in your image where the response exceeds your chosen threshold.
In addition to plotting lines/curves, the plot command can simply produce a scatter plot of the points it is given without connecting them. For instance,

plot(X, Y, 'r+');

displays the points in vectors X and Y using red plus marks. (Consult the plot help for other marker options.) Use this to display the locations of your detected "t"s on top of the original (non-inverted) image as red plus marks. Note:
- Don't forget to use hold on after displaying the image so that the plot doesn't eliminate it.
- Don't forget that rows are a y-coordinate and cols are an x-coordinate.
Attempt to adjust your threshold so that you can find all of the "t"s with no false positives (marks where there is not actually a "t".)
Use print to save the image of the page with your detected "t"s marked.
Are you able to to detect all the "t"s perfectly? Why or why not?

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.