Lab: Image Filtering

CSC 262 - Computer Vision - Weinman

Summary:

We compare filtering using separable kernels, several smoothing kernels and finally use filtering to detect image some interesting image features.

Deliverables

• The Matlab script used to generate your report and the template image from C.4c
• (5 points) Plot of 2-D Gaussian (A.8)
• (5 points) Image of 2-D Gaussian (A.9)
• (10 points) Image and observations of box filtering result (B.2)
• (10 points) Image and observations of Gaussian filtering result (B.4)
• (5 points) Filter evaluation (B.5)
• (10 points) Comparison and analysis of separable filtering (B.7)
• (10 points) Image and observations of correlation result (C.7)
• (10 points) Correlation detections (C.14)
• (10 points) Detection evaluation (C.15)
• (10 points) Professionalism of write-up

Extra Credit

• (10 points) Unsharp mask images (D.1)
• (5 points) Unsharp mask observations (D.2)

Preparation

1. Load the mandrill demo image and convert it from an indexed color image to a grayscale image using the following commands:

   load mandrill

   img = im2double(ind2gray(X,map));

   clear map;
2. Show the mandrill image so that you know what it looks like before filtering.

Exercises

A. Separability

1. Create a 1×10 boxcar smoothing filter using the ones function. (Be sure it is normalized!)
2. Verify that the 2-D box filter kernel is separable by using Equation (3.20) of Szeliski,
   

   K=vhT,

   to create a 10×10 box filter from your boxcar filter. (Note that he assumes that filters v and h are column vectors, rather than row vectors, so you'll have to adjust the equation ever so slightly.)
   The single quote operator ' tells Matlab to take the transpose of a matrix or vector.
3. Is the 2-D kernel also normalized?
4. Create a 1-D Gaussian kernel with a variance of 10 using the provided function gkern (this is not a standard Matlab function; it is in the toolbox you added to Matlab's search path in the first lab):

   gauss = gkern(10);
5. Plot the kernel and verify that it has a 1-D Gaussian appearance. How big is the kernel? Is it normalized?
6. Create and plot a few Gaussian kernels with other variances to see how the shape changes.
7. Use the same transformation (called an "outer product") that you used for the box filter to generate a 2-D Gaussian kernel. Is this 2-D version also normalized?
8. It is worthwhile to visualize your kernel two different ways, as an image and as a 3-D surface plot. Use surf to create a simple 3-D plot of your kernel. (You do not need to label the axes.) Does it have the expected 2-D Gaussian appearance?
9. Show your kernel as an image (you will likely need to tell imshow to ignore the conventional range of 1 as white, as you did in the image formation lab, in order to see the complete contrast).

B. Smoothing Filters

Ifilt = conv2(I,K,'same');

Isepfilt = conv2(vfilt,hfilt,I,'same');

1. Use your 1-D boxcar filter as a separable filter to apply a 2-D (10×10) box filter to the mandrill image.
2. Display and inspect this image. What artifacts can you find? (Hint, look near high-contrast fine structures.)
3. Use your 1-D Gaussian as a separable filter to apply a 2-D Gaussian to the mandrill.
4. Display and inspect this image. What differences do you notice from the box-filtered image?
5. Which filter seems qualitatively better for blurring an image? Why?
6. The commands tic and toc can be used to measure wall-clock time for a (series of) Matlab command(s). Use these to calculate the time it takes to filter the mandrill image using
   1. the two separable 1-D gaussians, versus
   2. the (equivalent) fully 2-D Gaussian kernel.
7. Report your times. How do the times compare? Can you quantitively relate the times theoretically? Is separable filtering of any advantage?

C. Correlation for Detection

1. Load a scanned image of a legal document in

   /home/weinman/courses/CSC262/images/legal.png
2. Convert this image from a uint8 image to a double (so that the largest value can be 1).
3. Create an "inverted" copy of the image so that the regions that are currently black (i.e., the letters) now becomes white, and vice-versa. The result should be white characters on a black background.
4. Next we will create a template to use for detection. While we could simply use array indexing to extract a region of the image/matrix, it is often helpful to do this graphically. The Matlab command imcrop does just that, i.e.,

   template = imcrop(L);

   will display the image L and a pair of crosshairs that you can use to select a square region of the image. Once you have placed a region it is also adjustable. Once you have are satisfied with the region selected, you can double-click to have it returned as the value of the function (assigned to template in our example).
   Use imcrop to extract a lower-case t from the inverted image. Be sure to include a small amount of space around the letter "t" (vertically at least), but not too much horizontally (since you don't want any neighboring characters in your examplar). My template was 15×11.
   Don't put these commands in your report - they'll show the original image (which we don't need) and make your report too variable/unrepeatable (which we don't want). Instead, follow these instructions
   1. Save your template as an image, to the same folder as the m file your report script will be in.

      imwrite(template, 'template.png');
   2. In your report m file, use imread to load your template and be sure to convert it to a double (in 0-1 range).
   3. Submit this template.png along with your .m file and the final report PDF.
5. Rescale your template values so their range is -1 to 1, rather than 0 to 1. (Thus a zero becomes a negative 1 and one-half would become zero):

   template = 2*template - 1;

   Consider why your template is being adjusted this way. You'll be asked about it in a moment after you've used it.
6. Recall that the difference between convolution and correlation is a "flipping" of the kernel. Because we want to use your template as an exemplar, we need to use correlation. By default, the Matlab function filter2 does this. The same border-handling issues are manifest, so we will stick to using 'same' as an option to return an image of the same size, as in

   result = filter2(template,Image,'same');

   Use this to calculate the correlation of your template on the inverted image.
7. Display and inspect the result (note that the range of the image is not necessarily 0-1). Are you seeing strong responses where you expect to see them?
8. In addition to the visual inspection, one useful data analysis to perform is investigating the population of the responses more quantitatively. The sort command takes a vector and returns the same data sorted (ascending, by default). Use this to display a plot of a sorted version of all the responses.
9. Using the figure window's zoom capabilities, navigate to the region of the plot where the strongest responses are. Which responses do you think are due to actual "t"s?
10. Inspecting this region of the sorted responses, guess at a threshold on the response value that you think separates the responses of actual "t"s from the responses of other image features.
11. Recall that the comparison operators (e.g., >, etc.) are vectorized in Matlab. Vectorized operations work over arrays (and images) just the same as over 1-D vectors.
    Additionally, in the same way that find returns the indices of non-zero elements of a vector, it also allows you to find all the row, column pairs of non-zero elements of an array. For instance,

    [rows,cols] = find(A);

    will have rows and cols as vectors whose corresponding entries (e.g., row(5) and col(5)) are the row and column of each point found in A.
    Use the find command to locate the row and column of all the points in your image where the response exceeds your chosen threshold.
12. In addition to plotting lines/curves, the plot command can simply produce a scatter plot of the points it is given without connecting them. For instance,

    plot(X, Y, 'r+');

    displays the points in vectors X and Y using red plus marks. (Consult the plot help for other marker options.) Use this to display the locations of your detected "t"s on top of the original (non-inverted) image as red plus marks. Note:
    • Use hold on after displaying the image so that the plot doesn't eliminate it.
    • rows are a y-coordinate and cols are an x-coordinate.
13. Attempt to adjust your threshold so that you can find all of the "t"s with no false positives (marks where there is not actually a "t".)
14. When you are satisfied with your threshold, display the image of the page with your detected "t"s marked.
15. Are you able to to detect all the "t"s perfectly? Why or why not?

D. Extra Credit: Unsharp Masking

1. Using any image of your choice, create a single plot (using subplot) showing
   • the original image
   • the blurred image
   • the difference image (normalize scale for visualization)
   • the final result
   For a more visually impressive result, apply the process (independently) to each channel of an RGB image.
2. Include your observations about the results. What changes do you see in the image? How do these correspond to the components from D.1? What actual (not predicted) effect does the choice of γ have on the results? What limitations can you find on the effect of this approach?

Acknowledgments

The images in the USC Image Database are intended for research purposes. USC-SIPI does not own the copyright of most of the images and the copyright status of many of the images is unknown. For more information on the copyright status of the images, please click here. If you plan on using any of the images in a publication, please read the copyright information before committing to using any of the images.