Image Processing

In this unit we introduce specific data structures in C for representing and processing images.

Introduction

As you likely know from experiences with MediaScheme in CSC 151, images commonly consist of a large number of dots or pixels, arranged in a grid pattern.

Pixels

While several image formats exist, perhaps the most common specification for each pixel involves a red, green, and blue component. Each color component is typically stored as one byte, a char in C. In this context, negative values are not meaningful, so each component is considered an unsigned char, taking values from 0 to 255. Further, the red, green, and blue (R/G/B) components naturally form a coherent quantity— a struct in the context of C. With this in mind, MyroC defines a pixel as follows:

/**
 * @brief Struct for a pixel
 */
typedef struct
{
  unsigned char R; // The value of the red component 
  unsigned char G; // The value of the green component 
  unsigned char B; // The value of the blue component 
} Pixel;

Within this framework, R/G/B values of 0 correspond to black and R/G/B values of 255 correspond to white. This leads to the following natural definitions:

Pixel blackPix = {0, 0, 0};
Pixel whitePix = {255, 255, 255);

Pictures

Perhaps the most conceptually-simple structure for a picture involves a two-dimensional array of R/G/B pixels. Each picture has a height and a width, and an overall picture is just a two-dimensional array with those dimensions. When working with a Scribbler 2 robot, the camera takes a picture that is 192 pixels high and 256 pixels wide. Of course, other cameras or images may have a different dimensions.

A pragmatic detail: You may recall from working with one- and two-dimensional arrays that the declaration of a two-dimensional array allocates space, but the array name just gives the base address, not the height and width dimensions, We cannot infer the dimensions of the array given only the variable name. For this reason, it is convenient to store the dimensions of an image together with the two-dimensional array. Thus, in much processing, the height, width, and pixel array are naturally part of a single package, so a picture is defined as a struct:

/**
 * @brief Struct for a picture object
 * @note the picture size is always 256 in width and 192 in height
 */
typedef struct 
{
  int height; /* The height of the image -- set to 192 for robot camera */
  int width;  /* The width of the image  -- set to 256 for robot camera */
  Pixel pix_array[192][256]; /* The array of pixels comprising the image */
} Picture;

Technical Aside

Since image processing is used in a wide variety of applications, several common formats are used to store image data. The approach here, with an array of R/G/B pixels, is conceptually simple. However, other formats are possible as well.

The camera in a Scribbler 2 robot actually uses a different color designation (YUV format). Behind the scenes, the Scribbler transmits YUV values to your workstation, where the rGetPicture function transforms the YUV color coding to the more common RGB format.

Since images can consume much space, various formats are used to compress file sizes to speed the transmission of images and to reduce storage requirements. Each format has specific advantages for certain purposes. The .jpg or JPEG format was created by the Joint Photographic Experts Group (hence the acronym) and is largely based on what people actually see. Since this format is particularly common, MyroC provides functions to convert between a raw RGB format and JPEG format:

• rSavePicture stores your RGB picture from main memory as a file using jpeg format.
• rLoadPicture loads a jpeg image from a file into main memory, resulting in an RGB struct.
• rDisplayPicture displays your RGB picture from main memory onto a window at your computer.

See the MyroC header file for details on each of these functions.

Examples

Splicing Pictures

The following program demonstrates how to capture images with the robot then iterate over the pixels in the image, assigning color values.

/* Example program taking two pictures using the Scribbler 2 and shows a
 *  picture composed of pieces of the two pictures */

#include <stdio.h>
#include <MyroC.h>

int
main()
{
  Picture  pic1,  pic2,  spliced;
  int width, height, midcol, midrow;

  rConnect ("/dev/rfcomm0");

  pic1 = rTakePicture();
  rTurnLeft (1, 1);
  pic2 = rTakePicture();

  rDisplayPicture (pic1, 5, "Picture 1");
  rDisplayPicture (pic2, 5, "Picture 2");

  height = pic1.height;
  width = pic1.width;
  midrow = height / 2;
  midcol = width / 2;

  spliced.height = height;
  spliced.width = width;

  int col,row;
  for (row = 0; row < height; row++)
    {
      for (col = 0; col < width ; col++)
        {
          if ( ((col < midcol) && (row < midrow))      // top-left quadrant
	       || ((col > midcol) && (row > midrow)) ) // bottom-right quadrant
	    spliced.pix_array[row][col]=pic1.pix_array[row][col];           
	  else
            spliced.pix_array[row][col]=pic2.pix_array[row][col]; 
        } // col
    } // row

  rDisplayPicture (spliced,-3,"Spliced Picture");
  rDisconnect();
  return 0;
} // main