OpenCV:A Development Enviroment for Computer Vision in C/C++

General Description:

OpenCV is basically an Open source computer vision library in C/C++. It is optimized and intended for real-time applications. OpenCV is OS/hardware/window-manager independent.Generic image/video loading, saving, and acquisition. It supports both low and high level API. OpenCV also provides interface to Intel's Integrated Performance Primitives (IPP) with processor specific optimization (Intel processors). OpenCV’s official web site is

http://www.intel.com/technology/computing/opencv/ and can be downloaded from the Sourceforge Page: http://sourceforge.net/projects/opencvlibrary/

Some Features of OpenCv:

Image data manipulation (allocation, release, copying, setting, conversion).

Image and video I/O (file and camera based input, image/video file output).

Matrix and vector manipulation and linear algebra routines (products, solvers, eigenvalues, SVD).

Various dynamic data structures (lists, queues, sets, trees, graphs).

Basic image processing (filtering, edge detection, corner detection, sampling and interpolation, color conversion, morphological operations, histograms, image pyramids).

Structural analysis (connected components, contour processing, distance transform, various moments, template matching, Hough transform, polygonal approximation, line fitting, ellipse fitting, Delaunay triangulation).

Camera calibration (finding and tracking calibration patterns, calibration, fundamental matrix estimation, homography estimation, stereo correspondence).

Motion analysis (optical flow, motion segmentation, tracking).

Object recognition (eigen-methods, HMM).

Basic GUI (display image/video, keyboard and mouse handling, scroll-bars).

Image labeling (line, conic, polygon, text drawing)

OpenCV naming conventions

Functions in openCv starts with the library name(ex: functions in cv.h starts with cv) prefix and then the Core functionality follows it.The function name sometimes ends with the datatype that it returns. Function naming conventions:

    cvActionTargetMod(...)

    Action = the core functionality (e.g. set, create)
    Target = the target image area (e.g. contour, polygon)
    Mod    = optional modifiers (e.g. argument type)
    Ex: cvFindExtrinsicCameraParams_64d(...)

Matrix data types:

    CV_<bit_depth>(SUF)C<number_of_channels>

    S = Signed integer
    U = Unsigned integer
    F = Float


    Ex: CV_8UC1 means an 8-bit unsigned single-channel matrix,

    CV_32FC2 means a 32-bit float matrix with two channels.

Image data types:

    IPL_DEPTH_<bit_depth>(SUF)

    E.g.: IPL_DEPTH_8U means an  8-bit unsigned image.

          IPL_DEPTH_32F means a 32-bit float image.

Header files:

    #include <cv.h>

    #include <cvaux.h>

    #include <highgui.h>

    #include <ml.h>

    #include <cxcore.h>   // unnecessary - included in cv.h

Library Definitions:

cv.h: Includes the essential computer vision and image processing functions.

cvaux.h: Obsolote library and includes some experimental codes.

highgui.h: Functions for user interface and drawing

ml.h: Machine Learning tools. For exampl: SVM(Support Vector Machine), ANN, Normal Bayes Classifier...etc

cxcore.h : This library includes Data Structures and Data types needed for image processing and computer vision.

cvcam.h: CvCam has important functions and interfaces for accessing camera and image retrieval.

Fundamental Data Structures in OpenCV

Image data structure

Image data structure is used for storing image in the memory and accessing image’s properties.

IPL image:

IplImage
  -- int  nChannels;

// Number of color channels (1,2,3,4)


  -- int  depth;    // Pixel depth in bits:
                    //   IPL_DEPTH_8U, IPL_DEPTH_8S,
                    //   IPL_DEPTH_16U,IPL_DEPTH_16S,
                    //   IPL_DEPTH_32S,IPL_DEPTH_32F,

                    //   IPL_DEPTH_64F
  -- int  width;    // image width in pixels
  -- int  height;   // image height in pixels

  -- char* imageData;// pointer to aligned image data

   // Note that color images are stored in BGR order
  -- int  dataOrder;// 0 - interleaved color channels,
                    // 1 - separate color channels
 // cvCreateImage can only create interleaved images
  -- int  origin;  // 0 - top-left origin,
    // 1 - bottom-left origin (Windows bitmaps style)
  -- int  widthStep;

// size of aligned image row in bytes
  -- int  imageSize;

// image data size in bytes = height*widthStep


  -- struct _IplROI *roi;

 // image ROI. when not NULL specifies image
               // region  to be processed.
  -- char *imageDataOrigin;

// pointer to the unaligned origin of image data
// (needed for correct image deallocation)

  -- int  align;

// Alignment of image rows: 4 or 8 byte alignment
// OpenCV ignores this and uses widthStep instead

  -- char colorModel[4];

//Color model-ignored by OpenCV

Matrices and vectors

Matrices in OpenCv is usually used for mathematical operations for image processing and computer vision on the images caputured.Matrices:

CvMat                      // 2D array

  -- int   type;

//elements type

//(uchar,short,int,float,double) and flags


  -- int   step; // full row length in bytes

-- int rows, cols; // dimensions
-- int height, width;
// alternative dimensions reference

-- union data;

      -- uchar*  ptr;

 // data pointer for an unsigned char matrix
      -- short*  s;

// data pointer for a short matrix
      -- int*    i;

// data pointer for an integer matrix
      -- float*  fl;

// data pointer for a float matrix
      -- double* db;

// data pointer for a double matrix
CvMatND  // N-dimensional array

  -- int   type;

// elements type

//(uchar,short,int,float,double) and flags


  -- int   dims;// number of array dimensions

  -- union data;
     -- uchar*  ptr;

// data pointer for an unsigned char matrix
     -- short*  s;

// data pointer for a short matrix
     -- int*    i;

// data pointer for an integer matrix
     -- float*  fl;

// data pointer for a float matrix

     -- double* db;

// data pointer for a double matrix

  -- struct dim[];

// information for each dimension
      -- size;

// number of elements in a given dimension

      -- step;

//distance between elements in a given dimension


CvSparseMat // SPARSE N-dimensional array

Arrays are very important data structures in programming. They can be used in many ways. Generic arrays:

CvArr*

// Used only as a function parameter to specify that the
// function accepts arrays of more than a single type,
//such as: IplImage*, CvMat* or even CvSeq*. The
// particular array type is determined at
// runtime by analyzing the first 4 bytes of the
//header of the actual array.

Scalars:

CvScalar

  -- double val[4]; //4D vector

Initializer function:

CvScalar s = cvScalar(double val0, double val1=0, double val2=0, double val3=0);

Example:

CvScalar s = cvScalar(20.0);

s.val[0]=10.0;

Note that the initializer function has the same name as the data structure only starting with a lower case character. It is not a C++ constructor.

Other data structures

Point data structure is usually used for storing specific points coordinates in the memory. Points:

CvPoint      p = cvPoint(int x, int y);

CvPoint2D32f p = cvPoint2D32f(float x, float y);

CvPoint3D32f p = cvPoint3D32f(float x, float y, float z);

E.g.:

p.x=5.0;

p.y=5.0;

Rectangular dimensions:

CvSize       r = cvSize(int width, int height);

CvSize2D32f  r = cvSize2D32f(float width, float height);

Rectangular dimensions with offset:

CvRect       r = cvRect(int x, int y, int width, int height);

Some GUI commands

Window management

Create and position a window:

  cvNamedWindow("win1", CV_WINDOW_AUTOSIZE);
  cvMoveWindow("win1", 100, 100);

// offset from the UL corner of the screen

Load an image:

  IplImage* img=0;
  img=cvLoadImage(fileName);
  if(!img) printf("Could not load image file: %s\n",fileName);

Display an image:

  cvShowImage("win1",img);

Can display a color or grayscale byte/float-image. A byte image is assumed to have values in the range [0..255]. A float image is assumed to have values in the range [0..1]. A color image is assumed to have data in BGR order.

Close a window:

  cvDestroyWindow("win1");

Resize a window:

  cvResizeWindow("win1",100,100); // new width/heigh in pixels

Input handling

Handle mouse events:

Define a mouse handler:

  void mouseHandler(int event, int x, int y, int flags, void* param)  {

    switch(event){

      case CV_EVENT_LBUTTONDOWN:

        if(flags & CV_EVENT_FLAG_CTRLKEY)

          printf("Left button down with CTRL pressed\n");

        break;

      case CV_EVENT_LBUTTONUP:

        printf("Left button up\n");

        break;

    }

  }

  x,y:   pixel coordinates with respect to the UL corner

  event: CV_EVENT_LBUTTONDOWN,   CV_EVENT_RBUTTONDOWN,   CV_EVENT_MBUTTONDOWN,

   CV_EVENT_LBUTTONUP,     CV_EVENT_RBUTTONUP,     CV_EVENT_MBUTTONUP,

       CV_EVENT_LBUTTONDBLCLK, CV_EVENT_RBUTTONDBLCLK, CV_EVENT_MBUTTONDBLCLK,

       CV_EVENT_MOUSEMOVE:

  flags: CV_EVENT_FLAG_CTRLKEY, CV_EVENT_FLAG_SHIFTKEY, CV_EVENT_FLAG_ALTKEY,

         CV_EVENT_FLAG_LBUTTON, CV_EVENT_FLAG_RBUTTON,  CV_EVENT_FLAG_MBUTTON

  mouseParam=5;

  cvSetMouseCallback("win1",mouseHandler,&mouseParam);

Handle keyboard events:

The keyboard does not have an event handler.

Get keyboard input without blocking:

  int key;

  key=cvWaitKey(10); // wait 10ms for input

Get keyboard input with blocking:

  int key;

  key=cvWaitKey(0); // wait indefinitely for input

The main keyboard event loop:

  while(1){

    key=cvWaitKey(10);
    if(key==27) break;

    switch(key){

      case 'h':

...

        break;

      case 'i':

        ...

      break;

    }

  }

Handle trackbar events:

Define a trackbar handler:

  void trackbarHandler(int pos)

  {

    printf("Trackbar position: %d\n",pos);

  }

  int trackbarVal=25;

  int maxVal=100;

  cvCreateTrackbar("bar1", "win1", &trackbarVal ,maxVal , trackbarHandler);

Get the current trackbar position:

  int pos = cvGetTrackbarPos("bar1","win1");

Set the trackbar position:

  cvSetTrackbarPos("bar1", "win1", 25);

A sample OpenCv Program

Here is a sample OpenCv program that captures frames from camera and display it in a window.

CameraCapture.c:

#include "cv.h"

#include "highgui.h"

#include <stdio.h>

// A Simple Camera Capture Program

int main() {

//Initializing the capture from camera

CvCapture* capture = cvCaptureFromCAM(0); // capture from video device #0

if( !capture ) {
fprintf( stderr, "ERROR: capture is NULL \n" );
getchar();

return -1;

}

// Create a window in which the captured images will be presented

cvNamedWindow( "capWindow", CV_WINDOW_AUTOSIZE );

// Show the image captured from the camera in the window and repeat

while( 1 ) {

// Get one frame
IplImage* frame = cvQueryFrame( capture );
//Check that if we captured the frame if we could not capture any frame display error!
if( !frame ) {
fprintf( stderr, "ERROR: frame is null...\n" );
getchar();
break;

}

cvShowImage( "capWindow", frame );

// Do not release the frame!

//If ESC key pressed, Key=0x10001B

//remove higher bits using AND operator

if( (cvWaitKey(10) & 255) == 27 ) break;

}

// Always Release the capture device and Destroy the window

cvReleaseCapture( &capture );

cvDestroyWindow( "capWindow" );

return 0;

}

As we see above to capture images from the camera, firstly we initialized the camera device via CvCapture and with the cvCaptureFromCam(0) we choose to grab the images from the local camera connected to computer. Then we checked the capture object if the program could detect a camera or not for capturing purpose.

We created a window by the cvNamedWindow() function which takes 2 parameters. First one is the name of the window and the second one is the size of the window.

In the while(1) loop we grabbed images from the camera by the cvQueryFrame() function which takes the CvCapture object and check if the frame is null. If frame is not null then we displayed the frame in the window by cvShowImage() which takes the name of the window that the image will be shown at and the IplImage object.

At the end of the program we released the CvCapture object and destroyed the window.