Something More for Research

Explorer of Research #HEMBAD

OpenCV Viola & Jones object detection in MATLAB

Posted by Hemprasad Y. Badgujar on January 19, 2015


In image processing, one of the most successful object detectors devised is theViola and Jones detector, proposed in their seminal CVPR paper in 2001. A popular implementation used by image processing researchers and implementers is provided by the OpenCV library. In this post, I’ll show you how run the OpenCV object detector in MATLAB for Windows. You should have some familiarity with OpenCV and with the Viola and Jones detector to work through this tutorial.

Steps in the object detector

MATLAB is able to call functions in shared libraries. This means that, using the compiled OpenCV DLLs, we are able to directly call various OpenCV functions from within MATLAB. The flow of our MATLAB program, including the required OpenCV external function calls (based on this example), will go something like this:

  1. cvLoadHaarClassifierCascade: Load object detector cascade
  2. cvCreateMemStorage: Allocate memory for detector
  3. cvLoadImage: Load image from disk
  4. cvHaarDetectObjects: Perform object detection
  5. For each detected object:
    1. cvGetSeqElem: Get next detected object of type cvRect
    2. Display this detection result in MATLAB
  6. cvReleaseImage: Unload the image from memory
  7. cvReleaseMemStorage: De-allocate memory for detector
  8. cvReleaseHaarClassifierCascade: Unload the cascade from memory

Loading shared libraries

The first step is to load the OpenCV shared libraries using MATLAB’sloadlibrary() function. To use the functions listed in the object detector steps above, we need to load the OpenCV libraries cxcore2410.dll, cv2410.dll andhighgui2410.dll. Assuming that OpenCV has been installed to "C:\Program Files\OpenCV", the libraries are loaded like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
opencvPath = 'C:\Program Files\OpenCV';
includePath = fullfile(opencvPath, 'cxcore\include');
loadlibrary(...
    fullfile(opencvPath, 'bin\cxcore2410.dll'), ...
    fullfile(opencvPath, 'cxcore\include\cxcore.h'), ...
        'alias', 'cxcore2410', 'includepath', includePath);
loadlibrary(...
    fullfile(opencvPath, 'bin\cv2410.dll'), ...
    fullfile(opencvPath, 'cv\include\cv.h'), ...
        'alias', 'cv2410', 'includepath', includePath);
loadlibrary(...
    fullfile(opencvPath, 'bin\highgui2410.dll'), ...
    fullfile(opencvPath, 'otherlibs\highgui\highgui.h'), ...
        'alias', 'highgui2410', 'includepath', includePath);

You will get some warnings; these can be ignored for our purposes. You can display the list of functions that a particular shared library exports with thelibfunctions() command in MATLAB For example, to list the functions exported by the highgui library:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
>> libfunctions('highgui2410')
Functions in library highgui2410:
cvConvertImage             cvQueryFrame
cvCreateCameraCapture      cvReleaseCapture
cvCreateFileCapture        cvReleaseVideoWriter
cvCreateTrackbar           cvResizeWindow
cvCreateVideoWriter        cvRetrieveFrame
cvDestroyAllWindows        cvSaveImage
cvDestroyWindow            cvSetCaptureProperty
cvGetCaptureProperty       cvSetMouseCallback
cvGetTrackbarPos           cvSetPostprocessFuncWin32
cvGetWindowHandle          cvSetPreprocessFuncWin32
cvGetWindowName            cvSetTrackbarPos
cvGrabFrame                cvShowImage
cvInitSystem               cvStartWindowThread
cvLoadImage                cvWaitKey
cvLoadImageM               cvWriteFrame
cvMoveWindow
cvNamedWindow

The first step in our object detector is to load a detector cascade. We are going to load one of the frontal face detector cascades that is provided with a normal OpenCV installation:

1
2
3
classifierFilename = 'C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml';
cvCascade = calllib('cv2410', 'cvLoadHaarClassifierCascade', classifierFilename, ...
    libstruct('CvSize',struct('width',int16(2410),'height',int16(2410))));

The function calllib() returns a libpointer structure containing two fairly self-explanatory fields, DataType and Value. To display the return value fromcvLoadHaarClassifierCascade(), we can run:

1
2
3
4
5
6
7
8
9
10
11
>> cvCascade.Value
ans =
               flags: 1.1125e+009
               count: 22
    orig_window_size: [1x1 struct]
    real_window_size: [1x1 struct]
               scale: 0
    stage_classifier: [1x1 struct]
         hid_cascade: []

The above output shows that MATLAB has successfully loaded the cascade file and returned a pointer to an OpenCV CvHaarClassifierCascade object.

Prototype M-files

We could now continue implementing all of our OpenCV function calls from the object detector steps like this, however we will run into a problem when cvGetSeqElem is called. To see why, try this:

libfunctions('cxcore2410', '-full')

The -full option lists the signatures for each imported function. The signature for the function cvGetSeqElem() is listed as:

[cstring, CvSeqPtr] cvGetSeqElem(CvSeqPtr, int32)

This shows that the return value for the imported cvGetSeqElem() function will be a pointer to a character (cstring). This is based on the function declaration in thecxcore.h header file:

CVAPI(char*)  cvGetSeqElem( const CvSeq* seq, int index );

However, in step 5.1 of our object detector steps, we require a CvRect object. Normally in C++ you would simply cast the character pointer return value to aCvRect object, but MATLAB does not support casting of return values fromcalllib(), so there is no way we can cast this to a CvRect.

The solution is what is referred to as a prototype M-file. By constructing a prototype M-file, we can define our own signatures for the imported functions rather than using the declarations from the C++ header file.

Let’s generate the prototype M-file now:

1
2
3
4
loadlibrary(...
    fullfile(opencvPath, 'bin\cxcore2410.dll'), ...
    fullfile(opencvPath, 'cxcore\include\cxcore.h'), ...
        'mfilename', 'proto_cxcore');

This will automatically generate a prototype M-file named proto_cxcore.m based on the C++ header file. Open this file up and find the function signature forcvGetSeqElem and replace it with the following:

1
2
3
% char * cvGetSeqElem ( const CvSeq * seq , int index );
fcns.name{fcnNum}='cvGetSeqElem'; fcns.calltype{fcnNum}='cdecl'; fcns.LHS{fcnNum}='CvRectPtr'; fcns.RHS{fcnNum}={'CvSeqPtr', 'int32'};fcnNum=fcnNum+1;

This changes the return type for cvGetSeqElem() from a char pointer to aCvRect pointer.

We can now load the library using the new prototype:

1
2
3
loadlibrary(...
    fullfile(opencvPath, 'bin\cxcore2410.dll'), ...
        @proto_cxcore);

An example face detector

We now have all the pieces ready to write a complete object detector. The code listing below implements the object detector steps listed above to perform face detection on an image. Additionally, the image is displayed in MATLAB and a box is drawn around any detected faces.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
opencvPath = 'C:\Program Files\OpenCV';
includePath = fullfile(opencvPath, 'cxcore\include');
inputImage = 'lenna.jpg';
%% Load the required libraries
if libisloaded('highgui2410'), unloadlibrary highgui2410, end
if libisloaded('cv2410'), unloadlibrary cv2410, end
if libisloaded('cxcore2410'), unloadlibrary cxcore2410, end
loadlibrary(...
    fullfile(opencvPath, 'bin\cxcore2410.dll'), @proto_cxcore);
loadlibrary(...
    fullfile(opencvPath, 'bin\cv2410.dll'), ...
    fullfile(opencvPath, 'cv\include\cv.h'), ...
        'alias', 'cv2410', 'includepath', includePath);
loadlibrary(...
    fullfile(opencvPath, 'bin\highgui2410.dll'), ...
    fullfile(opencvPath, 'otherlibs\highgui\highgui.h'), ...
        'alias', 'highgui2410', 'includepath', includePath);
%% Load the cascade
classifierFilename = 'C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml';
cvCascade = calllib('cv2410', 'cvLoadHaarClassifierCascade', classifierFilename, ...
    libstruct('CvSize',struct('width',int16(2410),'height',int16(2410))));
%% Create memory storage
cvStorage = calllib('cxcore2410', 'cvCreateMemStorage', 0);
%% Load the input image
cvImage = calllib('highgui2410', ...
    'cvLoadImage', inputImage, int16(1));
if ~cvImage.Value.nSize
    error('Image could not be loaded');
end
%% Perform object detection
cvSeq = calllib('cv2410', ...
    'cvHaarDetectObjects', cvImage, cvCascade, cvStorage, 1.1, 2, 0, ...
    libstruct('CvSize',struct('width',int16(40),'height',int16(40))));
%% Loop through the detections and display bounding boxes
imshow(imread(inputImage)); %load and display image in MATLAB
for n = 1:cvSeq.Value.total
    cvRect = calllib('cxcore2410', ...
        'cvGetSeqElem', cvSeq, int16(n));
    rectangle('Position', ...
        [cvRect.Value.x cvRect.Value.y ...
        cvRect.Value.width cvRect.Value.height], ...
        'EdgeColor', 'r', 'LineWidth', 3);
end
%% Release resources
calllib('cxcore2410', 'cvReleaseImage', cvImage);
calllib('cxcore2410', 'cvReleaseMemStorage', cvStorage);
calllib('cv2410', 'cvReleaseHaarClassifierCascade', cvCascade);

As an example, the following is the output after running the detector above on a greyscale version of the Lenna test image:

Note: If you get a segmentation fault attempting to run the code above, tryevaluating the cells one-by-one (e.g. by pressing Ctrl-Enter) – it seems to fix the problem.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Extracts from a Personal Diary

dedicated to the life of a silent girl who eventually learnt to open up

Num3ri v 2.0

I miei numeri - seconda versione

ThuyDX

Just another WordPress.com site

Algunos Intereses de Abraham Zamudio Chauca

Matematica, Linux , Programacion Serial , Programacion Paralela (CPU - GPU) , Cluster de Computadores , Software Cientifico

josephdung

thoughts...

Tech_Raj

A great WordPress.com site

Travel tips

Travel tips

Experience the real life.....!!!

Shurwaat achi honi chahiye ...

Ronzii's Blog

Just your average geek's blog

Karan Jitendra Thakkar

Everything I think. Everything I do. Right here.

VentureBeat

News About Tech, Money and Innovation

Chetan Solanki

Helpful to u, if u need it.....

ScreenCrush

Explorer of Research #HEMBAD

managedCUDA

Explorer of Research #HEMBAD

siddheshsathe

A great WordPress.com site

Ari's

This is My Space so Dont Mess With IT !!

%d bloggers like this: