4/25/2014

(OpenCV Study) Background subtraction and Draw blob to red rectangle (example source code)

I use MOG2 algorithm to background subtraction.

The process is
resize to small for more fast processing
to blur for avoid noise affection
morphology make blob and remove noise
findContour to draw blob rectangle

Example movie of result.
more detail refer to source code..
#include < stdio.h>
#include < iostream>

#include < opencv2\opencv.hpp>
#include < opencv2/core/core.hpp>
#include < opencv2/highgui/highgui.hpp>
#include < opencv2/video/background_segm.hpp>


#ifdef _DEBUG        
#pragma comment(lib, "opencv_core247d.lib")
#pragma comment(lib, "opencv_imgproc247d.lib")   //MAT processing
#pragma comment(lib, "opencv_objdetect247d.lib") //HOGDescriptor
//#pragma comment(lib, "opencv_gpu247d.lib")
//#pragma comment(lib, "opencv_features2d247d.lib")
#pragma comment(lib, "opencv_highgui247d.lib")
#pragma comment(lib, "opencv_ml247d.lib")
//#pragma comment(lib, "opencv_stitching247d.lib");
//#pragma comment(lib, "opencv_nonfree247d.lib");
#pragma comment(lib, "opencv_video247d.lib")
#else
#pragma comment(lib, "opencv_core247.lib")
#pragma comment(lib, "opencv_imgproc247.lib")
#pragma comment(lib, "opencv_objdetect247.lib")
//#pragma comment(lib, "opencv_gpu247.lib")
//#pragma comment(lib, "opencv_features2d247.lib")
#pragma comment(lib, "opencv_highgui247.lib")
#pragma comment(lib, "opencv_ml247.lib")
//#pragma comment(lib, "opencv_stitching247.lib");
//#pragma comment(lib, "opencv_nonfree247.lib");
#pragma comment(lib, "opencv_video247d.lib")
#endif 

using namespace cv;
using namespace std;



int main()
{

 //global variables
 Mat frame; //current frame
 Mat resize_blur_Img;
 Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method
 Mat binaryImg;
 //Mat TestImg;
 Mat ContourImg; //fg mask fg mask generated by MOG2 method
 Ptr< BackgroundSubtractor> pMOG2; //MOG2 Background subtractor
 
 pMOG2 = new BackgroundSubtractorMOG2(300,32,true);//300,0.0);
 
 char fileName[100] = "mm2.avi"; //video\\mm2.avi"; //mm2.avi"; //cctv 2.mov"; //mm2.avi"; //";//_p1.avi";
 VideoCapture stream1(fileName);   //0 is the id of video device.0 if you have only one camera   

 //morphology element
 Mat element = getStructuringElement(MORPH_RECT, Size(7, 7), Point(3,3) );   

 //unconditional loop   
 while (true) {   
  Mat cameraFrame;   
  if(!(stream1.read(frame))) //get one frame form video   
   break;
  
  //Resize
  resize(frame, resize_blur_Img, Size(frame.size().width/3, frame.size().height/3) );
  //Blur
  blur(resize_blur_Img, resize_blur_Img, Size(4,4) );
  //Background subtraction
  pMOG2->operator()(resize_blur_Img, fgMaskMOG2, -1);//,-0.5);
  
  ///////////////////////////////////////////////////////////////////
  //pre procesing
  //1 point delete
  //morphologyEx(fgMaskMOG2, fgMaskMOG2, CV_MOP_ERODE, element);
  morphologyEx(fgMaskMOG2, binaryImg, CV_MOP_CLOSE, element);
  //morphologyEx(fgMaskMOG2, testImg, CV_MOP_OPEN, element);

  //Shadow delete
  //Binary
  threshold(binaryImg, binaryImg, 128, 255, CV_THRESH_BINARY);

  //Find contour
  ContourImg = binaryImg.clone();
  //less blob delete
  vector< vector< Point> > contours;
  findContours(ContourImg,
            contours, // a vector of contours
            CV_RETR_EXTERNAL, // retrieve the external contours
            CV_CHAIN_APPROX_NONE); // all pixels of each contours

  vector< Rect > output;
  vector< vector< Point> >::iterator itc= contours.begin();
  while (itc!=contours.end()) {

   //Create bounding rect of object
   //rect draw on origin image
   Rect mr= boundingRect(Mat(*itc));
   rectangle(resize_blur_Img, mr, CV_RGB(255,0,0));
   ++itc;
  }
  

  ///////////////////////////////////////////////////////////////////

  //Display
  imshow("Shadow_Removed", binaryImg);
  imshow("Blur_Resize", resize_blur_Img);
  imshow("MOG2", fgMaskMOG2);

  if (waitKey(5) >= 0)   
   break;   
 }

}



30 comments:

  1. Anonymous5/9/14 16:43

    Great example, it helped me a lot. Thanks

    ReplyDelete
  2. could u please tell me how to get that dataset

    ReplyDelete
  3. Hi, what dataset do you tell? mm2.avi?
    mm2.avi is sample avi file in my local hard disk.
    Thank you.

    ReplyDelete
  4. can u tell me please why u used these last values (300,32) ?
    pMOG2 = new BackgroundSubtractorMOG2(300,32,true)

    ReplyDelete
  5. refer to this opencv document..->http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html#backgroundsubtractormog-operator
    first parameter is Length of the history
    second parameter is Threshold on the squared Mahalanobis distance to decide whether it is well described by the background model (see Cthr??). This parameter does not affect the background update. A typical value could be 4 sigma, that is, varThreshold=4*4=16

    In short,
    First is the average time for a background.
    Second is Threshold for separating the foreground

    For more understanding, we have to read the algorithm paper.

    ReplyDelete
  6. Thank you so much very helpful stay blessed

    ReplyDelete
  7. kindly share the document for algorithm

    ReplyDelete
  8. i have to design a system which can robustly works against noises and illumination changes also can track multiple objects will u please help me kindly

    ReplyDelete
  9. Hi htc onex, what help do you need?

    ReplyDelete
  10. how to over come the occlusion problems in mog2 background subtraction can u give me an idea or refer document

    ReplyDelete
  11. or give me an idea to extend this code please

    ReplyDelete
  12. I think that your intersting problme is tracking topic.
    The background subtraction can be use in object tracking.
    but the method may not work well because there is not prediction and dynamic equation.
    Popular method for tracking is kalman, particle(condensation), optical flow, camshift.
    There tracking method is need initial region or object for start tracking.
    The background subtraction can be use for this initial region.

    ReplyDelete
  13. Coud u please tell me how did u get the background model

    ReplyDelete
  14. is the program using linux?
    how if I want to apply it on Visual C using windows?

    ReplyDelete
  15. The example source code is C, C++, you can test the code in the VS on window os.
    Thank you

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. I've tried it on VS C, but C code not using Mat function?
    could you help me converting that program to c code?
    thx before

    ReplyDelete
  18. great help but I want to dry bounding rectangles on resultant image not on the original image. What modifications should I perform?

    ReplyDelete
  19. You should change Mat parameter in rectangle function. (line 99).
    binaryImg, resize_blur_Img, fgMaskMOG2 is same size that is resized to small.
    Origin image is frame. If you want to draw in origin image, you should divide rectangle vlaue as resizing scale factor.
    Thank you.

    ReplyDelete
  20. Hey bro, how to Get background model from BackgroundSubtractorMOG2 in c++
    Help mee, please !! thank you so much !!

    ReplyDelete
  21. I am using Opencv 3.0 and Visual Studio 2012 This Code Not working for me please help me out.

    ReplyDelete
  22. sir i can't my code compile as compiled it as
    g++ -ggdb `pkg-config --cflags --libs opencv` bg_sub.cpp -o bg
    mant error occurs.

    ReplyDelete
  23. I got error LNK1181: cannot open input file 'opencv_calib3d2410d.lib' Any help

    ReplyDelete
  24. Hi everybody, I simulated the code but I got some error.
    firstly: invalid new-expression of abstract class type ‘cv::BackgroundSubtractorMOG2’
    pMOG2 = new BackgroundSubtractorMOG2(300,32,true); //300,0.0);
    secondly: ‘class cv::BackgroundSubtractor’ has no member named ‘operator()’
    pMOG2->operator()(resize_blur_Img, fgMaskMOG2, -1);//,-0.5);
    Any help? I am new in Computer Vision.

    ReplyDelete
    Replies
    1. What opencv version do you use sir?

      Delete
    2. I'm getting the same error, I have opencv 3.2.0 what should I change?

      Delete
    3. refer to this post
      http://study.marearts.com/2017/04/opencv-background-subtraction-32.html

      Delete
  25. how to apply tracking method to moving objects in this code?

    ReplyDelete
    Replies
    1. You can compare bounding box between t time frame and t+1 time frame.
      What compare? how coordinate and how much overlapped..
      But this is very simple way, so easy to loose object when crossing and occlusion..
      But it will be good practice.
      Obviously, tracking is very difficult problem.
      Thank you.

      Delete