10/31/2016

Real-time N camera stitching Class.

As you can see in the following video,
I created a class that stitching n cameras in real time.
https://www.youtube.com/user/feelmare/search?query=stitching

refer to this label, about stitching in my blog.
http://study.marearts.com/search/label/Stitching

The class name is CMareStitching.
This class was created to stitching_detailed examples of opencv by referencing.
So if you try, everybody can create this result.

The principle of class ... it is the main idea of ​​separating the calculation online and offline calculations such as url.
http://study.marearts.com/2015/02/real-time-stitching-multi-video-to-one.html


Main part to use this class is as follows.
...
#include < stdio.h >
#include "MareStitching.h"

#include < opencv2\opencv.hpp >     


#ifdef _DEBUG     
#pragma comment(lib, "opencv_core249d.lib")
#pragma comment(lib, "opencv_imgproc249d.lib")   //MAT processing
#else     
#pragma comment(lib, "opencv_core249.lib")
#pragma comment(lib, "opencv_imgproc249.lib")   //MAT processing
#endif


using namespace std;
using namespace cv;


void main()
{
 
 
  //CvCapture* cap = cvCaptureFromCAM( CV_CAP_DSHOW );
  //cap->VI.listDevices();
 cout << "Input Camera device number(ex: 0 enter) \n";
 int a,b;
 cin >> a;
 cin >> b;


 VideoCapture stream1(a);   //0 is the id of video device.0 if you have only one camera
 VideoCapture stream2(b);   //0 is the id of video device.0 if you have only one camera
// VideoCapture stream3(2);   //0 is the id of video device.0 if you have only one camera
// VideoCapture stream4(3);   //0 is the id of video device.0 if you have only one camera 

 
 //camera open check!...
 if (!stream1.isOpened()) //check if video device has been initialised
 {
  cout << "cannot open camera 1" << endl;
  return ;
 }else{
  cout << "camera 1 open" << endl;
 }
 
 if (!stream2.isOpened()) //check if video device has been initialised
 {
  cout << "cannot open camera 2" << endl;
  return ;
 }else{
  cout << "camera 2 open" << endl;
 }
 
/* if (!stream3.isOpened()) //check if video device has been initialised
  cout << "cannot open camera 3" << endl;
 else{
  cout << "camera 3 open" << endl;
 }
  
 
 if (!stream4.isOpened()) //check if video device has been initialised
  cout << "cannot open camera 4" << endl;
 else{
  cout << "camera 4 open" << endl;
 }*/
  
  
 namedWindow("pano2",0);
 unsigned long AAtime=0, BBtime=0;  

 Mat panoImage;
 CMareStitching MPano;

 int mode=0;
 while (true) {
  Mat cameraFrame1;
  stream1.read(cameraFrame1); //get one frame form video
  Mat cameraFrame2;
  stream2.read(cameraFrame2); //get one frame form video
//  Mat cameraFrame3;
//  stream3.read(cameraFrame3); //get one frame form video
//  Mat cameraFrame4;
//  stream4.read(cameraFrame4); //get one frame form video

  //cout << cameraFrame1.size().width << " " << cameraFrame1.size().height << endl;

  if(mode == 0)
  {
   imshow("cam1", cameraFrame1);
   imshow("cam2", cameraFrame2);
//   imshow("cam3", cameraFrame3);
//   imshow("cam4", cameraFrame4);
  }

  if(mode == 1)
  {
   printf("panorama parameter processing...\n");
   vector< Mat > vImg_color;
   vImg_color.push_back( cameraFrame1 );
   vImg_color.push_back( cameraFrame2 );
//   vImg_color.push_back( cameraFrame3 );
//   vImg_color.push_back( cameraFrame4 );

   AAtime = getTickCount();
   if( MPano.firstStitching(vImg_color, panoImage) == false)
    cout << "stitching fail " << endl << endl;
   else
   {
    imshow("pano2", panoImage );
    mode = 2;
   } 
   //Processing time measurement   
   BBtime = getTickCount(); 
   printf("Processing time = %.2lf(sec) \n",  (BBtime - AAtime)/getTickFrequency() );      
  }

  if(mode == 2)
  {
   vector< Mat > vImg_color;
   vImg_color.push_back( cameraFrame1 );
   vImg_color.push_back( cameraFrame2 );
//   vImg_color.push_back( cameraFrame3 );
//   vImg_color.push_back( cameraFrame4 );

   AAtime = getTickCount();
   if( MPano.laterStitching(vImg_color, panoImage) == false)
    cout << "stitching fail " << endl << endl;
   else
   {    
    imshow("pano2", panoImage );
   }
   //Processing time measurement   
   BBtime = getTickCount(); 
   printf("Processing time = %.2lf(sec) \n",  (BBtime - AAtime)/getTickFrequency() ); 

   //imshow("cam1", cameraFrame1);
   //imshow("cam2", cameraFrame2);

  }

  char k = waitKey(10);
  if ( k == 'p') //processing command
   mode=1;
  else if (k == 'r') //re camera mode
   mode=0;
  else if (k == 'q') //quit
   break;
 }

 destroyAllWindows();

 
}
...

First enter the index to camera stitching.
Press "p" is a real-time panoramic image after matching relations calculation.
Pressing "r" again return to the camera image.
It is quit by pressing the "q".

When you press a key it must be pressed on the video screen active.
The command window active, and not input the key.

As you see the main source code, there is no easy interface only code. so, you have to modify the code for your purpose, ex) the camera number.

The project code is uploaded privately on github but not free.
https://github.com/MareArts/Realtime_N_camera_Stitching
Why not share FREE asked, 2 reasoning, an little effort of me and Fundraising operating costs of blog(for other planed project)... very Sorry.

Below url is header, lib, dll files for developing.
You have to set path in visual studio.
https://www.amazon.com/clouddrive/share/zv7xEoQ0iBm0Cnaw76aTdQuTE1pdPskba0hgqJdb42n?ref_=cd_ph_share_link_copy

And this url is demo files. you can test to download.
The url includes execute file and related dlls.
https://www.amazon.com/clouddrive/share/zOZXmvrYTb4Er0OYg5qezmk24OPKq0znWlZKly4ZC01?ref_=cd_ph_share_link_copy


And this project coded by win32, opencv 249, visual studio 2012.

Thank you.



4 comments:

  1. Hello, Mare,
    the following video is in your private channel, not for public viewing.
    https://www.youtube.com/my_videos?o=U&sq=stitching

    I am planning to do something similar, so I am interested in your video.
    By the way, Great job!
    Thanks!
    Peter

    ReplyDelete
    Replies
    1. Thank you for your interest.
      url has been modified

      Delete
  2. Hello Mare,

    I´ve got a question. When an object passes near the cameras, is the homography lost?

    Tx u and congrats, you have done a great job

    ReplyDelete
    Replies
    1. The homography matrix is calculated only once through the captured image from cameras.
      After routine, stitching is doing using the homography matrix.
      Thank you.

      Delete