Real-time stitching multi-video to one screen

* Introduction

- The solution shows panorama image from multi images. The panorama images is processing by real-time stitching algorithm.

- Each cameras has a limited field of view, but the solution can be monitoring large areas from merged into a panorama image.

- The performance is excellent with the following technical configuration.
 . real time image processing using GPU.
 . Accurate calculation of R, T, K (Rotation, Translation, Camera intrinsic) between each camera with nonlinear optimization
 . Color calibration using the exposure blending

- The solution can be applied efficiently and easy in Military Region, tourist attractions, intersections, ports

* Real-time N to 1 stitching algorithm

- Existing stitching algorithm is modified to separate 2 parts of offline and online processing for more efficient realtime processing.

- The Off-Line processing part is calculated first time or if the matching inaccurate. 

- On-Line processing part is a routine to create the panoramic image by warping (Warping) calculated by the matching, the blending value.

No ordered input images

- Feature extraction and to calculate the homography matrix between each image by evaluating (RANSAC), and set image position through matching rate.
- To get correct R, T using bundle adjustment

- Searching the overlap region, the blending coefficient is determined with respect to the non-overlapping region.

- Obtained R, T, K, and connected by warping the images and blending and complete the panorama finally

* Experiment

- 4 real-time video stitching speed of about 10~20 fps (Intel® core™ i5-3570 cpu 3.40GHz, NVIDIA Geforce GTX 650)

See the result on youtube

updating 42/06/2018
I have decided to sell source code ^^
If you have interest, go to here, you can buy source code


  1. Hi, how can you use STITCHER class with gpu? i tried to build openCV 3.1 with cuda. I set "Stitcher stitch = Stitcher::createDefault(true); " but the function "stitch" reported an error

    1. This is not gpu mode. However, I used this class in version 2.4.7
      refer to this url

    2. This comment has been removed by the author.

  2. Hey, what a nice work!
    Can you explain or refer some links for me about the bundle adjustment. I'm stuck in this step, my panorama alignment can't be center, it keep expanding at the end :(
    Thank you.

    1. It is difficult to answer in detail here ..
      This is to get several unknowns values at once(actually, it iterative method).
      In other words ... it use nonlinear optimization techniques to get various parameters of the camera.
      Sorry. I can not help you now...
      There is a lot of information on the internet.

    2. I can tell you some keywrods.
      - Levenberg-Marquardt
      - Newton's method, Gauss-Newton method
      - unconstrained nonlinear optimization

  3. Hi, How do you determine if you need to update

    1. I think there are 2 things.
      1. when matching error is bigger than set threshold.
      2. running time over than set threshold time.

      Thank you.

  4. 안녕하세요 . Real-time stitching multi-video to one screen 해당 소스를 구하고 싶습니다.
    소정의 비용이 필요 하다고 하셨는데 그비용이 어떻게 되는지 yjh@mog.kr로 답변 부탁 드립니다.

    1. 메일 보냈습니다. 감사합니다.

  5. Hi,The 4 cameras video-stitching,I saw the people moving across the seam,the stitching-artifacts is happening.
    How to reduce the facts? Now,it's a problem to me for some days,Do you have some ideas?

    1. I think that is focal length problem, if object which distance close than we stitched move across than it happen stitching-artifacts.
      I have several ideas but I didn't make code, so I am not sure what is solution.
      Thank you.

  6. 소정의 비용을 지불하고 소스 사용이 가능한지 문의 드립니다.

    1. I have the code and demo,you can connect to me yiweijian1995@gmail.com

  7. ihave a question. Does your stiching program support two rtsp input? now i know it support cam and image input