6/06/2021

Real-time stitching multi-video to one screen

* Introduction

- The solution shows panorama image from multi images. The panorama images is processing by real-time stitching algorithm.

- Each cameras has a limited field of view, but the solution can be monitoring large areas from merged into a panorama image.

- The performance is excellent with the following technical configuration.
 . real time image processing using GPU.
 . Accurate calculation of R, T, K (Rotation, Translation, Camera intrinsic) between each camera with nonlinear optimization
 . Color calibration using the exposure blending

- The solution can be applied efficiently and easy in Military Region, tourist attractions, intersections, ports



* Real-time N to 1 stitching algorithm

- Existing stitching algorithm is modified to separate 2 parts of offline and online processing for more efficient realtime processing.

- The Off-Line processing part is calculated first time or if the matching inaccurate. 

- On-Line processing part is a routine to create the panoramic image by warping (Warping) calculated by the matching, the blending value.



No ordered input images

- Feature extraction and to calculate the homography matrix between each image by evaluating (RANSAC), and set image position through matching rate.
- To get correct R, T using bundle adjustment

- Searching the overlap region, the blending coefficient is determined with respect to the non-overlapping region.

- Obtained R, T, K, and connected by warping the images and blending and complete the panorama finally


* Experiment


- 4 real-time video stitching speed of about 10~20 fps (Intel® core™ i5-3570 cpu 3.40GHz, NVIDIA Geforce GTX 650)



See the result on youtube



updating 42/06/2018
I have decided to sell source code ^^
If you have interest, go to here, you can buy source code


** 2021.06 updated ** 
realtime stitching SDK: 
μž‘μ„±μž: μ‹œκ°„:
Labels: , , , ,

19 comments:

  1. Unknown18/1/17 01:17

    Hi, how can you use STITCHER class with gpu? i tried to build openCV 3.1 with cuda. I set "Stitcher stitch = Stitcher::createDefault(true); " but the function "stitch" reported an error

    ReplyDelete
    Replies
    1. mare18/1/17 23:59

      This is not gpu mode. However, I used this class in version 2.4.7
      refer to this url
      http://study.marearts.com/2013/11/opencv-stitching-example-stitcher-class.html

      Delete
    2. Unknown27/3/17 09:37

      This comment has been removed by the author.

      Delete
  2. Unknown27/3/17 09:39

    Hey, what a nice work!
    Can you explain or refer some links for me about the bundle adjustment. I'm stuck in this step, my panorama alignment can't be center, it keep expanding at the end :(
    Thank you.

    ReplyDelete
    Replies
    1. mare28/3/17 17:18

      It is difficult to answer in detail here ..
      This is to get several unknowns values at once(actually, it iterative method).
      In other words ... it use nonlinear optimization techniques to get various parameters of the camera.
      Sorry. I can not help you now...
      There is a lot of information on the internet.

      Delete
    2. mare28/3/17 17:26

      I can tell you some keywrods.
      - Levenberg-Marquardt
      - Newton's method, Gauss-Newton method
      - unconstrained nonlinear optimization

      Delete
  3. Unknown23/6/17 00:00

    Hi, How do you determine if you need to update

    ReplyDelete
    Replies
    1. mare30/9/17 11:51

      I think there are 2 things.
      1. when matching error is bigger than set threshold.
      2. running time over than set threshold time.

      Thank you.

      Delete
  4. Unknown28/9/17 23:12

    μ•ˆλ…•ν•˜μ„Έμš” . Real-time stitching multi-video to one screen ν•΄λ‹Ή μ†ŒμŠ€λ₯Ό κ΅¬ν•˜κ³  μ‹ΆμŠ΅λ‹ˆλ‹€.
    μ†Œμ •μ˜ λΉ„μš©μ΄ ν•„μš” ν•˜λ‹€κ³  ν•˜μ…¨λŠ”λ° κ·ΈλΉ„μš©μ΄ μ–΄λ–»κ²Œ λ˜λŠ”μ§€ yjh@mog.kr둜 λ‹΅λ³€ 뢀탁 λ“œλ¦½λ‹ˆλ‹€.

    ReplyDelete
    Replies
    1. mare30/9/17 11:47

      메일 λ³΄λƒˆμŠ΅λ‹ˆλ‹€. κ°μ‚¬ν•©λ‹ˆλ‹€.

      Delete
  5. Unknown13/3/18 06:24

    Hi,The 4 cameras video-stitching,I saw the people moving across the seam,the stitching-artifacts is happening.
    How to reduce the facts? Now，it's a problem to me for some days,Do you have some ideas?

    ReplyDelete
    Replies
    1. mare11/4/18 18:32

      I think that is focal length problem, if object which distance close than we stitched move across than it happen stitching-artifacts.
      I have several ideas but I didn't make code, so I am not sure what is solution.
      Thank you.

      Delete
  6. Unknown2/10/18 00:27

    μ†Œμ •μ˜ λΉ„μš©μ„ μ§€λΆˆν•˜κ³  μ†ŒμŠ€ μ‚¬μš©μ΄ κ°€λŠ₯ν•œμ§€ 문의 λ“œλ¦½λ‹ˆλ‹€.
    jmlee@genobio.co.kr

    ReplyDelete
    Replies
    1. Unknown26/10/18 11:26

      I have the code and demo，you can connect to me yiweijian1995@gmail.com

      Delete
  7. εƒδΈ‡ι‡Œηš„θΏ½ε―»18/10/18 22:28

    ihave a question. Does your stiching program support two rtsp input? now i know it support cam and image input

    ReplyDelete
  8. olivermir6/9/19 04:10

    Computerized and some electronic machines are programmed to make the buttonhole in one step with no need to stop the machine to turn the fabric. computerized sewing machines

    ReplyDelete
  9. UsamaAk21/3/20 05:20

    You will need to know this if you buy any additional presser feet for the machine. the best starter sewing machine

    ReplyDelete
  10. SonijaRanker12/7/20 04:20

    Nice to be visiting your blog again, it has been months for me. Well this article that i’ve been waited for so long. I need this article to complete my assignment in the college, and it has same topic with your article. Thanks, great share. mini sewing machine

    ReplyDelete
  11. IanTaylor5/10/20 11:28

    Hi I would like to purchase the stitching class, but on your website it says "out of stock" is it still available for purchase, or is permanently removed?

    ReplyDelete

Subscribe to: Post Comments (Atom)