Real-time stitching multi-video to one screen

* Introduction

- The solution shows panorama image from multi images. The panorama images is processing by real-time stitching algorithm.

- Each cameras has a limited field of view, but the solution can be monitoring large areas from merged into a panorama image.

- The performance is excellent with the following technical configuration.
 . real time image processing using GPU.
 . Accurate calculation of R, T, K (Rotation, Translation, Camera intrinsic) between each camera with nonlinear optimization
 . Color calibration using the exposure blending

- The solution can be applied efficiently and easy in Military Region, tourist attractions, intersections, ports

* Real-time N to 1 stitching algorithm

- Existing stitching algorithm is modified to separate 2 parts of offline and online processing for more efficient realtime processing.

- The Off-Line processing part is calculated first time or if the matching inaccurate. 

- On-Line processing part is a routine to create the panoramic image by warping (Warping) calculated by the matching, the blending value.

No ordered input images

- Feature extraction and to calculate the homography matrix between each image by evaluating (RANSAC), and set image position through matching rate.
- To get correct R, T using bundle adjustment

- Searching the overlap region, the blending coefficient is determined with respect to the non-overlapping region.

- Obtained R, T, K, and connected by warping the images and blending and complete the panorama finally

* Experiment

- 4 real-time video stitching speed of about 10~20 fps (Intel® core™ i5-3570 cpu 3.40GHz, NVIDIA Geforce GTX 650)

See the result on youtube

 - https://www.youtube.com/watch?v=e6ociYYQS4U

 - https://www.youtube.com/watch?v=jP1Y8nd0mQc

 - https://www.youtube.com/watch?v=Zkrn-rooj0M

 - https://www.youtube.com/watch?v=g9pL7cyL8H0

 - https://www.youtube.com/watch?v=xqnHBaZU1R4

about source code and Stitching class here :
http://study.marearts.com/2016/10/real-time-n-camera-stitching-class.html

updating 42/06/2018
I have decided to sell source code ^^
If you have interest, go to here, you can buy source code

** 2021.06 updated ** 

realtime stitching SDK: 

https://www.marearts.com/Realtime-Stitching-SDK-ba67e56550a14688835f291f891af4ac

Dataset for car license plate detection

https://www.marearts.com/Dataset-for-car-license-plate-detection-6f1e623b60c94e54ac6f55bbd5a6a917

In Tensorflow, get the names of all the Tensors in a graph

To get all nodes in the graph: (type tensorflow.core.framework.node_def_pb2.NodeDef)

all_nodes = [n for n in tf.get_default_graph().as_graph_def().node]

To get all ops in the graph: (type tensorflow.python.framework.ops.Operation)

all_ops = tf.get_default_graph().get_operations()

To get all variables in the graph: (type tensorflow.python.ops.resource_variable_ops.ResourceVariable)

all_vars = tf.global_variables()

To get all tensors in the graph: (type tensorflow.python.framework.ops.Tensor)

all_tensors = [tensor for op in tf.get_default_graph().get_operations() for tensor in op.values()]

To get all placeholders in the graph: (type tensorflow.python.framework.ops.Tensor)

all_placeholders = [placeholder for op in tf.get_default_graph().get_operations() if op.type=='Placeholder' for placeholder in op.values()]

Tensorflow 2

To get the graph in Tensorflow 2, instead of tf.get_default_graph() you need to instantiate a tf.function first and access the graph attribute, for example:

graph = func.get_concrete_function().graph

where func is a tf.function

install python3.7 on ubuntu 20

 sudo add-apt-repository ppa:deadsnakes/ppa

sudo apt-get update

sudo apt-get install python3.7

Managing cuda version for different Tensorflow version

Tensorflow 1.15.0 or 1.14.0 may require cuda 10.0 or 10.1

And your latest version of Tensorflow 2.x use cuda 11.1 or 11.0

At this situation, you need to set proper path setting.

refer to below command

>sudo nano ~/.profile

# set PATH for cuda 10.0 installation

if [ -d "/usr/local/cuda-10.0/bin/" ]; then

export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}

export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

fi

# set PATH for cuda 10.1 installation

if [ -d "/usr/local/cuda-10.1/bin/" ]; then

export PATH=/usr/local/cuda-10.1/bin${PATH:+:${PATH}}

export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

fi

# set PATH for cuda 11.2 installation

if [ -d "/usr/local/cuda-11.2/bin/" ]; then

export PATH=/usr/local/cuda-11.2/bin${PATH:+:${PATH}}

export LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

fi

reboot after saving

Thank you.

python source code to print list of all subfolder and files

 

for path, subdirs, files in os.walk('/your_folder/'):

for name in files:

print(os.path.join(path, name))

Install python 3.8 on Linux

 

Following below step:

-------------

wget https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tgz

tar xvf Python-3.8.3.tgz

cd Python-3.8*/

./configure --enable-optimizations

make altinstall

-------------