Mr. Juan ask me that to categorize camera movement rate into big, middle, small.
He is researching to find meaningful scene in endoscopic video.
His one of approach is using optical flow.
refer to this page. http://study.marearts.com/2015/01/the-method-to-check-that-camera-is.html
Dense optical flow can know how much video moving.
The source code of reference page checked camera move or not by the percentage of movement.
There are 2 threshold values.
Count number of moved pixels, this is counted when pixels moved over than threshold-1 value.
And check whether the counted pixels is over than threshold-2 percent or not in all pixels.
By the way, he ask me to separate video moving by big, middle, small using k-means clustering algorithm without threshold.
In past, I introduced the usage of k-mean algorithm using openCV.
See this page. http://study.marearts.com/search/label/K-means
This work is separated by 3 steps.
Step 1 is to calculate movement rate in video using optical flow.
And save fame and movement rate information to txt file.
Step 2 is clustering by 3 class degree movement using k-means.
read step 1 file and write clustering information to txt file.
Step 3 is for display.
read step 3 txt file and display moving rate on the video.
Example source code of K-means algorithm in OpenCV,
The source code are two version, one is python and other is C++.
And I compare processing time, I do same condition such as same image, same parameter, and I checked same result.
The winner of processing speed is C++.
Despite C++ contain many "for" syntax but faster than python.
Check example source code.
This is input image
C++ version.
...
#include < stdio.h>
#include < iostream>
#include < opencv2\opencv.hpp>
#ifdef _DEBUG
#pragma comment(lib, "opencv_core247d.lib")
#pragma comment(lib, "opencv_imgproc247d.lib") //MAT processing
#pragma comment(lib, "opencv_highgui247d.lib")
#else
#pragma comment(lib, "opencv_core247.lib")
#pragma comment(lib, "opencv_imgproc247.lib")
#pragma comment(lib, "opencv_highgui247.lib")
#endif
using namespace cv;
using namespace std;
void main()
{
unsigned long AAtime=0, BBtime=0; //check processing time
unsigned long inAtime=0, inBtime=0;
AAtime = getTickCount(); //check processing time
inAtime = getTickCount(); //check processing time
Mat src = imread( "mare-08.jpg", 1 );
Mat samples(src.rows * src.cols, 3, CV_32F);
for( int y = 0; y < src.rows; y++ )
for( int x = 0; x < src.cols; x++ )
for( int z = 0; z < 3; z++)
samples.at< float>(y + x*src.rows, z) = src.at< Vec3b>(y,x)[z];
inBtime = getTickCount(); //check processing time
printf("in Data preparing %.2lf sec \n", (inBtime - inAtime)/getTickFrequency() ); //check processing time
inAtime = getTickCount(); //check processing time
int clusterCount = 5;
Mat labels;
int attempts = 10;
Mat centers;
kmeans(samples, clusterCount, labels, TermCriteria(CV_TERMCRIT_ITER|CV_TERMCRIT_EPS, 10, 1.0), attempts, KMEANS_RANDOM_CENTERS, centers );
inBtime = getTickCount(); //check processing time
printf("K mean processing %.2lf sec \n", (inBtime - inAtime)/getTickFrequency() ); //check processing time
inAtime = getTickCount(); //check processing time
Mat new_image( src.size(), src.type() );
for( int y = 0; y < src.rows; y++ )
{
for( int x = 0; x < src.cols; x++ )
{
int cluster_idx = labels.at< int>(y + x*src.rows,0);
new_image.at< Vec3b>(y,x)[0] = centers.at< float>(cluster_idx, 0);
new_image.at< Vec3b>(y,x)[1] = centers.at< float>(cluster_idx, 1);
new_image.at< Vec3b>(y,x)[2] = centers.at< float>(cluster_idx, 2);
}
}
inBtime = getTickCount(); //check processing time
printf("out Data Preparing processing %.2lf sec \n", (inBtime - inAtime)/getTickFrequency() ); //check processing time
BBtime = getTickCount(); //check processing time
printf("Total processing %.2lf sec \n", (BBtime - AAtime)/getTickFrequency() ); //check processing time
//imshow( "clustered image", new_image );
imwrite("clustered_image.jpg", new_image);
//waitKey( 0 );
}
...
result image and processing time of C++ version
Python Version
...
import numpy as np
import cv2
from matplotlib import pyplot as plt
e1 = cv2.getTickCount()
inA = cv2.getTickCount()
img = cv2.imread('mare-08.jpg')
Z = img.reshape((-1,3))
# convert to np.float32
Z = np.float32(Z)
inB = cv2.getTickCount()
print("in data preparing", (inB-inA)/cv2.getTickFrequency())
# define criteria, number of clusters(K) and apply kmeans()
inA = cv2.getTickCount()
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
K = 5
ret,label,center = cv2.kmeans(Z,K,criteria,10,cv2.KMEANS_RANDOM_CENTERS)
inB = cv2.getTickCount()
print("K-means ", (inB-inA)/cv2.getTickFrequency())
# Now convert back into uint8, and make original image
inA = cv2.getTickCount()
center = np.uint8(center)
res = center[label.flatten()]
res2 = res.reshape((img.shape))
inB = cv2.getTickCount()
print("out data preparing", (inB-inA)/cv2.getTickFrequency())
#print(center)
#print(label)
e2 = cv2.getTickCount()
time = (e2 - e1)/cv2.getTickFrequency()
print("total time", time, 1/time)
cv2.imshow('res2',res2)
cv2.waitKey(0)
cv2.destroyAllWindows()
