No module named 'yaml'

pip install pyyaml

delete over than specific megabyte in git history using bfg

firstly, install bfg

on mac

> brew install bfg

delete object if over than 50m in git commit history

> bfg --strip-blobs-bigger-than 50M

make sure & apply .gitignore

First, to check what files are you actually tracking
> git ls-tree --name-only --full-tree -r HEAD


Let say that you found unwanted files in a directory like cache/ so, it's safer to target that directory instead of all of your files.
So instead of:

> git rm -r --cached .


It's safer to target the unwanted file or directory:
> git rm -r --cached cache/


Then proceed to add all changes:
> git add .


and commit
> git commit -m ".gitignore is now working"

list up and sorting file(object) size in git / command

 use following cmd:

> git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | sed -n 's/^blob //p' | sort --numeric-sort --key=2 | cut -c 1-12,41- | $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

get YouTube video url from channel name, python youtube-dl library

 

install youtube-dl package first.

pip install YouTube-dl

<code>

import subprocess

direct_output = subprocess.check_output('youtube-dl --get-id https://www.youtube.com/channel/UCAwWYtU_DEdRFxEHl9879dRYBfQ/videos --no-check-certificate', shell=True)

Lines = direct_output.decode()

idv = Lines.split('\n')

play_url_list = []

for idv in Lines.split('\n'):

if idv == '':

continue

print(idv)

play_url_list.append( 'https://www.youtube.com/watch?v={}'.format(idv) )

</code>

Thank you.

Download or streaming YouTube video through cv2, python sample code

firstly, install pafy python package

> pip install pafy

if you some this related error "youtube_dl", install this. pafy works based on this package.

> pip install youtube-dl

you can check youtube-dl version:

> youtube-dl --version

if you also have some certification error, you can use --no-check-certification option

try this and test, this command will download video.

> sudo youtube-dl --no-check-certificate https://www.youtube.com/watch?v=N8A-BxTfTaY

ok then everything will be work using following code:

import pafy

from matplotlib import pyplot as plt

import cv2

url = "https://www.youtube.com/watch?v=TKbNDqcgjzw"

video = pafy.new(url, ydl_opts={'nocheckcertificate': True})

best = video.getbest(preftype="mp4")

cap = cv2.VideoCapture()

cap.open(best.url)

while(True):

# Capture frame-by-frame

ret, frame = cap.read()

# Display the resulting frame

cv2.imshow('frame',frame)

if cv2.waitKey(1) & 0xFF == ord('q'):

break

# When everything done, release the capture

cap.release()

cv2.destroyAllWindows()

Thank you.

python notebook clear output

 

from IPython.display import clear_output

for i in range(10):
    clear_output(wait=True)
    print("Hello World!")

generate .spec file with onefile option for pyinstaller

Use

pyi-makespec --onefile yourprogram.py

to generate a sample spec file for onefile mode.




reference :

https://stackoverflow.com/questions/47143315/using-onefile-with-a-spec-in-pyinstaller

rsync exclude file and folder

 

command is like that:

rsync -avz --progress ./source ./destination --exclude-from './exclude-list.txt'

you can make ignore file and folder like this:

 exclude-list.txt

*.txt
*.jpg
*.jpeg
*.zip
*.ipynb
.git
*.json
*.done
*.png
*.xml
*.pdf
*.pyc
*/build
*/dist
*/experiment
*/grapher_outputs
*.pkl
*/system_evaluation

linux screen command list in summary

*create screen
screen -S name

*leave with alive
Ctrl a, d

*enter screen 

screen -r name 

*kill specific screen
screen -S name -X quit

*show screen list
screen -ls

*detach specific screen

screen -d name

*kill all screen
killall screen

shuffle dict in python

 

code

import random

d = {'a':[1,2], 'b':[2,4], 'c':[3,5], 'd':[2,4]}

l = list(d.items())

random.shuffle(l)

d = dict(l)

print(d)

result

{'b': [2, 4], 'd': [2, 4], 'a': [1, 2], 'c': [3, 5]}

torch.nan_to_num not found error

torch.nan_to_num

This is for 1.8 version.

so use it instead of this.

temp[torch.isnan(temp)] = 0

temp is tensor

bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

 pip install lxml

get parameter from parameter store of aws system manager, python example code

firstly create parameter in parameter store of AWS system manager 

Then get the key from this code.

code

import boto3

ssm = boto3.client('ssm')

parameter = ssm.get_parameter(Name='/project/key_endpoint', WithDecryption=True)

print(parameter['Parameter']['Value'])

.

s3 bucket copy object to another bucket, python example

 code

def copy_s3_object(s3_resource, source_bucket_name, source_key, target_bucket_name, target_key):

copy_source = {'Bucket': source_bucket_name, 'Key': source_key}

s3_resource.meta.client.copy(copy_source, target_bucket_name, target_key)

s3_resource = boto3.resource('s3')

copy_s3_object(s3_resource, source_bucket_name, source_key, target_bucket_name, target_key)

.

aws s3 get all object more than 1000 python example code

simply to use paginator instance

example code

paginator = s3_client.get_paginator('list_objects_v2')

pages = paginator.paginate(Bucket='bucket', Prefix='folder1/')

for page in pages:

for obj in page['Contents']:

print(obj['Key'])

.

sci sparse -> tuple list -> sci sparse

 refer to below source code

source code start

from scipy.sparse import csr_matrix

#sci sparse to tuple list

c = A2.tocoo() #A2 is scipy.sparse.csr.csr_matrix

in_edge_idx = list(zip(c.row, c.col)) #make tuple list

#tuple list to sci sparse

two_list = list(map(list, zip(*in_edge_idx))) #tuple 2 tow list of list [[1,2,3], [2,3,4]]

rows = np.array(two_list[0]) #rows

cols = np.array(two_list[1]) #cols

data_num = len(rows) #number of edge

data = np.ones( data_num ) #edge value

dim = len(x_data) #N x N adj

#sci sparse -> tuple list -> sci sparse

re_edge_idx = csr_matrix((data, (rows, cols)), shape=(dim, dim))

print('in', A2, type(A2))

print('re', re_edge_idx, type(re_edge_idx))

#origin A2 and re-generated edge index same?

print( (A2!=re_edge_idx).nnz==0 )

source code end

VS code SSH fails to connect: Connecting was canceled

Type this one in server side.

rm -rf ~/.vscode-server/

draw roc curve using python sklearn, Matplotlib

import matplotlib.pyplot as plt

from sklearn.metrics import roc_auc_score, average_precision_score

from sklearn import metrics

gt = [1, 0, 1, 0, 1, 1] #origin

pre = [0.9, 0.5, 0.8, 0.4, 0.5, 0.8] #predict

fpr, tpr, thresholds = metrics.roc_curve(gt, pre)

roc_auc = metrics.auc(fpr, tpr)

fig, ax = plt.subplots(figsize=(10,7))

ax.plot(fpr, tpr, label='ROC curve (area = %0.2f)' % roc_auc)

ax.plot(np.linspace(0, 1, 100),

np.linspace(0, 1, 100),

label='baseline',

linestyle='--')

plt.title('Receiver Operating Characteristic Curve', fontsize=18)

plt.ylabel('TPR', fontsize=16)

plt.xlabel('FPR', fontsize=16)

plt.legend(fontsize=12) 

print gpu memory status in python

*install pynvml

https://pypi.org/project/pynvml/

pip install pynvml

*use below code in python code

from pynvml import *
nvmlInit()
h = nvmlDeviceGetHandleByIndex(0)
info = nvmlDeviceGetMemoryInfo(h)
print(f'total    : {info.total}')
print(f'free     : {info.free}')
print(f'used     : {info.used}')

remove duplicated tuple item in list (python code)

 

print(tuple_list)

tuple_list = [ tuple(sorted(tuple_list[i])) for i in range(len(tuple_list))]

tuple_list = list(set(tuple_list))

print(tuple_list)

before:

[(0, 0), (0, 1), (0, 3), (1, 0), (1, 1), (1, 2), (2, 1), (2, 2), (2, 3), (3, 0), (3, 2), (3, 3)]

After:
[(0, 1), (1, 2), (0, 0), (3, 3), (2, 3), (2, 2), (0, 3), (1, 1)]

No module named 'PIL'

pip install --upgrade pip

 pip install pillow

Pytorch, Infinite DataLoader using iter & next

 

# create dataloader-iterator

data_iter = iter(data_loader)

# iterate over dataset

# alternatively you could use while(True)

for i in range(NUM_ITERS_YOU_WANT)

try:

data = next(data_iter)

except StopIteration:

# StopIteration is thrown if dataset ends

# reinitialize data loader

data_iter = iter(data_loader)

data = next(data_iter)

python argparse example

import argparse

paser = argparse.ArgumentParser()

args = paser.parse_args("")

args.cuda = False

args.show_summary = False

args.device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

print(args.cuda)

find best (optimal) threshold using roc curve

 def plot_roc_curve(fpr, tpr):

    plt.plot(fpr, tpr, color='orange', label='ROC')
    plt.plot([0, 1], [0, 1], color='darkblue', linestyle='--')
    plt.xlabel('False Positive Rate')
    plt.ylabel('True Positive Rate')
    plt.title('Receiver Operating Characteristic (ROC) Curve')
    plt.legend()
    plt.show()

y_true = np.array([0,0, 1, 1,1])
y_scores = np.array([0.0,0.09, .05, .75,1])

fpr, tpr, thresholds = roc_curve(y_true, y_scores)
print(tpr)
print(fpr)
print(thresholds)
print(roc_auc_score(y_true, y_scores))
optimal_idx = np.argmax(tpr - fpr)
optimal_threshold = thresholds[optimal_idx]
print("Threshold value is:", optimal_threshold)
plot_roc_curve(fpr, tpr)

What AUC(area under curve) value is better ?

 What AUC(area under curve) value is better ?

0.9 ~ 1 : excellent
0.8 ~ 0.9: good
0.7 ~ 0.8 : normal
0.6 ~ 0.7 : poor
0.5 ~ 0.6 : fail

python measure processing time

 

from time import process_time

# Start the stopwatch / counter

t1_start = process_time()

###

#processing

###

# Stop the stopwatch / counter

t1_stop = process_time()

sec = t1_stop-t1_start

split train test dataset

 

import random

from sklearn.model_selection import train_test_split

random.shuffle(pkl_list)

pkl_train, pkl_test = train_test_split(pkl_list, test_size=0.2)

show image in jupyter notebook

 

from matplotlib import pyplot as plt

import numpy as np

import cv2

img = imread('xxx.png') #or image_data

img2 = img[:,:,::-1]

plt.imshow(img)

fix hangul separating issue in mac

 

from unicodedata import normalize

def nfd2nfc(data):

return normalize('NFC', data)

normalize('ㄷ ㅓ')

-> 더 

python change file name, get file name, dir, ext, check file exist in source code using os package

 

get file name and ext

import os

os.path.splitext("/path/to/some/file.txt")[0]

#/path/to/some/file

base = os.path.basename('/root/dir/sub/file.ext')

#'file.ext'

os.path.splitext(base)

#('file', '.ext')

os.path.splitext(base)[0]

#'file'

get dir

os.path.dirname("/path/to/some/file.txt")

#'/path/to/some'

change file name 

os.rename(r'C:\Users\Ron\Desktop\Test\Products.txt',r'C:\Users\Ron\Desktop\Test\Shipped Products.txt')

check file exist

os.path.isfile('./path_of_file')

sparse tensor to csr_matrix

from scipy.sparse import csr_matrix

import numpy as np

x = val_data.x

dim = len(x)

print(dim)

edge_index = val_data.edge_index

print(edge_index) #sparse tensor

row = edge_index[0].numpy()

col = edge_index[1].numpy()

edge_num = len(row)

data = np.ones( edge_num )

mtx = csr_matrix((data, (row, col)), shape=(dim, dim))

#print( type(mtx.toarray()), mtx.toarray().shape)

print( mtx.toarray(), type(mtx.toarray()), mtx.toarray().shape) 

let's image 

val_data.x is node features ex) 13x1000

val_data.edge_index is sparse edge index stored by torch tensor

now we want to convert it to csr_matrix

The above code is example for this case.

The print out is like this:

tensor([[ 0,  0,  0,  1,  1,  1,  1,  1,  2,  2,  2,  2,  3,  3,  3,  3,  3,  4,
          4,  4,  4,  4,  4,  5,  5,  5,  5,  5,  6,  6,  6,  6,  6,  7,  7,  7,
          8,  8,  8,  9,  9,  9,  9,  9, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11,
         11, 11, 12, 12, 12, 12],
        [ 1,  3, 10,  0,  2,  3, 10, 11,  1,  3, 11, 12,  0,  1,  2, 11, 12,  5,
          6,  8,  9, 11, 12,  4,  6,  7,  8,  9,  4,  5,  7,  9, 10,  5,  6,  8,
          4,  5,  7,  4,  5,  6, 10, 11,  0,  1,  6,  9, 11,  1,  2,  3,  4,  9,
         10, 12,  2,  3,  4, 11]])
[[0. 1. 0. 1. 0. 0. 0. 0. 0. 0. 1. 0. 0.]
 [1. 0. 1. 1. 0. 0. 0. 0. 0. 0. 1. 1. 0.]
 [0. 1. 0. 1. 0. 0. 0. 0. 0. 0. 0. 1. 1.]
 [1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 0. 0. 1. 1. 0. 1. 1. 0. 1. 1.]
 [0. 0. 0. 0. 1. 0. 1. 1. 1. 1. 0. 0. 0.]
 [0. 0. 0. 0. 1. 1. 0. 1. 0. 1. 1. 0. 0.]
 [0. 0. 0. 0. 0. 1. 1. 0. 1. 0. 0. 0. 0.]
 [0. 0. 0. 0. 1. 1. 0. 1. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 1. 1. 1. 0. 0. 0. 1. 1. 0.]
 [1. 1. 0. 0. 0. 0. 1. 0. 0. 1. 0. 1. 0.]
 [0. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0. 1. 0.]] <class 'numpy.ndarray'> (13, 13)

Thank you

Enjoy Pytorch!

error : Bad config encountered during initialization, when you run jupyter

 

type this

jupyter notebook --generate-config

run juypter again

good luck!

image augmentation by python

pip install imgaug

pip install imagecorruptions

github : https://github.com/aleju/imgaug

import numpy as np

import imgaug as ia

import imgaug.augmenters as iaa

import cv2

def agument_rewrite(file_list):

sometimes = lambda aug: iaa.Sometimes(0.1, aug)

seq = iaa.Sequential(

[

# apply the following augmenters to most images

sometimes(iaa.CropAndPad(percent=(-0.02, 0.02), pad_mode=ia.ALL, pad_cval=(0, 255))),

sometimes(iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05*255), per_channel=0.5)), # add gaussian noise to images

sometimes(iaa.Dropout(p=(0, 0.2))),

sometimes(iaa.CoarseDropout(0.02, size_percent=0.15, per_channel=0.5)),

sometimes(iaa.Solarize(0.5, threshold=(32, 128))),

sometimes(iaa.Cartoon()),

sometimes(iaa.MotionBlur(k=15)),

sometimes(iaa.AllChannelsCLAHE()),

sometimes(iaa.Emboss(alpha=(0.0, 1.0), strength=(0.5, 1.5))),

sometimes(iaa.ElasticTransformation(alpha=(0, 5.0), sigma=0.25)),

sometimes(iaa.PiecewiseAffine(scale=(0.01, 0.05))),

sometimes(iaa.imgcorruptlike.Snow(severity=2)),

sometimes(iaa.Superpixels(p_replace=0.3, n_segments=500)),

sometimes(iaa.Rain(speed=(0.1, 0.3))),

sometimes(iaa.Snowflakes(flake_size=(0.1, 0.4), speed=(0.01, 0.05))),

sometimes(iaa.Fog()),

],

random_order=True

)

for i, v in enumerate(file_list):

img = cv2.imread(v)

images_aug = seq(images=[img])[0] # done by the library

cv2.imwrite(v, images_aug)

print('{}/{} aug : {}'.format(i, len(file_list), v))

Fix indention in VS code

 

• On Windows Shift + Alt + F
• On Mac Shift + Option + F
• On Linux Ctrl + Shift + I

RuntimeError: set_sizes_contiguous is not allowed on Tensor created from .data or .detach(), in Pytorch 1.1.0

change old -> new 

old

v.data.resize_(data.size()).copy_(data)

NEW

with torch.no_grad():

    v.resize_(data.size()).copy_(data)

How to fix Python SSL CERTIFICATE_VERIFY_FAILED

 put this code on the top of code line:

import os, ssl

if (not os.environ.get('PYTHONHTTPSVERIFY', '') and

getattr(ssl, '_create_unverified_context', None)):

ssl._create_default_https_context = ssl._create_unverified_context

Get list from dir and separate train and test (python function)

from sklearn.model_selection import train_test_split

import random

import os

import glob

def train_test_split_from_dir(origin_dir, test_size=0.2):

os.chdir(origin_dir)

#get list

data_list = []

for file in glob.glob("*.jpg"):

data_list.append(file)

random.shuffle(data_list)

train_json_list, test_json_list = train_test_split(data_list, test_size=test_size)

return train_json_list, test_json_list

ROC & AUC example code in face detector model case

..

#https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
import numpy as np
from sklearn import metrics
import matplotlib.pyplot as plt

#model #1
y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0])
scores = np.array([0.64, 0.47, 0.46, 0.77, 0.72, 0.9, 0.85, 0.7, 0.87, 0.92, 0.89, 0.93, 0.85, 0.81, 0.88, 0.48, 0.1, 0.35, 0.68, 0.47])
fpr, tpr, thresholds = metrics.roc_curve(y, scores)
roc_auc = metrics.auc(fpr, tpr)

# plot
plt.title('Receiver Operating Characteristic')
plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()

..

Example model metrics using sklearn in face detector case

..

from sklearn.metrics import classification_report
#model 1
y_true = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
y_pred = [0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
target_names = ['Non Face', 'Face']
print(classification_report(y_true, y_pred, target_names=target_names, digits=3))

..

..

#model 2
y_true = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
y_pred = [0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
target_names = ['Non Face', 'Face']
print(classification_report(y_true, y_pred, target_names=target_names, digits=3))

..

extract year, month, day from file on Ubuntu, python example

...

import os, time

date_created_obj = time.localtime(os.path.getctime(full_path))

print('Year: {:4d}'.format(date_created_obj.tm_year)) # Year: 2020

print('Month: {:2d}'.format(date_created_obj.tm_mon)) # Month: 2

print('Day: {:2d}'.format(date_created_obj.tm_mday)) # Day: 10

...

how to merge two csr_matrix, example python source code

let's see the code.

..

from scipy.sparse import csr_matrix

import numpy as np

#first matrix

row = np.array([0, 0, 1, 2, 2, 2])

col = np.array([0, 2, 2, 0, 1, 2])

data = np.array([1, 1, 1, 1, 1, 1])

mtx = csr_matrix((data, (row, col)), shape=(3, 3))

#second matrix

row = np.array([0, 0, 1, 2, 2, 2])

col = np.array([0, 1, 2, 0, 1, 2])

data = np.array([1, 1, 1, 1, 1, 1])

mtx2 = csr_matrix((data, (row, col)), shape=(3, 3))

#merge two matrix

mtx3 = merge_two_csr_mtx(mtx, mtx2)

#check

print('1st\n',mtx)

print('2nd\n',mtx2)

print('merge\n',mtx3)

..

result

1st
   (0, 0) 1
  (0, 2) 1
  (1, 2) 1
  (2, 0) 1
  (2, 1) 1
  (2, 2) 1
2nd
   (0, 0) 1
  (0, 1) 1
  (1, 2) 1
  (2, 0) 1
  (2, 1) 1
  (2, 2) 1
merge
   (0, 0) 2.0
  (0, 1) 1.0
  (0, 2) 1.0
  (1, 2) 2.0
  (2, 0) 2.0
  (2, 1) 2.0
  (2, 2) 2.0

How to convert a scipy csr_matrix back into lists of row, col and data?

refer to code


..
Define matrix & check values

from scipy.sparse import csr_matrix

import numpy as np

row = np.array([0, 0, 1, 2, 2, 2])

col = np.array([0, 1, 2, 0, 1, 2])

data = np.array([1, 1, 1, 1, 1, 1])

mtx2 = csr_matrix((data, (row, col)), shape=(3, 3))

print(mtx2) #matrix print out

print(mtx2.toarray()) #print out by array

>

(0, 0) 1
  (0, 1) 1
  (1, 2) 1
  (2, 0) 1
  (2, 1) 1
  (2, 2) 1

>

[[1 1 0]
 [0 0 1]
 [1 1 1]]

..


...
get back the row, col and data value from matrix

c = mtx2.tocoo()

print(c.row)

print(c.col)

print(c.data)

>

[0 0 1 2 2 2]
[0 1 2 0 1 2]
[1 1 1 1 1 1]

...

sentence embedding, sentence to vector using bert

refer to source code

.

#pip install -U sentence-transformers

#https://github.com/UKPLab/sentence-transformers

from sentence_transformers import SentenceTransformer, LoggingHandler

# Load Sentence model (based on BERT) from URL

model = SentenceTransformer('bert-base-nli-mean-tokens')

# Embed a list of sentences

sentences = ['This framework generates embeddings for each input sentence',

'Sentences are passed as a list of string.',

'The quick brown fox jumps over the lazy dog.']

sentence_embeddings = model.encode(sentences)

# The result is a list of sentence embeddings as numpy arrays

for sentence, embedding in zip(sentences, sentence_embeddings):

print("Sentence:", sentence)

print("Embedding:", embedding.shape, type(embedding))

print("")

.

result is like this:

Sentence: This framework generates embeddings for each input sentence

Embedding: (768,) <class 'numpy.ndarray'>

Sentence: Sentences are passed as a list of string.

Embedding: (768,) <class 'numpy.ndarray'>

Sentence: The quick brown fox jumps over the lazy dog.

Embedding: (768,) <class 'numpy.ndarray'>

install poppler in ubuntu

Try to this command:

sudo apt-get update -y

sudo apt-get install -y poppler-utils

😁

Ways to sort list of dictionaries by values in Python – Using lambda function

.

#example list

dict_list = [{ "idx":1, "value1":32.44, "value2":123.2}, { "idx":2, "value1":32.414, "value2":133.2}, { "idx":3, "value1":32.244, "value2":113.2}]

#sort by ascending order

sorted_dict_list = sorted(dict_list, key = lambda i: i['value1'])

#sort by descending order

r_sorted_dict_list = sorted(dict_list, key = lambda i: i['value1'],reverse=True)

#show result

print(sorted_dict_list)

# [{'idx': 3, 'value1': 32.244, 'value2': 113.2}, {'idx': 2, 'value1': 32.414, 'value2': 133.2}, {'idx': 1, 'value1': 32.44, 'value2': 123.2}]

print(r_sorted_dict_list)

# [{'idx': 1, 'value1': 32.44, 'value2': 123.2}, {'idx': 2, 'value1': 32.414, 'value2': 133.2}, {'idx': 3, 'value1': 32.244, 'value2': 113.2}]

.

multi-thread example python source code

The code generate 10 multi threads for running single_function.
If you have look the pid in result, thread is finished by quickly proceeded.

..

import queue

from concurrent.futures import ThreadPoolExecutor

#function for thread

def single_function(input, pid, out_queue):

total = 0

for i in range(0,input):

for j in range(0, input):

for k in range(0, input):

total = total + 1

out_queue.put( {'index':pid, 'result':total })

#run thread

my_queue = queue.Queue()

with ThreadPoolExecutor(max_workers=10) as executor:

for pid in range(0, 10):

executor.submit(single_function, 100, pid, my_queue)

#get result of each thread

result = {}

while not my_queue.empty():

get = my_queue.get()

print(get)

#finish all thread

..

result

{'index': 1, 'result': 1000000}
{'index': 3, 'result': 1000000}
{'index': 2, 'result': 1000000}
{'index': 0, 'result': 1000000}
{'index': 5, 'result': 1000000}
{'index': 4, 'result': 1000000}
{'index': 8, 'result': 1000000}
{'index': 6, 'result': 1000000}
{'index': 9, 'result': 1000000}
{'index': 7, 'result': 1000000}