4/22/2019

OpenCV Simple Background Subtraction Example code


Step #1 
Simple background subtraction




code here

.



Step #2
remove noise + binary

code here

.

Step #3 & #4
Draw contour

Remove small area and draw rect



core here:

.



Thank you
☕️

3/19/2019

Python hstack, vstack example code

import numpy as np

#hstack #1
a = np.array((1,2,3))
b = np.array((2,3,4))
print(np.hstack((a,b)))
>
 [1 2 3 2 3 4]

#hstack #2
a = np.array([[1],[2],[3]])
b = np.array([[2],[3],[4]])
print(np.hstack((a,b)))
>
 [[1 2]
 [2 3]
 [3 4]]

#vstack #1
a = np.array([1, 2, 3])
b = np.array([2, 3, 4])
print(np.vstack((a,b)))
>
 [[1 2 3]
 [2 3 4]]


#vstack #2
a = np.array([[1], [2], [3]])
b = np.array([[2], [3], [4]])
print(np.vstack((a,b)))
>
[[1]
 [2]
 [3]
 [2]
 [3]
 [4]]

3/07/2019

python number list, remove duplication and sort

coordiX = [0, 5016, 40, 5012, 40, 5012, 40, 5012, 40, 5012, 3169, 4970, 3169, 4970, 3169, 4970, 3169, 3537, 3586, 4355, 4395, 4970, 2632, 4616, 2632, 4616, 2632, 4616, 2632, 4616, 3651, 3659, 3651, 3659, 3651, 3659, 3651, 3659, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2632, 4616, 2632, 4616, 2632, 4616, 2632, 4616, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2632, 4614, 2632, 4614, 2632, 4614, 2632, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 2640, 2630, 2640, 2630, 2640, 2630, 2640, 3652, 3660, 3652, 3660, 3652, 3660, 3652, 3660, 328, 4670, 328, 4670, 328, 4670, 328, 4670, 328, 4668, 328, 4668, 328, 4668, 328, 4668, 330, 4614, 330, 4614, 330, 2962, 330, 784, 808, 1206, 2694, 2962, 2692, 3868, 2692, 3404, 3428, 3868, 332, 866]

print("origin")
print(coordiX)
print("length:", len(coordiX))

print("remove duplication")
coordiX = list(set(coordiX))
print(coordiX)
print("length:", len(coordiX))

print("sort")
coordiX.sort()
print(coordiX)

output
origin
[0, 5016, 40, 5012, 40, 5012, 40, 5012, 40, 5012, 3169, 4970, 3169, 4970, 3169, 4970, 3169, 3537, 3586, 4355, 4395, 4970, 2632, 4616, 2632, 4616, 2632, 4616, 2632, 4616, 3651, 3659, 3651, 3659, 3651, 3659, 3651, 3659, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2632, 4616, 2632, 4616, 2632, 4616, 2632, 4616, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2632, 4614, 2632, 4614, 2632, 4614, 2632, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 4614, 2630, 2640, 2630, 2640, 2630, 2640, 2630, 2640, 3652, 3660, 3652, 3660, 3652, 3660, 3652, 3660, 328, 4670, 328, 4670, 328, 4670, 328, 4670, 328, 4668, 328, 4668, 328, 4668, 328, 4668, 330, 4614, 330, 4614, 330, 2962, 330, 784, 808, 1206, 2694, 2962, 2692, 3868, 2692, 3404, 3428, 3868, 332, 866]
length: 130

remove duplication
[0, 3586, 4355, 2692, 4614, 2694, 4616, 784, 2962, 5012, 5016, 3868, 40, 808, 4395, 1206, 4668, 4670, 3651, 3652, 2630, 2632, 328, 330, 3659, 3660, 3404, 332, 2640, 3537, 3169, 866, 3428, 4970]
length: 34

sort
[0, 40, 328, 330, 332, 784, 808, 866, 1206, 2630, 2632, 2640, 2692, 2694, 2962, 3169, 3404, 3428, 3537, 3586, 3651, 3652, 3659, 3660, 3868, 4355, 4395, 4614, 4616, 4668, 4670, 4970, 5012, 5016]

python 2d array and rows and cols

#make array
W = 2
H = 3


list2d = []
for i in range(0,H):
w_list =[]
for j in range (0,W):
w_list.append((i,j))
list2d.append(w_list)


#print 2d array
print(list2d)


#get row and col
Height = Rows = len(list2d)
Width = Cols = len(list2d[0])


#check values
print( Rows, Cols)
print( Height, Width)

#print all elements
for i in range(0,Rows):
for j in range (0,Cols):
print(list2d[i][j])


output
[[(0, 0), (0, 1)], [(1, 0), (1, 1)], [(2, 0), (2, 1)]]
3 2
3 2
(0, 0)
(0, 1)
(1, 0)
(1, 1)
(2, 0)
(2, 1)

3/06/2019

python dic to json, json to txt file (example source code)

This article is about how to convert dic type to json.

source code is like that:
Dic -> Json -> txt file 1
txt file 1 -> Json -> Dic -> Json -> txt file 2

So consequentially, txt file1 and txt file 2 would be same.

Then check source code.

#dictionary type
dic_type = {'dic_type': 'yes', 'json': 10}

#dic to json
import json
str_type = json.dumps(dic_type) #dic to json

#write json to file
f= open("./json.txt","w+")
f.write(str_type)
f.close

#dic from json
dic_type_from_json = json.loads(str_type)

#dic from file
f = open("./json.txt","r")
str_type_from_file = f.read()
f.close
dic_type_from_file = json.loads(str_type_from_file)

#check data type
print( type(str_type) )#Output str
print( type(dic_type_from_json) )#Output dict
print( type(dic_type_from_file) )#Output dict


#write json to file
f= open("./json2.txt","w+")
r = json.dumps(dic_type_from_file)
f.write(r)
f.close

#json.txt and json2.txt is same

2/23/2019

64base image to pil image and upload s3 bucket on aws lambda

This article is about how to convert base64 string image to byte and invoke to PIL image.
Then we can handle image whatever we want to ex) image resize ..

base64 image come from img_base64 = event['base64Image']
and then convert to byte data imgdata = base64.b64decode(img_base64)

then it can save to image file and load to Image.open(filename)
or
invoke to pil image directly : img2 = Image.open(io.BytesIO(imgdata))

In the source code, there is also example how to upload image to s3 bucket.
refer to below code.
//
import sys
sys.path.append("/opt")

import json
import boto3
import os
import io
from PIL import Image
import base64



ACCESS_KEY = os.environ.get('ACCESS_KEY')
SECRET_KEY = os.environ.get('SECRET_KEY')

def uploadToS3(bucket, s3_path, local_path):
    s3_client = boto3.client(
        's3',
        aws_access_key_id=ACCESS_KEY,
        aws_secret_access_key=SECRET_KEY,
    )
    s3_client.upload_file(local_path, bucket, s3_path)
    
    
    
def lambda_handler(event, context):
    
    #base64 image string
    img_base64 = event['base64Image']
    #string to byte
    imgdata = base64.b64decode(img_base64)
    
    #save some image file
    with open("/tmp/imageToSave.png", "wb") as fh:
        fh.write(imgdata)
    
    #open("/tmp/imageToSave.png",'rb')
    uploadToS3("input-image1", "imageToSave.png", "/tmp/imageToSave.png")
    
    #load image file to pil
    img = Image.open("/tmp/imageToSave.png")
    width, height = img.size
    
    #load 
    img2 = Image.open(io.BytesIO(imgdata))
    width2, height2 = img2.size
    
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello leon it is from Lambda!'),
        'width file': width,
        'heihgt file' : height,
        'width pil': width2,
        'heihgt pil' : height2
    }

//

These method is referenced by:
https://stackoverflow.com/questions/16214190/how-to-convert-base64-string-to-image
https://stackoverflow.com/questions/11727598/pil-image-open-working-for-some-images-but-not-others
https://stackoverflow.com/questions/16214190/how-to-convert-base64-string-to-image
https://stackoverflow.com/questions/6444548/how-do-i-get-the-picture-size-with-pil

Thanks for effort.




2/12/2019

python zip test sample code


numberList = [1, 2, 3, 4, 5]
strList = ['one', 'two', 'three', 'five']

# No iterables are passed
result = zip()
print('first zip: {}'.format(result))

# Two iterables are passed
result = zip(numberList, strList)
print('input value to zip: {}'.format(result))

# Converting itertor to list
print('zip to list: {}'.format(list(result)))

# Converting itertor to set
resultSet = set(result)
print('zip to set: {}'.format(resultSet))



result


2/05/2019

How to install Node.js and npm on Amazon Linux (CentOS 7) for lambda


npm & node version check
> node --version
> npm --version


install or upgrade npm & node
Change setup_8.x, in case you want to install version 8.x
>curl -sL https://rpm.nodesource.com/setup_10.x | bash -
>yum install nodejs


If you already installed old version npm & node, remove first.
>yum remove -y nodejs npm


I have installed node as 8.x.

Enjoy!


2/04/2019

Install python 3.6 on AmazonLinux and make Docker image


Run Amazon Linux Container (and tunnelling)
> docker run -v $(pwd):/outputs --name lambdapack -d amazonlinux:latest tail -f /dev/null

Run container bash shell
> docker exec -i -t lambdapack /bin/bash
Install python 3.6
bash# yum -y update
bash# yum -y upgrade
bash# yum install -y \
 wget \
 gcc \
 gcc-c++ \
 findutils \
 zlib-devel \
 zip

bash# yum install -y yum-utils
bash# yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
bash# yum install -y python36.x86_64
bash# yum install -y python36-devel.x86_64
bash# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
bash# python36 get-pip.py
bash# pip3 install virtualenv
bash# exit

Make docker Image
> docker commit lambdapack marearts/amazon_linux_py36:v1.0.0

Push Docker Image to Docker Hub
>docker login
..
>docker push marearts/amazon_linux_py36:v1.0.0


Done!
Take Care!




1/29/2019

How to install wget in macOS?

install by brew
>ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
>brew install wget --with-libressl

install by port
> sudo port install wget


Enjoy!





1/28/2019

tar (child): xz: Cannot exec: No such file or directory on Amazon Linux

yum install -y xz

docker command summarise


stop all containers:
docker kill $(docker ps -q)

remove all containers
docker rm $(docker ps -a -q)

remove all docker images
docker rmi $(docker images -q)

access(enter) docker container
docker exec -it docker_container_name sh

Exit from docker container
>exit

Commit container and build image
docker commit container_name image_name
ex) docker commit lambdapack lambda_image
ex) docker commit lambdapack marearts/lambda_image:v1.0.0

Docker build
>docker build --tag hello:0.1 .
>docker build --tag marearts/hello:0.1 .

Docker run shell and tunnelling 
>docker run -v $(pwd):/outputs --name nickname -d docker_image tail -f /dev/null
ex)
>docker run -v $(pwd):/outputs --name lambdapackgen2 -d marearts/awspy:0.1 tail -f /dev/null

Docker run shell

> docker run -it marearts/amazon_linux_py36:v1.0.0 /bin/bash

Docker run sh file in container
>docker exec -i -t container_name /bin/bash /outputs/shfile.sh
ex)
>docker exec -i -t lambdapackgen2 /bin/bash /outputs/buildPack_py3.sh

Restart Docker container
>docker restart <container id or name>
ex)
docker restart 59



it will be updated..


1/27/2019

summarise Azure function docker image deploy



*make python virtualenv
*activate python virtualenv !!
*install azure util cli
https://docs.microsoft.com/en-us/azure/azure-functions/functions-create-function-linux-custom-image#run-the-build-command

make project (app)
func init DocApp100 --docker

change directory
cd DocApp100

new function
func new --name MyHttpTrigger --template "HttpTrigger"

build docker image
docker build --tag marearts/image:v1 .

make new resource group
az group create --name DocGroup --location westeurope

storage account
az storage account create --name docstorage100 --location westeurope --resource-group DocGroup --sku Standard_LRS

create linux service plan
az appservice plan create --name docserviceplan --resource-group DocGroup --sku B1 --is-linux

create app and deploy
az functionapp create --name docapp100 --storage-account docstorage100 --resource-group DocGroup --plan docserviceplan --deployment-container-image-name marearts/image:v1

Configuration the function app
storageConnectionString=$(az storage account show-connection-string --resource-group DocGroup --name docstorage100 --query connectionString --output tsv)
az functionapp config appsettings set --name docapp100 --resource-group DocGroup --settings AzureWebJobsDashboard=$storageConnectionString AzureWebJobsStorage=$storageConnectionString

Test
curl https://docapp100.azurewebsites.net/api/MyHttpTrigger?name=myname


Azure function create command summarise.

reference:
https://docs.microsoft.com/en-us/azure/azure-functions/functions-run-local#v2
https://docs.microsoft.com/en-us/azure/azure-functions/functions-create-first-function-python


create project
func init PythonAzure

create azure function
func new

Test
func host test

create resource group
az group create --name mareartstest --location westeurope

create strage account
az storage account create --name mareartsteststorage --location westeurope --resource-group mareartstest --sku Standard_LRS

create app
az functionapp create --resource-group mareartstest --os-type Linux --consumption-plan-location westeurope --runtime python --name mareartstest --storage-account mareartsteststorage

deploy
func azure functionapp publish mareartstest

1/23/2019

python "argparse" simple usage


If you use "argparse" lib, you can get the argument parameters very easily in python.
Let's look below code.

test1.py
import argparse

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
#param add
ap.add_argument("-z", "--zip", type=int, required=True, help="input zip code")
#param add
ap.add_argument("-n", "--name", type=str, required=True, help="input your name")

#get param
args = ap.parse_args()
print(args.zip)
print(args.name)
print('---------')

#get param
args = vars(ap.parse_args())
print(args["zip"])
print(args["name"])
..

The code accepts 2 arguments those are 'zip' and 'name'
So how to input argument?

See the several example

>python test1.py -z 1234 -n kjh
1234
kjh
---------
1234
kjh

>python test1.py
usage: test1.py [-h] -z ZIP -n NAME
test1.py: error: the following arguments are required: -z/--zip, -n/--name


>python test1.py -h
usage: test1.py [-h] -z ZIP -n NAME
optional arguments:
  -h, --help            show this help message and exit
  -z ZIP, --zip ZIP     input zip code
  -n NAME, --name NAME  input your name

you can send argument param
-z or --zip for zip code
-n or --name for name string


Thank you.

reference : https://docs.python.org/3/library/argparse.html


1/10/2019

pipenv command summarising.

referenced from :

install
$ pip install pipenv
 brew install pipenv

make virtualenv
$ mkdir myenv
$ cd myenv
$ pipenv --python 3.6


install requirements pipenv
$ cd my_project
$ pipenv install

pipenv package install & uninstall
$ pipenv install beautifulsoup4
 pipenv uninstall beautifulsoup4

freeze → pip freeze > "requirements.txt"
$ pipenv lock

install package to development version
$ pipenv install --dev pytest


install development version requirements 
$ pipenv install --dev

activate pipenv virtualenv
$ pipenv shell

deactivate pipenv virtualenv
$ exit


run pipenv without activate
$ pipenv run which python
$ pipenv run python my_project.py


Python interpreter setting in Visual studio code