10/17/2021

install awscli for linux (python)


Try to do these command


>pip install awscli --upgrade --user

>sudo apt install awscli



Thank you.

www.marearts.com

10/13/2021

Get file list in s3 bucket folder (example python code)

 refer to source code

..

import boto3
import botocore

ACCESS_KEY = '**'
SECRET_KEY = '**'

s3_client = boto3.client(
's3',
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
region_name = 'eu-west-1'
)

bucket_name = '***'
folder_name = '***'

#check folder exist
try:
result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=folder_name )
for o in result['Contents']:
print(o['Key']) #file list in folder
except botocore.exceptions.ClientError as e:
print('no folder exist')

..


Thank you 

www.marearts.com



10/12/2021

aws dynamodb scan all item using paginate

 


Refer to code, Thank you!

..


import boto3

ACCESS_KEY = '***'
SECRET_KEY = '***'
dynamo_client = boto3.client(
'dynamodb',
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
region_name = 'eu-west-1'
)


def Search_Item(TABLE_NAME):
paginator = dynamo_client.get_paginator('scan')
operation_parameters = {'TableName': TABLE_NAME,}
page_iterator = paginator.paginate(**operation_parameters)
return page_iterator


TABLE_NAME ='****'

page_iterator = Search_Item(TABLE_NAME)
for page in page_iterator:
for item in page['Items']:
print(item['UID']['S'])

..


www.marearts.com

🙇🏻‍♂️


10/03/2021

Docker container stop by image name

 Stop docker container by image name:

imagename='mydockerimage'
docker stop $(docker ps | awk '{split($2,image,":"); print $1, image[1]}' | awk -v image=$imagename '$2 == image {print $1}')

Stop docker container by image name and tag:

imagename='mydockerimage:latest'
docker stop $(docker ps | awk -v image=$imagename '$2 == image {print $1}')

9/27/2021

docker swarm simple command

*install virtualbox

-+-

#install virtualbox on terminal
#osx
brew install virtualbox
#linux
sudo apt install virtualbox

-+-

 

* install docker-machine

...

#osx
curl -L https://github.com/docker/machine/releases/download/v0.16.2/docker-machine-`uname -s`-`uname -m` >/usr/local/bin/docker-machine && chmod +x /usr/local/bin/docker-machine
#linux
curl -L https://github.com/docker/machine/releases/download/v0.16.2/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine && chmod +x /tmp/docker-machine && sudo cp /tmp/docker-machine /usr/local/bin/docker-machine

...


*docker-machine

...

#docker-machine basic cmd
docker-machine ls
docker-machine rm name
docker-machine kill name
docker-machine create name

...



9/24/2021

python save & load argparse

 


refer to code:

..

from argparse import ArgumentParser
import json

parser = ArgumentParser()
parser.add_argument('--seed', type=int, default=8)
parser.add_argument('--resume', type=str, default='a/b/c.ckpt')
parser.add_argument('--surgery', type=str, default='190', choices=['190', '417'])
args = parser.parse_args()

with open('commandline_args.txt', 'w') as f:
json.dump(args.__dict__, f, indent=2)

parser = ArgumentParser()
args = parser.parse_args()
with open('commandline_args.txt', 'r') as f:
args.__dict__ = json.load(f)

print(args)

..


Thank you!

🙇🏻‍♂️

1D sequence (signal) data resampling (python)

 import numpy as np

import matplotlib.pyplot as plt
from scipy.signal import resample
from scipy.interpolate import interp1d

def ResampleLinear1D(original, targetLen):
original = np.array(original, dtype=np.float)
index_arr = np.linspace(0, len(original)-1, num=targetLen, dtype=np.float)
index_floor = np.array(index_arr, dtype=np.int) #Round down
index_ceil = index_floor + 1
index_rem = index_arr - index_floor #Remain

val1 = original[index_floor]
val2 = original[index_ceil % len(original)]
interp = val1 * (1.0-index_rem) + val2 * index_rem
assert(len(interp) == targetLen)
return interp

if __name__=="__main__":

original = np.sin(np.arange(256)/10.0)
targetLen = 100

# Method 1: Use scipy interp1d (linear interpolation)
# This is the simplest conceptually as it just uses linear interpolation. Scipy
# also offers a range of other interpolation methods.
f = interp1d(np.arange(256), original, 'linear')
plt.plot(np.apply_along_axis(f, 0, np.linspace(0, 255, num=targetLen)))

# Method 2: Use numpy to do linear interpolation
# If you don't have scipy, you can do it in numpy with the above function
plt.plot(ResampleLinear1D(original, targetLen))

# Method 3: Use scipy's resample
# Converts the signal to frequency space (Fourier method), then back. This
# works efficiently on periodic functions but poorly on non-periodic functions.
plt.plot(resample(original, targetLen))

plt.show()