3/27/2020

Python : How to copy files from one location to another using shutil.copy()

Just check the example code

..

import shutil
# Copy file to another directory
newPath = shutil.copy('sample1.txt', '/home/varung/test')
print("Path of copied file : ", newPath)
#Path of copied file :  /home/varung/test/sample1.txt

#Copy a file with new name
newPath = shutil.copy('sample1.txt', '/home/varung/test/sample2.txt')
print("Path of copied file : ", newPath)
#Path of copied file :  /home/varung/test/sample2.txt

# Copy a symbolic link as a new link
newPath = shutil.copy('/home/varung/test/link.csv', '/home/varung/test/sample2.csv')
print("Path of copied file : ", newPath)
#Path of copied file :  /home/varung/test/sample2.csv

# Copy target file pointed by symbolic link
newPath = shutil.copy('/home/varung/test/link.csv', '/home/varung/test/newlink.csv', follow_symlinks=False)
print("Path of copied file : ", newPath)
#Path of copied file :  /home/varung/test/newlink.csv
..

How to delete a file or folder? (python code)

Use os or shut library
os.remove() removes a file.
os.rmdir() removes an empty directory.
shutil.rmtree() deletes a directory and all its contents.


EX)
import os
os.remove("/tmp/<file_name>.txt")
os.rmdir("/tmp/")

import shutil
shutil.rmtree("/tmp/")


ex2)
if os.path.isdir('./runs/ex'):
    shutil.rmtree('./runs/ex')
os.mkdir('./runs/ex')

3/23/2020

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
...


simple solutin :
ssh-keygen -R <host>

Ex)
ssh-keygen -R 192.168.3.10
ssh-keygen -R ubuntu@ec2-30.30.30.eu-west-1.compute.amazonaws.com





3/22/2020

use docker without sudo

1.Create the docker group.
> sudo groupadd docker

2.Add your user to the docker group.
> sudo usermod -aG docker $USER

3.Log out and log back in so that your group membership is re-evaluated.
> newgrp docker 

4.Verify that you can run docker commands without sudo.
>docker run hello-world

3/21/2020

transfer local docker image to another machine


Transfer docker image A -> B
1. Make zip file on A 
docker save -o ./example.tar image_name

2. Transfer example.tar  A to B using scp, rsync, or in some way

3. Load image on B
docker load -i ./example.tar

install docker on ubuntu 18.04

Step 1: Update Software Repositories
sudo apt-get update

Step 2: Uninstall Old Versions of Docker
sudo apt-get remove docker docker-engine docker.io

Step 3: Install Docker
sudo apt install docker.io

Step 4: Start and Automate Docker
sudo systemctl start docker
sudo systemctl enable docker

Step 5 (Optional): Check Docker Version
docker --version

3/17/2020

get unique value from list (python source code)

..
import numpy as np

def unique(list1):
x = np.array(list1)
x = np.unique(x)
return list(x)

list1 = [10, 20, 10, 30, 40, 40]
list1 = unique(list1)
print(list1)
..
output
[10, 20, 30, 40]
..

3/11/2020

shuffle two related list python

method #1
..
a=['aaaaa','bbbb','cccc','dddd']
b=[1, 2, 3, 4]

from sklearn.utils import shuffle
list_1, list_2 = shuffle(a, b)
print(list_1, list_2)
#['cccc', 'aaaaa', 'dddd', 'bbbb'] [3, 1, 4, 2]
..

method #2
...
list1_shuf = []
list2_shuf = []
index_shuf = list(range(len(a)))
shuffle(index_shuf)
for i in index_shuf:
list1_shuf.append(a[i])
list2_shuf.append(b[i])

print(list1_shuf)
print(list2_shuf)
#['cccc', 'aaaaa', 'bbbb', 'dddd']
#[3, 1, 2, 4]
...



3/02/2020

Using Virtual Environments in Jupyter Notebook and Python

activate user virtualenv
>source yourenv/bin/activate

install ipykernel
>(yourenv) pip install --user ipykernel


Add you virtualenv to jupyter
>(yourenv) python -m ipykernel install --user --name=myenv

Then you can choose myenv to create new file.



key commands:
pip install --user ipykernel
python -m ipykernel install --user --name=myenv

3/01/2020

denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

error:
denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

this is worked for me
>eval $( aws ecr get-login --no-include-email --region eu-west-1 )