12/31/2017

Making virtualenv using mkvirtualenv or virtualenv, simple command memo.


- create virtual env wrapper & deactivate
mkvirtualenv env-py2 -p python2 #(or python3)
workon env-py2
deactivate

- create virtual env & deactivate
sudo pip install virtualenv
virtualenv env
sourse env/bin/activate
source deactivate

- delete virtual env
rmvirtualenv envname

- virtualenv dependencies save and install
#On the first, save
pip freeze > requirements.txt
#On the second, install(note! after activating target env)
pip install -r requirements.txt

-show all virtualenv names
lsvirtualenv -l

- PyCharm virtualenv wrapper setting
We have to set directory path manually in preference setting page.
The path is like this in my case.
And then you can select this virtualenv on run/debug configuration.



12/30/2017

opencv histogram stretching, example source code.

Histogram stretching
It's just adjusting the range with the same ratio.

For example, there is a range of numbers.
60, 61, 62, 63, 64, 65
Stretching is to extent other range, such as 0~255.
So, if we stitching 60~65 to 0~255, numbers will be like that.
60 -> 0
61 -> 51
62 -> 102
63 -> 153
64 -> 204
65 -> 255

so, in case of histogram, origin histogram will be stretched like that

image source : https://stackoverflow.com/questions/41118808/difference-between-contrast-stretching-and-histogram-equalization

then, let's look at code and result with real image.

origin input image


histogram


stretched image


histogram of stretched image


source code
<gist start>

<gist end>




Django + install rich html text editor (WYSIWYG, ckeditor)


I recommend you do this on virtual-env.

1. install packages

pip install django-ckeditor
pip install image (optional)

2. Set settings.py

Add ckeditor in INSTALLED_APPS
INSTALLED_APPS = [ 'django.contrib.admin',                   
                   'django.contrib.auth',                   
                    ... 
                   'ckeditor',]

Add this contents end of settings.py
CKEDITOR_UPLOAD_PATH = "uploads/" 
CKEDITOR_CONFIGS = { 
    'default': { 'toolbar': None, },
}

3. Set url.py

Add "url(r'^ckeditor/', include('ckeditor_uploader.urls'))," in urlpatterns
urlpatterns = [ url(r'^admin/', admin.site.urls),                
               ... 
                url(r'^ckeditor/', include('ckeditor_uploader.urls')),                
]

4. set models.py

Add import RichTextField
change "models.TextField()" to "RichTextField()"

from ckeditor.fields import RichTextField 
# Create your models here 
class OpenCV_Post(models.Model): 
    author = models.ForeignKey('auth.User') 
    title = models.CharField(max_length=200)
    # text = models.TextField()    
    text = RichTextField()

5. migration!

>>python manage.py makemigrations
>>python manage.py migrate


The Result on Admin






reference
https://www.youtube.com/watch?v=W8PTD7SszDI

12/28/2017

Hue histogram example opencv source code

Hue histogram example source code.

input


Hue histogram output


source code
<gist start>

<gist end>

calcHist for RGB image, opencv histogram example

A Example source code for rgb histogram, the source code uses calcHist function in opencv.

Important things in the source code are the part of split rgb mat to vector[3] and drawing part.
Read the code carefully, so then you can understand easliy. ^^

input


the result of rgb histogram


<gist code start>

<gist code end>



reference : https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/histogram_calculation/histogram_calculation.html

calcHist for gray image, opencv Histogram example

This is example to use calcHist for grayimage.

input image


output


source code

end code


Gray image histogram without opencv function calHist

As you known, there is function for making histogram in Opencv, that is calcHist function.
But at this time, let's try get histogram without use calcHist.

Input image, we are going to convert from rgb to gray.


And this is result of histogram


So.. see the source code. I will be not difficult. ^^



12/25/2017

tensorflow gpu install window error : self_check.py...ImportError: Could not find 'cudnn64_6.dll...


hmm......I have taken for about 5 hours for solve this problem....

error is like that:

Traceback (most recent call last):
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\platform\self_check.py", line 87, in preload_check
    ctypes.WinDLL(build_info.cudnn_dll_name)
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] ์ง€์ •๋œ ๋ชจ๋“ˆ์„ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 30, in <module>
    self_check.preload_check()
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\platform\self_check.py", line 97, in preload_check
    % (build_info.cudnn_dll_name, build_info.cudnn_version_number))
ImportError: Could not find 'cudnn64_6.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Note that installing cuDNN is a separate step from installing CUDA, and this DLL is often found in a different directory from the CUDA DLLs. You may install the necessary DLL by downloading cuDNN 6 from this URL: https://developer.nvidia.com/cudnn

In conclusion..
just use cudnn 6.0, that is Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0

I have tried many time
cuda 9.1 + cudnn 7.x
cuda 8.0 + cudnn 7.x
...
but I never successed

tensorflow official site recommend
cuda 8.0 + cudnn 6.1
https://www.tensorflow.org/install/install_windows
I just ignore this mention, because you know I thought the document is outdated.



This is simple tutorial for install tensorflow-gpu in window.

1. install anaconda.

conda create -n tensorflow python=3.5 

2. install cuda 8.0 and cudnn 6.x

move cudnn header, lib and dll to cuda 8.0 folder
http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installwindows

3. install tensorflow-gpu

pip install --ignore-installed --upgrade tensorflow-gpu 

4. check tensorflow-gpu

>>> import tensorflow as tf
>>> tf.test.is_built_with_cuda()
Ture






12/24/2017

django + mysql setting (Mac or Linux)

A. mysql install

1. download mysql

https://dev.mysql.com/downloads/mysql/



* site may ask to join, but we don't need to join for download file
* install -> in case of Mac, show default password on popup window.

2. set path

$nano ~/.bash_profile 
add this sentence 

#mysql
export PATH=/usr/local/mysql/bin:$PATH

3. start mysql & set password

$ sudo /usr/local/mysql/support-files/mysql.server start

create a username with a password
$ mysqladmin -u root&nbsp;password yourpassword
or
change the password
$ mysqladmin -u root -p'oldpassword' password newpassword

4. create mysql database

login
$ mysql -u root -p
Enter password:

create database
CREATE DATABASE taskbuster_db;



B. django setting

1. install pymysql

$pip install pymysql

2. django settings.py

import pymysql
pymysql.install_as_MySQLdb()

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',        
'NAME': 'django_locker', # DB name        
'USER': '', # database user : root        
'PASSWORD': '', # database pass         
'HOST': '', # localhost        
'PORT': '', # database port('' or 3306)    }
}


3. migration

$python manage.py makemigrations

$python manage.py migrate



Anaconda simple command


version check
$ conda -V 
conda 3.7.0

update
conda update conda

create virtual env
conda create -n yourenvname python=x.x anaconda

activate
source activate yourenvname

install additional python packages
conda install -n yourenvname [package]

deactivate
source deactivate

remove virtual env
conda remove -n yourenvname --all

show all virtual env list
conda info --envs
or
conda env list

anaconda uninstall
conda install anaconda-clean   # install the package anaconda clean
anaconda-clean --yes           # clean all anaconda related files and directories
rm -rf ~/anaconda3             # removes the entire anaconda directory
#rm -rf /user/username/anaconda3 or rm -rf /Users/username/anaconda3

rm -rf ~/.anaconda_backup       # anaconda clean creates a back_up of files/dirs, remove it    
                            # (conda list; cmd shouldn't respond after the clean up)

minianaconda install
bash Miniconda3-latest-MacOSX-x86_64.sh
https://conda.io/docs/user-guide/install/macos.html#install-macos-silent





12/14/2017

arrays - How to access a column(or row) in a list of lists in python

OK, we have like that list:

ListOfList = [ ['abcd',1], ['efgh',2], ['ijkl',3]]
print(ListOfList)



Let access list in list.
print('row 0', [row[0] for row in ListOfList] )
print('row 1', [row[1] for row in ListOfList] )




Very easy isn't it?
Enjoy~!


python element in list and element no in list, simple example


element in list

if 2 in [1, 2, 3]:
    print('2 in list')
else:
    print('2 in not list')


if 4 in [1, 2, 3]:
    print('4 in list')
else:
    print('4 in not list')


if (1,2) in [(1, 2) , (3, 2)]:
    print('(1,2) in list')
else:
    print('(1,2) in not list')

result



element no in list
if 4 not in [1, 2, 3]:
    print('4 not in list')
else:
    print('4 in list')


if (1,2) not in [(1, 2), (3, 2)]:
    print('(1,2) in not linst')
else:
    print('(1,2) in list')

result





12/13/2017

TensorFlow Basic, print tf.Variable, ^^

All things start from the session

import tensorflow as tf

#define a variable to hold normal random values
normal_rv = tf.Variable( tf.truncated_normal([
10,10],stddev = 0.1))

#initialize the variable
init_op = tf.initialize_all_variables()

#run the graph
with tf.Session() as sess:
    sess.run(init_op) #execute init_op
    #print the random values that we sample
   
print (sess.run(normal_rv)

or

import tensorflow as tf

def create_weights(shape):
    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))

weights = create_weights(shape=[10,10])
init_op = tf.initialize_all_variables()

#run the graph
with tf.Session() as sess:
    sess.run(init_op) #execute init_op
    #print the random values that we sample
    print (sess.run(weights))






12/12/2017

Removing duplicates in lists using set, python

Simple example using set


>>> t = [1, 2, 3, 5, 6, 7, 8]
>>> t = t + [1]
>>> t = t + [2]

>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> t
[1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> list(set(t))
[1, 2, 3, 5, 6, 7, 8]
>>> s = [1, 2, 3]
>>> list(set(t) - set(s))
[8, 5, 6, 7]

reference : https://stackoverflow.com/questions/7961363/removing-duplicates-in-lists

12/10/2017

Python: Split string with multiple delimiters [duplicate]


String split using delimiter

let's see how to split string using delimiter
There are 2 issues those are single delimiter and multi delimiters.

import re
#single delimiter
inputStr = 'This\nis\nmy\ndesk.'
split_inpuStr = inputStr.split('\n')
print('single',split_inpuStr)

#double delimiters
inputStr2 = 'This\r\nis\nmy\ndesk.'
split_inpuStr2 = re.split('\r\n|\n', inputStr2)
print('double',split_inpuStr2)

#multi delimiters
a = 'Beautiful,is; better*than\nugly'
b = re.split('; |, |\*|\n',a)
print('multi',b)


reference

https://stackoverflow.com/questions/4998629/python-split-string-with-multiple-delimiters
https://stackoverflow.com/questions/10393157/splitting-a-string-with-multiple-delimiters-in-python

python regular auto generator
http://ss2r.marearts.com

Django queryset simple tutorial


I have 4 objects in my table. The table name is 'Post'.



In models.py, the db class is like that:
class Post(models.Model):
    author = models.ForeignKey('auth.User')
    title = models.CharField(max_length=200)
    text = models.TextField()
    created_date = models.DateTimeField(
            default=timezone.now)
    published_date = models.DateTimeField(
            blank=True, null=True)

    def publish(self):
        self.published_date = timezone.now()
        self.save()

    def __str__(self):
        return self.title


So, let's train how to access the db and handle this.
Above all, you have to makemigrations, migrate after adding the db class code.

>python manage.py makemigrations
>python manage.py migrate


----
OK, train to the QuerySet (we call the dango query as QuerySet),
input this command in the console.
>python manage.py shell


OK, so now let's see the some commands
I will introduce very simple and useful commands.


1. show all objects

>from webapp.models import Post
: db import
>Post.objects.all()




2. access specific object

- first object
>Post.objects.all().first()
- last object
>Post.objects.all().last()
- 3rd object
>Post.objects.all()[2]




very easy right? ๐Ÿ˜ด


3. element value change in object and save

> obj = Post.objects.all()[2]
>obj.title = 'a_title_3'
>obj.save()






4. object add and save

>>> from blog.models import Blog
>>> b = Blog(name='Beatles Blog', tagline='All the latest Beatles news.')
>>> b.save()


5. Checking for empty queryset in Django

if table_name.objects.all().count() == 0:
     print('none')


6. auto increase primary field

id = models.AutoField(primary_key=True)


7. Query datetime by today, yesterday's date 

visitor table class is like that:
class visitor_ss2r(models.Model):
    id = models.AutoField(primary_key=True)
    count = models.PositiveIntegerField(default=0)
    day_visiting = models.DateField(auto_now=True)

today search
(notice!) import datetime 
visitor.objects.get(day_visiting=datetime.date.today())
or
visitor.objects.filter(day_visiting=datetime.date.today()).count()
yesterday search
today = datetime.date.today()
yesterday = today - datetime.timedelta(days = 1)
visitor.objects.get(day_visiting=today)
or
visitor.objects.filter(day_visiting=yesterday).count()

8. object delete

outdated_day = today - datetime.timedelta(days = 30)
obs = visitor_ss2r.objects.filter(day_visiting=outdated_day)
if len(obs) > 0:
    obs.delete()


9. admin page DB field view

Add def __str__(self): .....  show below code!

class visitor_ss2r(models.Model):
    id = models.AutoField(primary_key=True)
    count = models.PositiveIntegerField(default=0)
    day_visiting = models.DateField(auto_now=True)

    def __str__(self):
        return 'Date: ' + str(self.day_visiting)

then you can see like that in admin page


more detail
in admin.py, we can make column list like that:

class visitor_ss2r_Admin(admin.ModelAdmin):
    list_display = ('id', 'count', 'day_visiting')
admin.site.register(visitor_ss2r, visitor_ss2r_Admin)



R. reference

Fastest way to get the first object from a queryset in django?https://stackoverflow.com/questions/5123839/fastest-way-to-get-the-first-object-from-a-queryset-in-django

Making queries (Django officials page)
https://docs.djangoproject.com/en/2.0/topics/db/queries/