Making virtualenv using mkvirtualenv or virtualenv, simple command memo.

- create virtual env wrapper & deactivate

mkvirtualenv env-py2 -p python2 #(or python3)
workon env-py2
deactivate

- create virtual env & deactivate

sudo pip install virtualenv
virtualenv env
sourse env/bin/activate
source deactivate

- delete virtual env

rmvirtualenv envname

- virtualenv dependencies save and install

#On the first, save
pip freeze > requirements.txt

#On the second, install(note! after activating target env)
pip install -r requirements.txt

-show all virtualenv names

lsvirtualenv -l

- PyCharm virtualenv wrapper setting
We have to set directory path manually in preference setting page.
The path is like this in my case.

And then you can select this virtualenv on run/debug configuration.

opencv histogram stretching, example source code.

Histogram stretching
It's just adjusting the range with the same ratio.

For example, there is a range of numbers.
60, 61, 62, 63, 64, 65
Stretching is to extent other range, such as 0~255.
So, if we stitching 60~65 to 0~255, numbers will be like that.
60 -> 0
61 -> 51
62 -> 102
63 -> 153
64 -> 204
65 -> 255

so, in case of histogram, origin histogram will be stretched like that

image source : https://stackoverflow.com/questions/41118808/difference-between-contrast-stretching-and-histogram-equalization

then, let's look at code and result with real image.

origin input image


histogram


stretched image


histogram of stretched image


source code
<gist start>

<gist end>

Django + install rich html text editor (WYSIWYG, ckeditor)

I recommend you do this on virtual-env.

1. install packages

pip install django-ckeditor
pip install image (optional)

2. Set settings.py

Add ckeditor in INSTALLED_APPS

INSTALLED_APPS = [ 'django.contrib.admin',                   

                   'django.contrib.auth',                   

                    ... 
                   'ckeditor',]

Add this contents end of settings.py

CKEDITOR_UPLOAD_PATH = "uploads/" 

CKEDITOR_CONFIGS = { 
    'default': { 'toolbar': None, },

}

3. Set url.py

Add "url(r'^ckeditor/', include('ckeditor_uploader.urls'))," in urlpatterns

urlpatterns = [ url(r'^admin/', admin.site.urls),                

               ... 
                url(r'^ckeditor/', include('ckeditor_uploader.urls')),                

]

4. set models.py

Add import RichTextField
change "models.TextField()" to "RichTextField()"

from ckeditor.fields import RichTextField 
# Create your models here 

class OpenCV_Post(models.Model): 
    author = models.ForeignKey('auth.User') 
    title = models.CharField(max_length=200)
    # text = models.TextField()    

    text = RichTextField()

5. migration!

>>python manage.py makemigrations
>>python manage.py migrate

The Result on Admin






reference
https://www.youtube.com/watch?v=W8PTD7SszDI

Hue histogram example opencv source code

Hue histogram example source code.

input


Hue histogram output


source code
<gist start>

<gist end>

calcHist for RGB image, opencv histogram example

A Example source code for rgb histogram, the source code uses calcHist function in opencv.

Important things in the source code are the part of split rgb mat to vector[3] and drawing part.
Read the code carefully, so then you can understand easliy. ^^

input


the result of rgb histogram


<gist code start>

<gist code end>



reference : https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/histogram_calculation/histogram_calculation.html

calcHist for gray image, opencv Histogram example

This is example to use calcHist for grayimage.

input image


output


source code

end code

Gray image histogram without opencv function calHist

As you known, there is function for making histogram in Opencv, that is calcHist function.
But at this time, let's try get histogram without use calcHist.

Input image, we are going to convert from rgb to gray.


And this is result of histogram


So.. see the source code. I will be not difficult. ^^

tensorflow gpu install window error : self_check.py...ImportError: Could not find 'cudnn64_6.dll...

hmm......I have taken for about 5 hours for solve this problem....

error is like that:

Traceback (most recent call last):
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\platform\self_check.py", line 87, in preload_check
    ctypes.WinDLL(build_info.cudnn_dll_name)
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] 지정된 모듈을 찾을 수 없습니다
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 30, in <module>
    self_check.preload_check()
  File "C:\Users\mare\Anaconda3\envs\mare4\lib\site-packages\tensorflow\python\platform\self_check.py", line 97, in preload_check
    % (build_info.cudnn_dll_name, build_info.cudnn_version_number))
ImportError: Could not find 'cudnn64_6.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Note that installing cuDNN is a separate step from installing CUDA, and this DLL is often found in a different directory from the CUDA DLLs. You may install the necessary DLL by downloading cuDNN 6 from this URL: https://developer.nvidia.com/cudnn

In conclusion..
just use cudnn 6.0, that is Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0

I have tried many time
cuda 9.1 + cudnn 7.x
cuda 8.0 + cudnn 7.x
...
but I never successed

tensorflow official site recommend
cuda 8.0 + cudnn 6.1
https://www.tensorflow.org/install/install_windows
I just ignore this mention, because you know I thought the document is outdated.



This is simple tutorial for install tensorflow-gpu in window.

1. install anaconda.

conda create -n tensorflow python=3.5 

2. install cuda 8.0 and cudnn 6.x

move cudnn header, lib and dll to cuda 8.0 folder
http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installwindows

3. install tensorflow-gpu

pip install --ignore-installed --upgrade tensorflow-gpu 

4. check tensorflow-gpu

>>> import tensorflow as tf
>>> tf.test.is_built_with_cuda()
Ture

django + mysql setting (Mac or Linux)

A. mysql install

1. download mysql

https://dev.mysql.com/downloads/mysql/

* site may ask to join, but we don't need to join for download file

* install -> in case of Mac, show default password on popup window.

2. set path

$nano ~/.bash_profile 

add this sentence 

#mysql

export PATH=/usr/local/mysql/bin:$PATH

3. start mysql & set password

$ sudo /usr/local/mysql/support-files/mysql.server start

create a username with a password

$ mysqladmin -u root password yourpassword

or

change the password

$ mysqladmin -u root -p'oldpassword' password newpassword

4. create mysql database

login

$ mysql -u root -p
Enter password:

create database

CREATE DATABASE taskbuster_db;

B. django setting

1. install pymysql

$pip install pymysql

2. django settings.py

import pymysql
pymysql.install_as_MySQLdb()

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',        

'NAME': 'django_locker', # DB name        

'USER': '', # database user : root        

'PASSWORD': '', # database pass         

'HOST': '', # localhost        

'PORT': '', # database port('' or 3306)    }
}

3. migration

$python manage.py makemigrations

$python manage.py migrate

Anaconda simple command

version check

$ conda -V 
conda 3.7.0

update

conda update conda

create virtual env

conda create -n yourenvname python=x.x anaconda

activate

source activate yourenvname

install additional python packages

conda install -n yourenvname [package]

deactivate

source deactivate

remove virtual env

conda remove -n yourenvname --all

show all virtual env list

conda info --envs

or

conda env list

anaconda uninstall

conda install anaconda-clean   # install the package anaconda clean

anaconda-clean --yes           # clean all anaconda related files and directories

rm -rf ~/anaconda3             # removes the entire anaconda directory

#rm -rf /user/username/anaconda3 or rm -rf /Users/username/anaconda3

rm -rf ~/.anaconda_backup       # anaconda clean creates a back_up of files/dirs, remove it    

                            # (conda list; cmd shouldn't respond after the clean up)

minianaconda install

bash Miniconda3-latest-MacOSX-x86_64.sh

https://conda.io/docs/user-guide/install/macos.html#install-macos-silent

arrays - How to access a column(or row) in a list of lists in python

OK, we have like that list:

ListOfList = [ ['abcd',1], ['efgh',2], ['ijkl',3]]
print(ListOfList)

Let access list in list.

print('row 0', [row[0] for row in ListOfList] )
print('row 1', [row[1] for row in ListOfList] )

Very easy isn't it?
Enjoy~!

python element in list and element no in list, simple example

element in list

if 2 in [1, 2, 3]:
    print('2 in list')
else:
    print('2 in not list')


if 4 in [1, 2, 3]:
    print('4 in list')
else:
    print('4 in not list')


if (1,2) in [(1, 2) , (3, 2)]:
    print('(1,2) in list')
else:
    print('(1,2) in not list')

result



element no in list

if 4 not in [1, 2, 3]:
    print('4 not in list')
else:
    print('4 in list')


if (1,2) not in [(1, 2), (3, 2)]:
    print('(1,2) in not linst')
else:
    print('(1,2) in list')

result

TensorFlow Basic, print tf.Variable, ^^

All things start from the session

import tensorflow as tf

#define a variable to hold normal random values
normal_rv = tf.Variable( tf.truncated_normal([10,10],stddev = 0.1))

#initialize the variable
init_op = tf.initialize_all_variables()

#run the graph
with tf.Session() as sess:
    sess.run(init_op) #execute init_op
    #print the random values that we sample
    print (sess.run(normal_rv)

or

import tensorflow as tf

def create_weights(shape):

    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))

weights = create_weights(shape=[10,10])
init_op = tf.initialize_all_variables()

#run the graph
with tf.Session() as sess:
    sess.run(init_op) #execute init_op
    #print the random values that we sample

    print (sess.run(weights))

reference : https://stackoverflow.com/questions/33633370/how-to-print-the-value-of-a-tensor-object-in-tensorflow

Removing duplicates in lists using set, python

Simple example using set


>>> t = [1, 2, 3, 5, 6, 7, 8]
>>> t = t + [1]

>>> t = t + [2]

>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> t
[1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> list(set(t))
[1, 2, 3, 5, 6, 7, 8]
>>> s = [1, 2, 3]
>>> list(set(t) - set(s))
[8, 5, 6, 7]

reference : https://stackoverflow.com/questions/7961363/removing-duplicates-in-lists

Python: Split string with multiple delimiters [duplicate]

String split using delimiter

let's see how to split string using delimiter

There are 2 issues those are single delimiter and multi delimiters.

import re
#single delimiter
inputStr = 'This\nis\nmy\ndesk.'
split_inpuStr = inputStr.split('\n')
print('single',split_inpuStr)

#double delimiters
inputStr2 = 'This\r\nis\nmy\ndesk.'
split_inpuStr2 = re.split('\r\n|\n', inputStr2)
print('double',split_inpuStr2)

#multi delimiters
a = 'Beautiful,is; better*than\nugly'
b = re.split('; |, |\*|\n',a)
print('multi',b)

reference

https://stackoverflow.com/questions/4998629/python-split-string-with-multiple-delimiters
https://stackoverflow.com/questions/10393157/splitting-a-string-with-multiple-delimiters-in-python

python regular auto generator
http://ss2r.marearts.com

Django queryset simple tutorial

I have 4 objects in my table. The table name is 'Post'.



In models.py, the db class is like that:

class Post(models.Model):
    author = models.ForeignKey('auth.User')
    title = models.CharField(max_length=200)
    text = models.TextField()
    created_date = models.DateTimeField(
            default=timezone.now)
    published_date = models.DateTimeField(
            blank=True, null=True)

    def publish(self):
        self.published_date = timezone.now()
        self.save()

    def __str__(self):
        return self.title

So, let's train how to access the db and handle this.
Above all, you have to makemigrations, migrate after adding the db class code.

>python manage.py makemigrations
>python manage.py migrate

----
OK, train to the QuerySet (we call the dango query as QuerySet),
input this command in the console.
>python manage.py shell


OK, so now let's see the some commands
I will introduce very simple and useful commands.

1. show all objects

>from webapp.models import Post
: db import
>Post.objects.all()

2. access specific object

- first object
>Post.objects.all().first()
- last object
>Post.objects.all().last()
- 3rd object
>Post.objects.all()[2]

very easy right? 😴

3. element value change in object and save

> obj = Post.objects.all()[2]
>obj.title = 'a_title_3'
>obj.save()

4. object add and save

>>> from blog.models import Blog
>>> b = Blog(name='Beatles Blog', tagline='All the latest Beatles news.')
>>> b.save()

5. Checking for empty queryset in Django

if table_name.objects.all().count() == 0:
     print('none')

6. auto increase primary field

id = models.AutoField(primary_key=True)

7. Query datetime by today, yesterday's date 

visitor table class is like that:

class visitor_ss2r(models.Model):
    id = models.AutoField(primary_key=True)
    count = models.PositiveIntegerField(default=0)
    day_visiting = models.DateField(auto_now=True)

today search

(notice!) import datetime 

visitor.objects.get(day_visiting=datetime.date.today())
or
visitor.objects.filter(day_visiting=datetime.date.today()).count()

yesterday search

today = datetime.date.today()
yesterday = today - datetime.timedelta(days = 1)
visitor.objects.get(day_visiting=today)
or
visitor.objects.filter(day_visiting=yesterday).count()

8. object delete

outdated_day = today - datetime.timedelta(days = 30)
obs = visitor_ss2r.objects.filter(day_visiting=outdated_day)
if len(obs) > 0:
    obs.delete()

9. admin page DB field view

Add def __str__(self): .....  show below code!

class visitor_ss2r(models.Model):
    id = models.AutoField(primary_key=True)
    count = models.PositiveIntegerField(default=0)
    day_visiting = models.DateField(auto_now=True)

    def __str__(self):
        return 'Date: ' + str(self.day_visiting)

then you can see like that in admin page


more detail
in admin.py, we can make column list like that:

class visitor_ss2r_Admin(admin.ModelAdmin):
    list_display = ('id', 'count', 'day_visiting')
admin.site.register(visitor_ss2r, visitor_ss2r_Admin)

R. reference

Fastest way to get the first object from a queryset in django?https://stackoverflow.com/questions/5123839/fastest-way-to-get-the-first-object-from-a-queryset-in-django

Making queries¶ (Django officials page)
https://docs.djangoproject.com/en/2.0/topics/db/queries/