10/13/2018

python, get index and value in for loop

Use 'enumerate'

for index, item in enumerate(values):
print(index, item)

Python opencv, get ROI Mat (cropping)

Please refer to this function..

def getROImat(inMat, x1, y1, x2, y2):
# Crop image
imCrop = inMat[int(y1):int(y2), int(x1):int(x2)]
return imCrop


10/10/2018

pbtxt file for coco v2 data (90 categories contents)

item {
  name: "/m/01g317"
  id: 1
  display_name: "person"
}
item {
  name: "/m/0199g"
  id: 2
  display_name: "bicycle"
}
item {
  name: "/m/0k4j"
  id: 3
  display_name: "car"
}
item {
  name: "/m/04_sv"
  id: 4
  display_name: "motorcycle"
}
item {
  name: "/m/05czz6l"
  id: 5
  display_name: "airplane"
}
item {
  name: "/m/01bjv"
  id: 6
  display_name: "bus"
}
item {
  name: "/m/07jdr"
  id: 7
  display_name: "train"
}
item {
  name: "/m/07r04"
  id: 8
  display_name: "truck"
}
item {
  name: "/m/019jd"
  id: 9
  display_name: "boat"
}
item {
  name: "/m/015qff"
  id: 10
  display_name: "traffic light"
}
item {
  name: "/m/01pns0"
  id: 11
  display_name: "fire hydrant"
}
item {
  name: "/m/02pv19"
  id: 13
  display_name: "stop sign"
}
item {
  name: "/m/015qbp"
  id: 14
  display_name: "parking meter"
}
item {
  name: "/m/0cvnqh"
  id: 15
  display_name: "bench"
}
item {
  name: "/m/015p6"
  id: 16
  display_name: "bird"
}
item {
  name: "/m/01yrx"
  id: 17
  display_name: "cat"
}
item {
  name: "/m/0bt9lr"
  id: 18
  display_name: "dog"
}
item {
  name: "/m/03k3r"
  id: 19
  display_name: "horse"
}
item {
  name: "/m/07bgp"
  id: 20
  display_name: "sheep"
}
item {
  name: "/m/01xq0k1"
  id: 21
  display_name: "cow"
}
item {
  name: "/m/0bwd_0j"
  id: 22
  display_name: "elephant"
}
item {
  name: "/m/01dws"
  id: 23
  display_name: "bear"
}
item {
  name: "/m/0898b"
  id: 24
  display_name: "zebra"
}
item {
  name: "/m/03bk1"
  id: 25
  display_name: "giraffe"
}
item {
  name: "/m/01940j"
  id: 27
  display_name: "backpack"
}
item {
  name: "/m/0hnnb"
  id: 28
  display_name: "umbrella"
}
item {
  name: "/m/080hkjn"
  id: 31
  display_name: "handbag"
}
item {
  name: "/m/01rkbr"
  id: 32
  display_name: "tie"
}
item {
  name: "/m/01s55n"
  id: 33
  display_name: "suitcase"
}
item {
  name: "/m/02wmf"
  id: 34
  display_name: "frisbee"
}
item {
  name: "/m/071p9"
  id: 35
  display_name: "skis"
}
item {
  name: "/m/06__v"
  id: 36
  display_name: "snowboard"
}
item {
  name: "/m/018xm"
  id: 37
  display_name: "sports ball"
}
item {
  name: "/m/02zt3"
  id: 38
  display_name: "kite"
}
item {
  name: "/m/03g8mr"
  id: 39
  display_name: "baseball bat"
}
item {
  name: "/m/03grzl"
  id: 40
  display_name: "baseball glove"
}
item {
  name: "/m/06_fw"
  id: 41
  display_name: "skateboard"
}
item {
  name: "/m/019w40"
  id: 42
  display_name: "surfboard"
}
item {
  name: "/m/0dv9c"
  id: 43
  display_name: "tennis racket"
}
item {
  name: "/m/04dr76w"
  id: 44
  display_name: "bottle"
}
item {
  name: "/m/09tvcd"
  id: 46
  display_name: "wine glass"
}
item {
  name: "/m/08gqpm"
  id: 47
  display_name: "cup"
}
item {
  name: "/m/0dt3t"
  id: 48
  display_name: "fork"
}
item {
  name: "/m/04ctx"
  id: 49
  display_name: "knife"
}
item {
  name: "/m/0cmx8"
  id: 50
  display_name: "spoon"
}
item {
  name: "/m/04kkgm"
  id: 51
  display_name: "bowl"
}
item {
  name: "/m/09qck"
  id: 52
  display_name: "banana"
}
item {
  name: "/m/014j1m"
  id: 53
  display_name: "apple"
}
item {
  name: "/m/0l515"
  id: 54
  display_name: "sandwich"
}
item {
  name: "/m/0cyhj_"
  id: 55
  display_name: "orange"
}
item {
  name: "/m/0hkxq"
  id: 56
  display_name: "broccoli"
}
item {
  name: "/m/0fj52s"
  id: 57
  display_name: "carrot"
}
item {
  name: "/m/01b9xk"
  id: 58
  display_name: "hot dog"
}
item {
  name: "/m/0663v"
  id: 59
  display_name: "pizza"
}
item {
  name: "/m/0jy4k"
  id: 60
  display_name: "donut"
}
item {
  name: "/m/0fszt"
  id: 61
  display_name: "cake"
}
item {
  name: "/m/01mzpv"
  id: 62
  display_name: "chair"
}
item {
  name: "/m/02crq1"
  id: 63
  display_name: "couch"
}
item {
  name: "/m/03fp41"
  id: 64
  display_name: "potted plant"
}
item {
  name: "/m/03ssj5"
  id: 65
  display_name: "bed"
}
item {
  name: "/m/04bcr3"
  id: 67
  display_name: "dining table"
}
item {
  name: "/m/09g1w"
  id: 70
  display_name: "toilet"
}
item {
  name: "/m/07c52"
  id: 72
  display_name: "tv"
}
item {
  name: "/m/01c648"
  id: 73
  display_name: "laptop"
}
item {
  name: "/m/020lf"
  id: 74
  display_name: "mouse"
}
item {
  name: "/m/0qjjc"
  id: 75
  display_name: "remote"
}
item {
  name: "/m/01m2v"
  id: 76
  display_name: "keyboard"
}
item {
  name: "/m/050k8"
  id: 77
  display_name: "cell phone"
}
item {
  name: "/m/0fx9l"
  id: 78
  display_name: "microwave"
}
item {
  name: "/m/029bxz"
  id: 79
  display_name: "oven"
}
item {
  name: "/m/01k6s3"
  id: 80
  display_name: "toaster"
}
item {
  name: "/m/0130jx"
  id: 81
  display_name: "sink"
}
item {
  name: "/m/040b_t"
  id: 82
  display_name: "refrigerator"
}
item {
  name: "/m/0bt_c3"
  id: 84
  display_name: "book"
}
item {
  name: "/m/01x3z"
  id: 85
  display_name: "clock"
}
item {
  name: "/m/02s195"
  id: 86
  display_name: "vase"
}
item {
  name: "/m/01lsmm"
  id: 87
  display_name: "scissors"
}
item {
  name: "/m/0kmg4"
  id: 88
  display_name: "teddy bear"
}
item {
  name: "/m/03wvsk"
  id: 89
  display_name: "hair drier"
}
item {
  name: "/m/012xff"
  id: 90
  display_name: "toothbrush"
}

10/09/2018

google object detection api - error : Value Error: First Step Cannot Be Zero


This error is because of
schedule{
 step:0
 learning_rate: 0.00019...
}

So just change whole optimizer block to latest models optimizer block.
(This code is in the config file!)

before
optimizer {
momentum_optimizer {
learning_rate {
manual_step_learning_rate {
initial_learning_rate: 0.000199999994948
schedule {
step: 0
learning_rate: 0.000199999994948
}
schedule {
step: 900000
learning_rate: 1.99999994948e-05
}
schedule {
step: 1200000
learning_rate: 1.99999999495e-06
}
}
}
momentum_optimizer_value: 0.899999976158
}
use_moving_average: false
}


modified (example)
optimizer {
momentum_optimizer {
learning_rate {
cosine_decay_learning_rate {
learning_rate_base: 0.0399999991059
total_steps: 25000
warmup_learning_rate: 0.0133330002427
warmup_steps: 2000
}
}
momentum_optimizer_value: 0.899999976158
}
use_moving_average: false
}

or make step 0-> step 100 or something else, (not zero)


refer to this article
https://github.com/tensorflow/models/issues/3794

10/03/2018

show all PYTHONPATH in command line (tip)



python -c "import sys; print(sys.path)"
or
echo $PYTHONPATH

has type str, but expected one of: bytes (tf.train.Example)

make str to bytes

for example
#String to bytes
my_str = "file name"
my_str_as_bytes = str.encode(my_str)
type(my_str_as_bytes) # ensure it is byte representation
#byte to string
my_decoded_str = my_str_as_bytes.decode()
type(my_decoded_str) # ensure it is string representation

Pandas simple tip

import pandas

read csv file without header
df = pandas.read_csv('./train.csv', header=None, index_col=False)
print(df)

                0     1     2     3     4      5
0    0101_003.png   770   946  2070  2973  table
1    0110_099.png   270  1653  2280  2580  table
2    0113_013.png   303   343  2273  2953  table
3    0140_007.png   664  1782  1814  2076  table
4    0146_281.png   704   432  1744  1552  table
5    0146_281.png   682  1740  1800  2440  table
6    0147_090.png   326   413  2106  1616  table
7    0147_090.png   760  1843  1643  2393  table
8    0147_125.png   310   338  2310   912  table
9    0147_125.png   754  1184  1798  1514  table
10   0147_256.png   590   366  1940  1520  table
..            ...   ...   ...   ...   ...    ...
410  9529_050.png   104  2234  2040  2512  table
411  9530_051.png    90   470  2394  1682  table
412  9531_070.png   166   368  2328  1088  table
413  9531_070.png   148  1100  2340  1788  table
414  9531_073.png    50   260  2336  2876  table
415  9532_146.png   563   490  2440  2853  table
416  9533_038.png  1278   454  2326  1358  table
417  9533_038.png  1270  1774  2328  2368  table

[418 rows x 6 columns]

several way to read first column
print(df[0].tolist())
print(df.values.tolist()[:][0])
print(df.iloc[:,0].values.tolist())

['0101_003.png', '0110_099.png', '0113_013.png', '0140_007.png', '0146_281.png', '0146_281.png', '0147_090.png', '0147_090.png', '0147_125.png', '0147_125.png', '0147_256.png', '0147_256.png', '0148_271.png', '0148_479.png', '0151_180.png', '0151_208.png', '0154_080.png', '0154_474.png', '0155_081.png', '0199_384.png', '0203_075.png', '0206_007.png', '0206_048.png',
...
'9522_041.png', '9522_055.png', '9522_055.png', '9525_037.png', '9525_043.png', '9525_043.png', '9525_043.png', '9526_017.png', '9526_028.png', '9526_028.png', '9526_028.png', '9527_018.png', '9527_024.png', '9527_024.png', '9528_043.png', '9528_043.png', '9528_061.png', '9528_061.png', '9528_061.png', '9529_050.png', '9529_050.png', '9529_050.png', '9530_051.png', '9531_070.png', '9531_070.png', '9531_073.png', '9532_146.png', '9533_038.png', '9533_038.png']

pandas for each row
for index, row in df.iterrows():
print(index, row[0], row[1])

0 0101_003.png 770
1 0110_099.png 270
2 0113_013.png 303
3 0140_007.png 664
4 0146_281.png 704
5 0146_281.png 682
6 0147_090.png 326
7 0147_090.png 760
8 0147_125.png 310
9 0147_125.png 754
10 0147_256.png 590
11 0147_256.png 368
...