3/03/2016

Deep learning study - tensor flow install on window #7

TensorFlow is running on Linux and Mac based.
Windows environment is not yet supported.

But if we use docker s/w that virtual machine similar with virtual box, we can run tensor flow on window environment.
In other words, docker create virtual environment for linux environment.
So how we use the tensorflow result that learned based on linux or mac in window?

I have not tried it yet.
There are api for c++ version in tensorflow.
There is no learning part, C++ version is included only evaluating part.
But if I success build c++ version api of tensor flow in window environment, we available the learning result in window after learning from linux.

I will try this plan after test basic tensorflow.

Then, let's use the tensorflow using docker s/w on window.

Firstly,
Tutorial about installation in linux and mac, refer to official site.
https://www.tensorflow.org/versions/r0.7/get_started/os_setup.html

---------------
#install docker
0. docker install guide
First page to install docker for various environments
https://docs.docker.com/engine/installation/

1. download docker toolbox
download docker for widnow and mac user
https://www.docker.com/products/docker-toolbox


2. install
document for install docker in window
https://docs.docker.com/engine/installation/windows/






3. run docker
excute Docker Quickstart terminal.
After setting (first takes about 1~2 minutes)

You can see the whale illustration.

verify your setup, type this command
$ docker run hello-world

then you can this message.

---------------
#install TensorFlow
type this command
$docker-machine ip default
then you can check notebook server ip.


type this command
$ docker run -it b.gcr.io/tensorflow/tensorflow
or
$ docker run -p 8888:8888 -it b.gcr.io/tensorflow/tensorflow

when you enter the command firstly, tensorflw is installed once.
and run notebook server.
but in my case, first command is not run notebook server.
If you problem with me, type second command.

---------------
#run notebook
type this on you browser
http://192.168.99.100:8888
then you can see notebook web page.



---------------
#test TensorFlow

*setup container
docker run -p 8888:8888 -d --name tfu b.gcr.io/tensorflow-udacity/assignments
*starting your container
docker start tfu
*stopping your container
docker stop tfu
show detail here
https://discussions.udacity.com/t/jupyter-notebooks-docker-windows-progress-not-being-saved/46116/4


#build a local Docker container
move to your directory
"....\tensorflow-master\tensorflow\examples\udacity"
tensorflow-master is downloaded on github

and write command (must be typing '.' )
docker build -t imageName .
about this issue refer to
https://discussions.udacity.com/t/difficulty-starting-docker-for-assignments-in-deep-learning-course/45201


----------------
*docker useful command
//show images
docker images

//show running containers
docker ps

//remove image
docker rmi imageID

//remove container
docker rm containerID





3/02/2016

Deep learning study - minimizing cross entropy #6


Deep learning flow is like that.
Expressed as a formula, It is D(S(Wx+b),L).
So It is Multinational logistic classification.


The smaller value of the cross entropy function D means close to the correct classification.



So the smaller the value of the sum of D function for all the input and labeling values can be said correct w, b is got good.

Then how to get optimization values of w, b?
That is optimization problem.

Usually using the gradient descent method via a derivative of these problems.



Sorry, my english is not good. ^^



3/01/2016

Deep learning study - cross entropy #5


The whole process of deep learning is as described above.
The final step in cross entropy is compared to the classification and labeling values.



S and L is vector. L is final result that is determined by the person.
D function means distance value.
The smaller distance value means that the result is correct.
Thus, W, b parameter adjust, the D value should be smaller.

This is matlab test.

case 1.
S = [0.7 0.2 0.1];
L = [1.0 0 0];
- sum( L.*log(S) )

=>  0.3567

case 2.
S = [0.7 0.2 0.1];
L = [0 0 1.0];
- sum( L.*log(S) )

 2.3026



In case 1, since the result of softmax is similar to the label, distance is small than case 2.



2/29/2016

SVD function built-in matlab convert to c++ code using matlab coder app.

I had a brief introduction about matlab coder in here.
http://study.marearts.com/2016/02/matlab-coder-simple-test-and-practical.html

Well, I have a interesting question.
It is also used to convert the MATLAB built-in functions to c ++ code?
example.. SVD function.

SVD(Singular Value Decomposition) is very useful function for solving linear algebra problem.
But it is difficult to find the source only pure c code, Often including a linear algebra as big library.

So, this article aims to convert SVD built in matlab function to c code and use the converted c code in Visual studio.

First, a start by calculating a fixed value of 4x2  matrix.



This is my example matrix and result.
This is an example in matlab help document.


First, let's look matlab code, it is very simple.
MySVD.m 
///
function [U, S, V] = MySVD(X)

[U, S, V]= svd(X);
///

testMySVD.m
///
X=[ 1 2
    3 4
    5 6
    7 8
    ];
    
[U, S, V] = MySVD(X);

U
S
V
U*S*V'
///

result


Now, let's convert MySVD.m to c code using matlab corder.

However, options for the input X is set to 4x2 double.


next,
An example using the code of MySVD c code in Visual Studio.
The result of visual studio is same with matlab.

MySVD C code is uploaded in github.




///
#include < iostream>
#include "MySVD.h"
using namespace std;


void main()
{
 double X[]={1, 3, 5, 7, 2, 4, 6, 8};
 double U[16]; //4x4
 double S[8]; //4x2
 double V[4]; //2x2

 //extern void MySVD(const real_T X[8], real_T U[16], real_T S[8], real_T V[4]);
 
 MySVD(X, U, S, V);

 cout << "U out" << endl;
 for(int i=0; i< 4; ++i) //row
 {
  for(int j=0; j< 4; ++j) //col
   cout << U[i+j*4] << " ";
  cout << endl;
 }
 cout << endl;

 cout << "S out" << endl;
 for(int i=0; i< 4; ++i) //row
 {
  for(int j=0; j< 2; ++j) //col
  {
   cout << S[i+j*4] << " ";
   
  }
  cout << endl;
 }
 cout << endl;

 

 cout << "V out" << endl;
 for(int i=0; i< 2; ++i) //row
 {
  for(int j=0; j< 2; ++j) //col
  {
   cout << V[i+j*2] << " ";
   
  }
  cout << endl;
 }
 cout << endl;

}
///

The sign is a little different
But value of U* S * V 'is as correct.
And sign is irrelevant since the opposite direction to the perpendicular direction.

Next let's study the use of nxm svd

First, let's create the code from matlab data type of NxN

:inf x :inf is nxn.
After code build, see the this c++ code

///
#include < iostream>
#include "MySVD.h"
#include "MySVD_emxAPI.h" //for emxArray_real_T
using namespace std;


void main()
{
double X[] = { 1, 3, 5, 7, 2, 4, 6, 8, 1, 2, 3, 4};
 double U[16]; //4x4
 double S[12]; //4x3
 double V[9]; //3x3



 emxArray_real_T *inputX, *outputV, *outputS, *outputU;
 inputX = emxCreateWrapper_real_T(&(X[0]), 4, 3);
 outputU = emxCreateWrapper_real_T(&(U[0]), 4, 4);
 outputS = emxCreateWrapper_real_T(&(S[0]), 4, 3);
 outputV = emxCreateWrapper_real_T(&(V[0]), 3, 3);

 //extern void MySVD(const emxArray_real_T *X, emxArray_real_T *U, emxArray_real_T *S, emxArray_real_T *V);
 MySVD(inputX, outputU, outputS, outputV);

 cout << "U out" << endl;
 for (int i = 0; i< 4; ++i) //row
 {
  for (int j = 0; j< 4; ++j) //col
   cout << U[i + j * 4] << " ";
  cout << endl;
 }
 cout << endl;

 cout << "S out" << endl;
 for (int i = 0; i< 4; ++i) //row
 {
  for (int j = 0; j< 3; ++j) //col
  {
   cout << S[i + j * 4] << " ";

  }
  cout << endl;
 }
 cout << endl;


 cout << "V out" << endl;
 for (int i = 0; i< 3; ++i) //row
 {
  for (int j = 0; j< 3; ++j) //col
  {
   cout << V[i + j * 3] << " ";

  }
  cout << endl;
 }
 cout << endl;

}
///


The input of dynamic size is set by "emxArray_real_T" type.
for example const emxArray_real_T *X

transfer double value to emxArray_real_T, we can use "emxCreateWrapper_real_T" function.
For this function, don't forget include "MySVD_emxAPI.h" or "xxxx_emxAPI.h".

You code like this.
inputX = emxCreateWrapper_real_T(&(X[0]), 4, 2);
or
outputU = emxCreateWrapper_real_T(&(U[0]), 4, 4);

double address is connected with emxArray_real_T.

Please check I have posted the source code in Github.

SVD 4x2 test
https://github.com/MareArts/Matlab-corder-test-SVD/tree/master/MySVD_test_4x2input
SVD 4xn test
https://github.com/MareArts/Matlab-corder-test-SVD/tree/master/MySVD_test_4xninput
SVD nx2 test
https://github.com/MareArts/Matlab-corder-test-SVD/tree/master/MySVD_test_nx2input
SVD 4x4 test
https://github.com/MareArts/Matlab-corder-test-SVD/tree/master/MySVD_test_NxN_input







Matlab coder simple test and practical use in Visual studio

Matlab coder is the tool that convert matlab code into C/C++.
If you very glad immediately to hear this, You will sympathize the needs with me.

This post is simple test that how to use matlab coder and how to utilize in the Visual studio.
see below video, then you will be able to very easily know.




1. make matlab function


2. execute matlab coder app in matlab app tab


 3. set file of project name.


4. set input parameter data type


 5. set static  c/c++ library and check code generate.

6. in more setting,
no check support non finite numbers in speed tab for non-necessary code no generation.
,no check make option, and check language c -> c++



7. build and view report.



8. Create empty project in VS, add generated c++ file to project
   function call and test.





See this video more detail.


note!, in the video, mex option selection is wrong action, static c/c++ library selection is right.