Deep learning study - cross entropy #5

The whole process of deep learning is as described above.

The final step in cross entropy is compared to the classification and labeling values.

S and L is vector. L is final result that is determined by the person.

D function means distance value.

The smaller distance value means that the result is correct.

Thus, W, b parameter adjust, the D value should be smaller.

This is matlab test.

case 1.

S = [0.7 0.2 0.1];

L = [1.0 0 0];

- sum( L.*log(S) )

=>  0.3567

case 2.

S = [0.7 0.2 0.1];

L = [0 0 1.0];

- sum( L.*log(S) )

 2.3026

In case 1, since the result of softmax is similar to the label, distance is small than case 2.