Tensorflow RNN LSTM weight save and restore example code

I have been struggled for long time for save and restore the result of LSTM params.
Today, I have succeeded, I hope anyone helping this my example code.


Below code is example to learning for
input: hihell -> output: ihello

gist code start
This code is referenced by this(https://github.com/MareArts/DeepLearningZeroToAll/blob/master/lab-12-1-hello-rnn.py)

gist code end

There are 4 variable for trainable
name rnn/basic_lstm_cell/weights:0, shape (10, 20)
name rnn/basic_lstm_cell/biases:0, shape (20,1)
name fully_connected/weights:0, shape (5, 5)
name fully_connected/biases:0, shape (5,1)

And I have checked the values which are same after "global_variables_initializer"
The result is same and prediction result is also same.

OK, then let's move more complicated RNN design.
This example code for 2 layer LSTM and 2 batch condition.
gist code start

gist code end


Maintain

• https://github.com/MareArts/rnn_save_restore_test

Reference

• reference code
• https://github.com/MareArts/DeepLearningZeroToAll
• print trainable value
• https://stackoverflow.com/questions/43158606/how-to-get-weights-from-tensorflow-fully-connected
• LSTM save & restore comment
• https://github.com/tensorflow/tensorflow/issues/13438
• https://stackoverflow.com/questions/40442098/saving-and-restoring-a-trained-lstm-in-tensor-flow
• RNN initialize 
• https://stackoverflow.com/questions/37969065/tensorflow-best-way-to-save-state-in-rnns
• fully connected weight, bias save
• https://stackoverflow.com/questions/43158606/how-to-get-weights-from-tensorflow-fully-connected
• trainable variable 
• https://stackoverflow.com/questions/41951657/using-tf-trainable-variables-to-show-names-of-trainable-variables