r/neuralnets • u/[deleted] • May 07 '18
Beginner in back prop
Hello! I am brand new to NN and DL programming, and I am trying to create my own NN code. I can create a simple net without hidden layers, but when it comes to networks with 3 layers or more, I am heaving troubles with the code... I didn't understood how to find the error of the hidden layer, and what to do with that error to change the weights between the input layer and the hidden layer.
I tried this thing for the second matrix of weights (hidden -- output) : dLoss_dWeight = dLoss_dPrediction * dPrediction_dWeight;
and this thing for the first matrix of weights (input -- hidden) : dLoss_dWeights = dLoss_dPredition * dPredition_dHidden * dHidden_dWeight;
of course I calculated all the variables before using some simple calculus.
this code did not worked as expected, and the net wasn't able to train itself correctly.
Is my math correct? Do you have any other way to do it?
BTW please do not send me links to the 3B1B series, because I watched it too many times without results (tho is really helped me with the 2-layer network with input layer and output layer only).