decision boundary
epoch 0 · train – · test –
epoch 0 | train –
● A ● B ○ –what to noticewhat to look for
One hidden layer is the whole story here. Strip it away (take the layer count to zero) and watch the boundary stay a straight line at roughly half wrong forever - the collapse argument from Maths of Learning ch2, running live. Put it back and the four corners separate in seconds.
instruments
loss
log scale– train –-- test –
gradient magnitude by layer
Gradients shrink as they travel back – the products-of-slopes fact from Maths of Learning ch3, live.