Hyperparameter tuning

Sort of hyperparameter : learning rate, very important #hidden units #mini-batch size #layers learning rate decay 등등.. 중요도에는 차이가 있음 How to choice Try random values : Don't use a grid 어떤 hyperparameter가 중요한지 알 수 없는 단계에서는, 격자에서 순서대로 선정하기 보다는, 아예 랜덤으로 선택하는 것이 낫다. Coarse to fine : 먼저 무작위로 선택하다가 성능이 […]

Read More

Week 2 - Python and Vectorization

Vectorization if non-verctorized:

if vectorized:

it is much faster

Vectorizing Logistic Regression b is real number, it will be automatically changed to vector to be added each element of matrix (python broadcasting)

A note on python/numpy vectors to simplify code and to avoid bug, don't use rank 1 array

Read More