[Kaggle Study] 3. Learning Rate

What is Learning Rate?

• Learning rate is a crucial hyperparameter in machine learning that controls the degree of updates on the model's weights.
• Generally, a loss function represents a hyperplane(초평면) in multi-dimensional space characterized by complex curvatures.
• If the weight updates are too large and overshoot the global minimum(the point where the loss function could be minimized), it becomes impossible to find the optimal solution(optimal weights and biases). 
  • If the learning rate is too high, the weight cahnges can be too large, causing it to overshoot the global minimum.
  • When the learning rate is appropriate, weight changes are stable, allowing the model to efficiently reach the global minimum.
• Therefore, it's necessary to adjust the amount of weight updates to avoid missing the global minimum. 

reference: https://www.jeremyjordan.me/nn-learning-rate/

 

 

Reference:

Do it! 딥러닝 입문

 

 

Learn to embrace failure, for it is in those moments that you learn the most.

- Max Holloway -