
Source: Giphy
Wanna Make AI Series #3
Hello all, welcome to the 3rd part of the "Wanna Make AI Series". Today we will be learning about the loss function and gradient descent and why do we need it and try out some codes to visualize it. Before we begin make sure to check out the first part and the second part of this series as info in those posts is required to build your knowledge step by step.
Also, you can get all the code of this series on our Github.
So, lets begin and dive into the topics.
![]() |
Source: Giphy |
Some basic understandings
![]() |
High loss in left model and low loss in right model |
Squared Loss (or L2 Loss)
where:
D is a data set containing many labeled examples, which are
N is the number of examples in D.
Although MSE is commonly-used in machine learning, it is neither the only practical loss function nor the best loss function for all circumstances.
Reducing the Loss
Till now we saw that how can we determine the loss that a model is incurring. Now, we want that our model should be having the least loss. Hence, for this there are various ways like:
- Iterative Approach
- Gradient Descent or Batch Gradient Descent
- Stochastic Gradient Descent
- Mini Batch Gradient Descent
![]() |
Source: Github |