Stochastic Gradient Descent

Stochastic Gradient Descent (SGD) is an optimization algorithm that updates model parameters using the gradient from a single (or small batch of) randomly chosen training sample(s) rather than the full dataset. This makes updates faster but noisier, which can help escape local minima. It iteratively moves parameters in the direction that reduces the loss function until convergence.