Simultaneous Perturbation Stochastic Approximation

Simultaneous Perturbation Stochastic Approximation (SPSA) is an optimization method that estimates the gradient of an objective function using only two function evaluations per iteration, regardless of the problem’s dimension. It does this by randomly perturbing all parameters simultaneously and using the difference in the resulting function values to approximate the gradient. SPSA is efficient for noisy or high-dimensional problems where computing exact or finite-difference gradients is costly.