Minimizing a function means walking downhill on its landscape. But how you step matters enormously. Plain gradient descent follows the steepest slope and zig-zags down a narrow valley; momentum builds up speed and glides through; Newton's method reads the curvature and cuts almost straight to the bottom. Drag the start and race all three at once.
Drag the ring on the landscape to move the shared start point.
gradient descent x ← x − η∇fmomentum velocity carries throughNewton x ← x − H⁻¹∇fThe landscape is an ill-conditioned bowl — a long, narrow valley — with a mild bend. Gradient descent only knows the local slope, so it bounces between the steep walls and crawls along the shallow floor; too large a rate and it diverges. Momentum accumulates a velocity that cancels the side-to-side bouncing and reinforces the downhill drift, so it glides through the valley far faster — the same trick behind the optimizers (Adam, RMSProp) that train neural networks. Newton's method uses the second derivative, the Hessian, to rescale the step by the local curvature, so it heads almost straight to the minimum in a handful of steps — powerful, but each step needs the full curvature, which is why large problems approximate it. Same landscape, same start; the only difference is how much each method knows.