Strong convexity and L-smoothness together form a 'quadratic sandwich' that bounds a function between two parabolas, controlling how well gradient descent behaves. Strong convexity sets a floor on curvature (ensuring the gradient carries meaningful signal about distance to the minimum), while L-smoothness sets a ceiling (ensuring the landscape doesn't change too abruptly between steps). Their ratio — the condition number κ = L/μ — measures how thick the sandwich is and directly predicts gradient descent difficulty: a large κ causes zigzagging because a single step size can't serve both high- and low-curvature directions. From a spectral perspective, these properties correspond to global bounds on Hessian eigenvalues. A practical verification trick reduces checking either property to testing plain convexity of a modified function, avoiding explicit eigenvalue computation.

17m read timeFrom fedemagnani.github.io
Post cover image
Table of contents
Strong convexity — the function can’t be too flatL-smoothness — the function can’t be too steepThe quadratic sandwichWhat goes wrong without one slice of breadThe spectral perspective — reading the Hessian’s eigenvaluesThe verification trickWrapping upAppendix: from Lipschitz gradients to the descent lemma
296 Impressions