Optimization: Convex Geometry & Fenchel Duality
Optimization: Convex Geometry & Fenchel Duality
💡 Key Takeaways
- Understand convex geometry basics (convex sets, norms, and duality) and how geometric structure shapes optimization.
- Learn the Fenchel conjugate and Fenchel duality theorem, and how to derive dual problems from primal convex programs.
- Apply dual concepts—dual norms, polar sets, and support functions—to bound and analyze optimization problems.
- Explore concrete Fenchel conjugates for common functions (norms, indicator functions) with simple examples to illustrate duality in action.
❓ Frequently Asked Questions
What is convex geometry, and why is it important in optimization?
Convex geometry studies convex sets and convex functions—objects where line segments stay inside the set and the function lies below chords. Key concepts include convex sets, extreme points, support functions, and dual cones. In optimization, convexity guarantees global optima and often enables efficient algorithms and duality-based methods.
What is Fenchel duality?
Fenchel duality links a primal optimization problem to a dual problem by means of Fenchel conjugates. For example, min_x f(x) + g(Ax) has a dual max_y −f*(Aᵀy) − g*(−y). Under mild convexity and regularity (e.g., Slater-type conditions), the primal and dual optimal values coincide.
What is a Fenchel conjugate?
For a function f, its Fenchel conjugate is f*(y) = sup_x {⟨x,y⟩ − f(x)}. It captures how steeply f grows in direction y and serves as a key building block in Fenchel duality (e.g., conjugates of indicators and norms relate to dual objects). For example, if f(x) = (1/2)||x||², then f*(y) = (1/2)||y||².
When does Fenchel duality give exact solutions (no duality gap)?
Strong duality holds for convex, proper, lower semicontinuous functions with a suitable constraint qualification (e.g., Slater’s condition). In that case, the primal optimum equals the dual optimum, and dual variables provide optimal subgradients.