# Convex functions

Course completion
33%
$\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator{\dom}{dom} \DeclareMathOperator{\sigm}{sigm} \DeclareMathOperator{\softmax}{softmax} \DeclareMathOperator{\sign}{sign}$

The particular case of convex functions plays an important role w.r.t. to the problem of local minima.

Definition 1.3 (convex function)
Let $f$ be a function defined over some domain $\dom f$. $f$ is said to be convex, iff $$\forall\mathbf{a},\mathbf{b}\in\dom f,\forall k\in[0;1],f(k\mathbf{a}+(1-k)\mathbf{b})\leq kf(\mathbf{a})+(1-k)f(\mathbf{b})$$

This means that any point between $\mathbf{a}$ and $\mathbf{b}$ has an image by $f$ which is below the line segment joining $(\mathbf{a},f(\mathbf{a}))$ and $(\mathbf{b},f(\mathbf{b}))$ as depicted in the figure below.

Figure 2: The graph of a convex function $f$. The values of $f$ between two points $a$ and $b$ are always below the chord, i.e. the line segment between $(\mathbf{a},f(\mathbf{a}))$ and $(\mathbf{b},f(\mathbf{b}))$.

Convex functions have the very interesting property that any local minimum is in fact a global minimum, thus simplifying the problem for practitioners.

Next: Continuous differentiable functions