Sigmoid Function

If the input $x$ to the sigmoid function is a small negative number, the output is very close to 0. For example:

$$\begin{align} \Large \sigma(-4) &= \frac{1}{1+e^{-(-4)}} \\[1em] &= 0.01798621 \end{align}$$

Indeed, we can use the limit to show that $\sigma(x)$ approaches 0 as $x$ tends to $-\infty$:

$$\begin{align} \Large \lim_{x\to-\infty} \sigma(x) &= \lim_{x\to-\infty} \frac{1}{1+e^{-x}} \\[1em] &= \lim_{x\to-\infty} \frac{1}{1+e^{\infty}} \\[1em] &= 0 \end{align}$$

If $x$ is a large positive number, then the output is very close to 1:

$$ \begin{align} \Large \sigma(4) &= \frac{1}{1+e^{-(4)}} \\[1em] &= 0.9820138 \end{align} $$

Again, we can use the limit to show that $\sigma(x)$ approaches 1 as $x$ tends to $\infty$:

$$\begin{align} \Large \lim_{x\to\infty} \sigma(x) &= \lim_{x\to\infty} \frac{1}{1+e^{-x}} \\[1em] &= \lim_{x\to\infty} \frac{1}{1+e^{-\infty}} \\[1em] &= 1 \end{align} $$

If $x$ is exactly 0, the output is $0.5$:

$$\begin{align} \Large \sigma(0) &= \frac{1}{1+e^{-(0)}} \\[1em] &= 0.5 \end{align}$$

Now that we’ve seen how the sigmoid function behaves at its limits let’s move on to its derivative.