Figure 5
The graph plots f of x on the vertical axis and x on the horizontal axis from minus 5 to 5. Five curves are shown. Softplus of x increases smoothly from near 0 and rises steeply for positive x. Tanh of x stays between minus 1 and 1 and approaches these limits at large magnitudes of x. Swish of x starts slightly negative, dips below 0, then increases and follows a rising trend for positive x. Sinus of x oscillates between minus 1 and 1 with peaks and troughs across the range. Re L U of x is 0 for negative x and increases linearly for positive x.Activation functions utilised within neural network architectures