# Rectified Linear Unit Functions (RELU) A RELU function maps input to output: ![[20230821145842 relu.png]] With [[Sigmoid Function]], learning is slower as the gradient tends towards zero. With ReLUs, this is not the case. This makes gradient descend much more performant: ![[20230821152646 sigmoid.png]]