# Activation Functions Activation Functions decide whether a neuron in a [[Neural Networks (NNs)]] will activate or not. It basically states whether the inputs are relevant or should be ignored. Without an activation function, a Neural Network is just a linear regression model Activation Functions take a weighted sum of the neuron's inputs, called the dot product of the inputs (cfr [[Forward Propagation]]) and map it to a non-linear space (e.g., a value between 0 and 1). Activation Functions are thus non-linear transformations. Examples: - [[Sigmoid Function]] - [[Rectified Linear Unit Functions (RELU)]]