Activation function — An activation function decides how strongly an artificial neuron passes its signal to the next layer. It adds non-linearity, which lets neural networks learn complex patterns instead of only straight-line relationships. Common examples include ReLU and GELU. Without activation functions, deep networks would be no more powerful than simple ones.

Added October 21, 2026