# Neural Networks (NNs) Neural Networks were originally inspired by the brain, but have very little in common. While the human brain consists of billions of interconnected neurons that communicate through complex electrochemical processes, artificial neural networks (ANNs) are composed of layers of simple computational units called "nodes" or "neurons". ## Artificial Neurons Artificial Neurons mimic the way actual brain neurons work: - Many dendrites send information to the Soma (input) - The Soma processes the information (processing) and sends it to the Axon (output) ## Overview (Artificial) Neural Networks take a set of inputs and output an answer. A Neural Network (NN) is a big mathematical equation that maps inputs to a specific output. An artificial Neural Network is composed of multiple layers: ![[ANN-layers.png]] A neural network (NN), whether tiny or very large, is formed by stacking neurons together. Each neuron is like a Lego brick. Each neuron implements a function (e.g., a [[Rectified Linear Unit Functions (RELU)]]) that maps input to output. The first layer, usually called the **input layer**, feeds the input into the network. The set of nodes that produce the output are called the **output layer**. Any sets of nodes in between the input and the output layers are called the **hidden layers**. When fed with many examples of inputs and correct outputs, ML algorithms figure out what the neurons of each layer should recognize/do. In the case of face recognition (e.g., identifying the presence of a face and mapping it to an identity), the first layers identify edges, while subsequent layers recognize objects, faces, and the final layer can map to the corresponding identity (if recognized) ## Artificial Neural Network Example ![[nn-example-demand-prediction.png]] ## Main topics associated with Artificial Neural Networks - [[Forward Propagation]] - [[Backpropagation]] - [[Activation Functions]] - Without an activation function, a Neural Network is just a linear regression model ## Processing Example ![[Neural Network processing example.png]] The above Neural Network takes two inputs: `x1` and `x2`. It has one hidden layer with two nodes and an output layer with one node. To calculate `z1,1`, we do: `(x1 * w1) + (x2 * w2) + b1,1`. To calculate `z1,2`, we do: `(x1 * w3) + (x2 * w4) + b1,2`. Assuming we use the [[Sigmoid Function]]: `sigmoid(z) = 1 / 1 + e^-z`, we can compute the activation of the nodes: - `a1,1 = sigmoid(z1,1)` - `a1,2 = sigmoid(z1,2)` Those activations serve as the input to the output layer `z2`: - `z2 = a1,1 * w5 + a1,2 * w6 + b2` Finally, we can compute the output of the network: - `a2 = sigmoid(z2)` Real neural networks are composed of many hidden layers and many more nodes in each layer. ## Generalized neural network A general neural network takes `n` inputs, has many hidden layers, each with `m` nodes, and an output layer. For instance: ![[ANN-layers.png]] ## References - ## Related - [[Machine Learning (ML)]] - [[Deep Learning]] - [[WebNN API]] - [[WebMachineLearning]]