Neural-Network Basics

Key jargon

Term	Plain-language meaning
Neuron	A small computation that combines inputs, weights, a bias, and an activation.
Weight	A learned number controlling how strongly one signal affects another.
Loss function	A score describing how wrong a prediction is for the training objective.
Backpropagation	The procedure that computes how each parameter contributed to the loss.

Key concepts

A neural network is a differentiable function with learned parameters, not a digital brain.
Training repeatedly predicts, measures error, computes gradients, and updates weights.

Concept map

flowchart LR
    A["Input features"] --> B["Weighted layers"]
    B --> C["Prediction and loss"]
    C --> D["Backpropagation updates weights"]

Mental model

A neural network is a parameterized function. Inputs are converted to numbers, passed through layers, and transformed into outputs. Weights store learned numerical relationships; activations introduce nonlinear behavior; a loss function measures error; backpropagation computes how parameters contributed to that error; an optimizer updates them.

Training is repeated approximation—not a database write of one fact into one address.

Vocabulary

Term	Practical meaning
Parameter	Learned number affecting the function
Hyperparameter	Human-selected setting such as learning rate or batch size
Epoch	One pass over a training dataset
Overfitting	Memorizing training-specific patterns that fail to generalize
Regularization	Technique discouraging brittle or overly complex fits
Gradient	Direction and sensitivity of loss change

Misconceptions

More parameters do not guarantee better task performance.
Low training loss does not prove useful generalization.
A model’s internal representation is not a human-readable knowledge graph.

Exercise

Draw a tiny network with three inputs, one hidden layer, and two outputs. Label which values come from data, which are learned, and which are chosen by the builder.

Transformers and attention