Deep Learning & Neural Networks

The big picture: what deep learning is and why it works now

Artificial intelligence is the broad goal of making machines act intelligently. Machine learning is the sub-field where behaviour is learned from data instead of being written as explicit rules. Deep learning is the part of machine learning that uses neural networks with many layers. The "deep" refers to the number of stacked layers, each one transforming the output of the previous one into a slightly more abstract representation.

The key difference from classical machine learning is where the features come from. A classical pipeline for spotting cats needs a human to design features (edge histograms, color statistics, texture descriptors), and a model such as logistic regression or a random forest learns on top of them. A deep network takes raw pixels and learns the features itself: early layers discover edges, middle layers combine edges into textures and parts, and late layers respond to whole concepts such as "cat face". This is called representation learning or end-to-end learning.

A short history, and the three reasons it took off

Neural networks are old; what changed is that they finally became trainable at scale. The history is a sequence of booms and "AI winters" (periods when funding and interest dried up because promises outran results).

When deep learning is (and is not) the right tool

A good real-world illustration: on a balanced movie-review sentiment dataset (50,000 reviews), a well-regularized linear SVM or logistic regression on bag-of-words features reached an F1 of about 0.875, while a vanilla RNN trained without gradient clipping stayed at chance level (F1 about 0.57) and a small attention model reached about 0.86. Deep learning is not automatically better; it needs the right architecture, tricks and data. That case study appears again in the RNN section.

Neurons, perceptrons and multi-layer networks

The building block is the artificial neuron, loosely inspired by biology. A biological neuron receives signals on its dendrites, integrates them in the cell body, and fires along its axon if the total exceeds a threshold; synapses strengthen or weaken with learning. The artificial version keeps only the arithmetic.

The perceptron and its limits

The perceptron is a single neuron with a step activation: output 1 if z > 0, else 0. Its decision boundary w·x + b = 0 is a straight line (a hyperplane in higher dimensions), so it can only solve linearly separable problems. AND and OR are separable; XOR (output 1 when exactly one input is 1) is not, because no single line separates (0,1) and (1,0) from (0,0) and (1,1). Adding a hidden layer fixes this: one hidden neuron can compute OR, another NAND, and the output neuron ANDs them together.

XOR is not linearly separable       A 2-2-1 network solves it

 x2                                 x1 ---+--> [h1 = OR ] ---+
  1 |  (1)        (0)                      \  /                 \
    |                                       \/                   [ y = AND(h1,h2) ]
    |                                       /\                   /
  0 |  (0)        (1)               x2 ---+--> [h2 = NAND] ---+
    +--------------------- x1
       0           1               Hidden layer bends the space so
  No single line separates           the output layer can use a line.
  the 1s from the 0s.

The multi-layer perceptron (MLP)

An MLP (also called a feedforward or fully connected network) stacks layers of neurons. Every neuron in one layer connects to every neuron in the next. Information flows one way, input to output, with no loops. A layer with n inputs and m outputs is just a matrix multiplication plus a bias vector, followed by an element-wise activation.

Input layer        Hidden layer 1      Hidden layer 2      Output layer
 (features)        (4 units, ReLU)     (3 units, ReLU)     (softmax)

   x1  o----------->  o ----------------->  o  ------------>  o  P(cat)
       \  \   /  /    o                     o                o  P(dog)
   x2  o---X-X-------> o ----------------->  o  ------------>  o  P(bird)
       /  /   \  \    o
   x3  o-------------->
                  every arrow = one learnable weight; every unit also has a bias
 Parameters: (3*4+4) + (4*3+3) + (3*3+3) = 16 + 15 + 12 = 43

Why non-linearity is essential

If every activation were the identity, two layers would compute W2(W1x + b1) + b2 = (W2W1)x + (W2b1 + b2), which is a single linear layer. A hundred linear layers are still one linear layer. The activation "bends" the space between layers, so stacked layers can carve curved decision boundaries and compose features. Real-world problems are not turned into linear ones; instead, the network builds a non-linear function out of many simple linear pieces joined by non-linearities. With ReLU, the network is literally a piecewise-linear function with a huge number of pieces.

Universal approximation, and why depth still matters

The universal approximation theorem says a network with one hidden layer and a suitable non-linear activation can approximate any continuous function on a bounded region to any accuracy, given enough hidden units. It is an existence result: it says nothing about how many units you need or whether gradient descent will find the weights. In practice, depth is far more efficient than width: deep networks reuse intermediate features (edges to parts to objects), and some functions need exponentially many units in a shallow network but only polynomially many in a deep one.

Worked example: one forward pass by hand

Take a 2-2-1 network with ReLU hidden units and a sigmoid output. Input x = [1, 2].

1. Hidden neuron 1 weights [0.5, −0.2], bias 0.1: z₁ = 0.5×1 + (−0.2)×2 + 0.1 = 0.2, so h₁ = ReLU(0.2) = 0.2.
2. Hidden neuron 2 weights [0.3, 0.4], bias −0.5: z₂ = 0.3 + 0.8 − 0.5 = 0.6, so h₂ = 0.6.
3. Output neuron weights [1.0, −0.5], bias 0.1: z = 1.0×0.2 + (−0.5)×0.6 + 0.1 = 0.0.
4. Output activation: ŷ = σ(0) = 1/(1 + e⁰) = 0.5. The model is exactly undecided; training will push the weights so that ŷ moves toward the true label.

The numbers inside hidden nodes (0.2, 0.6) are activations, not weights. Weights live on the connections. Weights can be any real numbers, do not need to sum to 1, and become decimals simply because they are updated continuously.

Matrix form and batching

Real code processes a mini-batch of B examples at once. Stack the inputs as rows of a matrix X of shape (B, n). Then a layer is H = f(X WT + b), where the bias is broadcast across rows. This is why GPUs matter: a whole batch through a layer is one large matrix multiplication.

import torch
import torch.nn as nn

class MLP(nn.Module):
    def __init__(self, in_dim=20, hidden=64, n_classes=3, p_drop=0.1):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(in_dim, hidden),   # W1: (64, 20), b1: (64,)
            nn.ReLU(),
            nn.Dropout(p_drop),
            nn.Linear(hidden, hidden),
            nn.ReLU(),
            nn.Linear(hidden, n_classes) # raw scores (logits), no softmax here
        )

    def forward(self, x):                # x: (batch, in_dim)
        return self.net(x)               # (batch, n_classes)

model = MLP()
x = torch.randn(32, 20)                  # a batch of 32 examples
logits = model(x)
print(logits.shape)                      # torch.Size([32, 3])
print(sum(p.numel() for p in model.parameters()))  # 20*64+64 + 64*64+64 + 64*3+3 = 5,699

Activation functions

An activation function is applied element-wise after each linear layer. It provides the non-linearity, and its derivative controls how well gradients flow backward. Choosing one is about three things: does it add non-linearity, does it keep gradients healthy, and is it cheap to compute.

The activation zoo

Worked numbers

• σ(0) = 0.5, σ(2) = 1/(1 + e⁻²) = 1/1.135 ≈ 0.881, σ(−4) ≈ 0.018. The derivative at z = 4 is 0.982 × 0.018 ≈ 0.018: almost no learning signal.
• ReLU(3) = 3, ReLU(−2) = 0. ReLU does not squash to [0, 1]; its output is unbounded above.
• GELU(−1) ≈ −0.159 while ReLU(−1) = 0; GELU(2) ≈ 1.955, nearly identical to ReLU for positive inputs.
• Softmax of logits [2, 1, 0]: e² = 7.389, e¹ = 2.718, e⁰ = 1, sum 11.107, giving [0.665, 0.245, 0.090]. Higher score, higher probability, and everything sums to 1.

Why ReLU is non-linear even though it looks linear

ReLU is linear on each side of zero, but the kink at zero breaks global linearity. Different neurons switch on for different inputs, so the network becomes a patchwork of many linear regions whose boundaries depend on the input. That patchwork can approximate any curve. ReLU is not differentiable exactly at 0; frameworks simply use a subgradient (0 or 1) there, and since landing exactly on 0 is rare with real-valued inputs, it causes no practical problem.

The dying ReLU problem

If a neuron's pre-activation becomes negative for every input (often after a large gradient step pushes its bias very negative), it outputs 0 and its gradient is 0, so its weights never update again. Fixes: Leaky ReLU/PReLU/ELU/GELU, He initialization, a lower learning rate, and normalization layers that keep pre-activations centered.

Quick decision guide

import torch
import torch.nn.functional as F

z = torch.tensor([-2.0, -0.5, 0.0, 0.5, 2.0])
print(F.relu(z))            # tensor([0.0, 0.0, 0.0, 0.5, 2.0])
print(torch.sigmoid(z))     # tensor([0.119, 0.378, 0.500, 0.622, 0.881])
print(torch.tanh(z))        # tensor([-0.964, -0.462, 0.000, 0.462, 0.964])
print(F.gelu(z))            # small negative values survive
print(F.silu(z))
print(F.softmax(z, dim=0))  # sums to 1.0

Loss functions

The loss function turns "how wrong is the model?" into a single number that gradient descent can minimize. Strictly, a loss is the error on one example and the cost (or objective) is the average over the dataset or batch, often plus a regularization term. In practice and in libraries, "loss" is used for both, and nn.MSELoss() returns the batch mean by default.

Regression losses

Business costs should drive the choice. If the true demand is 100 and model A predicts 120 while model B predicts 80, both have the same squared and absolute error, yet over-forecasting may mean wasted inventory while under-forecasting means lost sales. An asymmetric loss such as quantile loss encodes that preference directly.

Classification losses

Worked example. A spam email (y = 1). If the model predicts p = 0.9, BCE = −ln(0.9) ≈ 0.105 (small). If it predicts p = 0.1, BCE = −ln(0.1) ≈ 2.303. If it confidently says p = 0.05, BCE = −ln(0.05) ≈ 3.0, which is "very high". As p approaches 0 for a true positive, the loss goes to infinity. Correct and confident gives low loss; wrong and confident is punished hardest.

Why cross-entropy and not MSE for classification?

• Probabilistic meaning: cross-entropy is the negative log-likelihood of a categorical (or Bernoulli) model, so minimizing it is maximum likelihood estimation.
• Better gradients: with a sigmoid or softmax output, the gradient of cross-entropy with respect to the logits is simply p − y. With MSE, the gradient is multiplied by σ'(z), which is near zero when the model is confidently wrong, so learning stalls exactly when it is needed most.
• Convexity in the last layer: logistic regression with cross-entropy is convex; with MSE it is not.

Entropy, cross-entropy and KL in one line each

• Entropy H(p) = −Σ p log p: how unpredictable a distribution is (a fair coin has 1 bit).
• Cross-entropy H(p, q) = −Σ p log q: the average cost of encoding outcomes from the true distribution p using the model's distribution q.
• KL divergence KL(p || q) = H(p, q) − H(p): the extra cost of using q instead of p. With one-hot labels H(p) = 0, so cross-entropy equals KL.

Calibration and confidence

A model's "confidence" is the probability it assigns to its predicted class. A model is well calibrated if predictions made with 80% confidence are right about 80% of the time. Accuracy measures correctness; calibration measures whether the probabilities can be trusted. Modern deep nets tend to be over-confident. Temperature scaling (dividing logits by a learned T > 1 on a validation set) fixes calibration without changing the ranking or accuracy. Label smoothing also helps. The decision threshold (0.5 by default) is separate from the loss and should be tuned on validation data for the precision/recall trade-off you need.

What is a "good" loss value?

There is no universal target. A loss value depends on the loss type, the number of classes and the noise in the data. Useful reference points: for K balanced classes, a model that predicts uniformly has cross-entropy ln(K) (0.693 for 2 classes, 2.303 for 10). Judge progress by: loss decreasing over time, training and validation loss staying close, and beating a sensible baseline. Do not train until the training loss reaches 0: real data contains noise, and driving training loss to zero usually means memorization.

import torch, torch.nn as nn

logits = torch.tensor([[2.0, 1.0, 0.0]])      # one example, 3 classes
target = torch.tensor([0])                    # class index, not one-hot
print(nn.CrossEntropyLoss()(logits, target))  # -log(0.665) = 0.408

bin_logit = torch.tensor([-2.944])            # sigmoid(-2.944) = 0.05
print(nn.BCEWithLogitsLoss()(bin_logit, torch.tensor([1.0])))  # about 3.0

# class imbalance: weight the rare class more
loss_fn = nn.CrossEntropyLoss(weight=torch.tensor([1.0, 5.0, 1.0]),
                              label_smoothing=0.1)

Gradient descent and backpropagation

Training means finding weights that minimize the loss. The loss surface of a neural network is a function of millions of weights, non-convex, full of saddle points and flat regions. We cannot solve for the minimum directly, so we walk downhill using the gradient: the vector of partial derivatives of the loss with respect to every weight. It points in the direction of steepest increase, so we step the opposite way.

Batch, stochastic and mini-batch gradient descent

Terminology: an iteration (or step) is one parameter update on one mini-batch; an epoch is one full pass over the training set. With 50,000 examples and batch size 100, one epoch is 500 iterations. "SGD" in libraries usually means mini-batch SGD.

Backpropagation is the chain rule, applied efficiently

A network is a composition of functions, so the loss depends on an early weight through a chain of intermediate values. The chain rule says: if L depends on ŷ, which depends on z, which depends on w, then

1. Forward pass Feed the input through the network, computing and caching every intermediate value (pre-activations z and activations a). Without a prediction there is no loss and no learning.
2. Compute the loss Compare the prediction with the label.
3. Backward pass Starting from ∂L/∂L = 1, walk the computation graph in reverse. At each node multiply the incoming (upstream) gradient by the node's local derivative and pass it to the node's inputs. Gradients arriving from multiple paths are summed.
4. Update After all gradients are known, the optimizer updates every weight in all layers together. Gradients are computed output-first, but the updates happen simultaneously in one step.

Backprop and the optimizer are different things: backprop only computes gradients; the optimizer (SGD, Adam...) uses them to update weights. Both happen only during training, never during validation or inference.

Why backprop is fast

Computing each gradient separately by nudging one weight and re-running the network (finite differences) costs one forward pass per weight: O(W) passes for W weights, hopeless for a billion parameters. Backprop (reverse-mode automatic differentiation) gets all W gradients for roughly the cost of two to three forward passes. The price is memory: every intermediate activation from the forward pass must be stored for the backward pass, which is why activation memory often dominates training memory.

Worked example: one full training step by hand

A tiny network: one input, one ReLU hidden unit, one sigmoid output, binary cross-entropy loss. Input x = 2.0, label y = 1. Parameters w₁ = 0.3, b₁ = 0.1, w₂ = 0.5, b₂ = 0.2, learning rate η = 0.1.

x=2.0 --(w1=0.3, b1=0.1)--> z1=0.7 --ReLU--> h=0.7 --(w2=0.5, b2=0.2)--> z2=0.55 --sigmoid--> y_hat=0.634
                                                                                        |
                                                                          L = -ln(0.634) = 0.456

All gradients were negative, so every parameter increased, pushing ŷ toward the target 1. Notice that the size of each weight's update is proportional to the activation feeding into it (w₁ got a bigger gradient than b₁ because x = 2): connections that contributed more to the error receive more "blame". If the hidden unit had been a sigmoid instead of a ReLU, the backward pass would have multiplied by σ'(z₁) ≤ 0.25, which is the seed of the vanishing-gradient problem.

Autograd does this for you

import torch

x, y = torch.tensor(2.0), torch.tensor(1.0)
w1 = torch.tensor(0.3, requires_grad=True); b1 = torch.tensor(0.1, requires_grad=True)
w2 = torch.tensor(0.5, requires_grad=True); b2 = torch.tensor(0.2, requires_grad=True)

h = torch.relu(w1 * x + b1)
logit = w2 * h + b2
loss = torch.nn.functional.binary_cross_entropy_with_logits(logit, y)
loss.backward()                           # fills .grad on every leaf tensor

print(round(loss.item(), 3))              # 0.456
print(w1.grad, b1.grad, w2.grad, b2.grad) # -0.366, -0.183, -0.256, -0.366

with torch.no_grad():                     # manual SGD step
    for p in (w1, b1, w2, b2):
        p -= 0.1 * p.grad
        p.grad = None                     # gradients accumulate unless cleared

Local minima, saddle points and "good enough"

Neural network losses are non-convex, so there is no guarantee of reaching the global minimum. In high dimensions, most critical points are saddle points (downhill in some directions, uphill in others) rather than bad local minima, and research suggests that most local minima of large networks have loss close to the global minimum. Mini-batch noise and momentum help escape saddles. Success is judged not by the lowest training loss but by low, stable validation loss. Convex problems like linear regression have a single global minimum; deep learning aims for robust generalization, not exact optimality. Flat minima (wide valleys) tend to generalize better than sharp ones.

Gradient checking

When you implement a custom layer or backward function, compare the analytic gradient with a numerical one: (L(w + ε) − L(w − ε)) / 2ε with ε ≈ 1e−5, in double precision. A relative error below about 1e−6 means the gradient is correct. PyTorch provides torch.autograd.gradcheck.

Optimizers: SGD, momentum, RMSProp, Adam, AdamW

Plain gradient descent uses one global learning rate and only the current gradient. That causes three problems: it zig-zags across narrow valleys (steep in one direction, shallow in another), it crawls across flat regions and saddle points, and it treats rarely-updated parameters the same as frequently-updated ones. Modern optimizers fix these by remembering past gradients (momentum) and by scaling the step per parameter (adaptive learning rates).

The update rules

Why Adam needs bias correction

m and v start at zero, so in the first steps they are biased toward zero (after step 1, m = 0.1g with β₁ = 0.9). Dividing by (1 − β^t) rescales them so early estimates are unbiased; the correction fades as t grows. Without it, the first updates would be badly scaled, especially the second-moment estimate with β₂ = 0.999.

Adam versus AdamW: the weight-decay subtlety

With plain SGD, adding an L2 penalty (λ/2)||θ||² to the loss is exactly the same as weight decay. With Adam, it is not: the L2 gradient λθ gets divided by √v̂ like every other gradient, so weights with large historical gradients are barely regularized. AdamW decouples the decay, shrinking every weight by the same fraction each step. This gives more predictable regularization and better generalization, which is why AdamW is the standard for Transformers. Biases and normalization parameters are usually excluded from weight decay.

Which optimizer should you use?

Other names you may hear: LAMB/LARS (layer-wise adaptive rates for very large batch sizes), Adafactor (factorized second moments to save memory), Lion (sign-based update, less memory), Sophia and Shampoo (approximate second-order information). Second-order methods such as Newton's method use curvature (the Hessian) but are too expensive for millions of parameters.

import torch

model = torch.nn.Linear(10, 2)
sgd   = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9, nesterov=True, weight_decay=5e-4)
adam  = torch.optim.Adam(model.parameters(), lr=3e-4)
adamw = torch.optim.AdamW(model.parameters(), lr=3e-4, betas=(0.9, 0.999), weight_decay=0.01)

# exclude biases and norm parameters from weight decay (common for Transformers)
decay, no_decay = [], []
for name, p in model.named_parameters():
    (no_decay if p.ndim == 1 else decay).append(p)
opt = torch.optim.AdamW([{"params": decay, "weight_decay": 0.01},
                         {"params": no_decay, "weight_decay": 0.0}], lr=3e-4)

Learning rate, schedules and warmup

The learning rate controls step size. Too high: the loss oscillates, spikes or becomes NaN. Too low: training is painfully slow and may settle in a poor region. The best value also changes during training: early on you want big steps to make fast progress, later small steps to settle into a minimum. A schedule changes the learning rate over time.

Why warmup exists

At step 0 the weights are random, the gradients can be large and poorly aligned, and Adam's second-moment estimates are based on very few samples, so they are noisy. A full learning rate at that moment can push the model into a bad region or cause immediate divergence (loss spikes to NaN). Warming up linearly over a few hundred to a few thousand steps lets the statistics stabilize. Warmup matters most for Transformers, large batch sizes and adaptive optimizers.

Finding a good learning rate

1. LR range test Train for a few hundred steps while increasing the learning rate exponentially from 1e−7 to 1. Plot loss against learning rate (log scale).
2. Pick a value about 10x below the point where the loss is falling fastest or just before it blows up.
3. Search on a log scale If tuning by hand, try 1e−4, 3e−4, 1e−3, 3e−3 rather than evenly spaced values.
4. Scale with batch size The linear scaling rule: when you multiply the batch size by k, multiply the SGD learning rate by roughly k (with warmup). For Adam, square-root scaling is a common heuristic.
5. Fine-tuning Use a learning rate 10-100x smaller than for training from scratch (for example 1e−5 to 5e−5 for BERT-style models), so pretrained features are not destroyed.

import math, torch

opt = torch.optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.01)
total_steps, warmup = 10_000, 500

def lr_lambda(step):
    if step < warmup:
        return step / warmup
    progress = (step - warmup) / max(1, total_steps - warmup)
    return 0.5 * (1 + math.cos(math.pi * progress))   # multiplier on base lr

sched = torch.optim.lr_scheduler.LambdaLR(opt, lr_lambda)
# in the loop: loss.backward(); opt.step(); sched.step(); opt.zero_grad()

# built-in alternatives
# torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=total_steps)
# torch.optim.lr_scheduler.OneCycleLR(opt, max_lr=1e-3, total_steps=total_steps)
# torch.optim.lr_scheduler.ReduceLROnPlateau(opt, factor=0.1, patience=3)  # call .step(val_loss)

Weight initialization

Before training, weights must be set to something. The choice matters more than it seems: bad initialization makes activations and gradients shrink or grow exponentially with depth, so the network either learns nothing or blows up on the first step.

Why not all zeros (or all the same value)?

If every weight in a layer starts equal, every neuron in that layer computes the same output, receives the same gradient and gets the same update. They stay identical forever: a layer of 512 units behaves like one unit. This is the symmetry problem. Random initialization breaks the symmetry so different neurons learn different features. Biases can safely start at zero because the random weights already break symmetry.

Why not just "small random numbers"?

The variance of a neuron's output is roughly fan_in × Var(w) × Var(x). If Var(w) is too small, the signal shrinks by a constant factor each layer and vanishes; too large and it explodes. The fix is to choose Var(w) so that the variance of activations (forward) and gradients (backward) stays roughly constant from layer to layer.

Special cases worth knowing: initialize the LSTM forget-gate bias to 1 so the cell remembers by default early in training; initialize the last BatchNorm γ in each residual block to 0 so each block starts as an identity function; initialize a classifier's final bias to the log of class priors on imbalanced data so the initial loss is sensible.

import torch.nn as nn

def init_weights(m):
    if isinstance(m, (nn.Linear, nn.Conv2d)):
        nn.init.kaiming_normal_(m.weight, nonlinearity="relu")
        if m.bias is not None:
            nn.init.zeros_(m.bias)

model.apply(init_weights)   # PyTorch layers already use a He-uniform variant by default

Overfitting, underfitting and regularization

The goal is not to fit the training data; it is to perform well on data the model has never seen (generalization). Deep nets have enough capacity to memorize random labels perfectly, so controlling overfitting is a core skill.

Diagnosing with training and validation loss

Example: training loss 0.05 with validation loss 0.30 signals overfitting; 0.10 versus 0.12 is a good fit. There are no universal numeric thresholds; the gap and the trend matter. The expected test error decomposes as bias² + variance + irreducible noise. Deep learning adds a twist called double descent: past the point where a model can exactly fit the training set, test error can go down again as models grow, which is why very large, well-regularized models generalize well.

loss
  |\                                     validation loss
  | \                        ___....----''
  |  \                 _.-''
  |   \            _.-'         <- gap grows: overfitting
  |    '.___...--''   best checkpoint (early stopping here)
  |         '--..__        ^
  |                ''--..__|___ training loss
  +------------------------------------------------ epochs

The regularization toolbox

How dropout works, precisely

Why it helps: no neuron can rely on a specific partner being present, so each learns features that are useful on their own; the network behaves like an ensemble of exponentially many sub-networks sharing weights. Where it is placed: after activations in fully connected layers, on embeddings and attention weights in Transformers (usually 0.1), and between (not inside) recurrent steps of RNNs. It is used less in modern CNNs, which rely on BatchNorm and augmentation. Monte Carlo dropout keeps dropout on at inference and averages several passes to estimate uncertainty.

import torch.nn as nn

model = nn.Sequential(
    nn.Linear(128, 64), nn.ReLU(), nn.Dropout(p=0.5),
    nn.Linear(64, 10)
)
model.train()   # dropout active, BatchNorm uses batch statistics
model.eval()    # dropout off, BatchNorm uses running statistics

# augmentation for images (torchvision)
from torchvision import transforms as T
train_tf = T.Compose([T.RandomResizedCrop(224), T.RandomHorizontalFlip(),
                      T.ColorJitter(0.2, 0.2, 0.2), T.ToTensor(),
                      T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
val_tf = T.Compose([T.Resize(256), T.CenterCrop(224), T.ToTensor(),
                    T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

Train, validation and test splits

Training data updates the weights. Validation data chooses hyperparameters, checkpoints and early stopping, and never updates weights. The test set is touched once at the end to report an unbiased estimate. Tuning on the test set leaks information and inflates results. For time series, split by time (never shuffle future into past); for users or patients, split by entity so the same person does not appear on both sides.

Normalization: BatchNorm, LayerNorm, RMSNorm

As weights change during training, the distribution of each layer's inputs keeps shifting and can drift toward huge or tiny values. Normalization layers re-standardize activations inside the network, which makes optimization much smoother, allows higher learning rates, and reduces sensitivity to initialization.

Tensor of activations: N (batch) x C (channels) x H*W (spatial)

 BatchNorm              LayerNorm              InstanceNorm           GroupNorm
 normalize over         normalize over         normalize over         normalize over
 N, H, W  per C         C, H, W per N          H, W per (N, C)        H, W and a group
                                                                      of channels per N
   C ->                    C ->                   C ->                   C ->
 N [#|  |  ]            N [########]           N [#|  |  ]            N [###|   ]
 | [#|  |  ]            | [        ]           | [ |  |  ]            | [   |   ]
 v [#|  |  ]            v [        ]           v [ |  |  ]            v [   |   ]

BatchNorm: training versus inference

During training, BatchNorm normalizes with the current mini-batch's mean and variance and updates those exponential running averages. During inference it uses the running averages, so a single example gets a deterministic output. This is why model.eval() matters: forgetting it at inference makes predictions depend on whatever else is in the batch; forgetting model.train() during training freezes the statistics. Fine-tuning with tiny batches often works better with BatchNorm layers frozen in eval mode.

Why normalization works

The original explanation was reducing "internal covariate shift" (changing input distributions to each layer). Later research found the main effect is a smoother loss landscape: gradients become more predictable, so larger learning rates are safe. Normalization also decouples the scale of weights from the function computed, and BatchNorm's batch noise acts as a mild regularizer. Rule of thumb: BatchNorm for CNNs, LayerNorm or RMSNorm for Transformers and RNNs, GroupNorm when batches are tiny.

Where to place it: pre-norm versus post-norm

In CNNs the classic order is Conv → BatchNorm → ReLU (the conv bias becomes redundant because BatchNorm subtracts the mean, so set bias=False). In Transformers, the original design applied LayerNorm after the residual addition (post-norm), which needs careful warmup. Modern models use pre-norm (x + Sublayer(LayerNorm(x))), which keeps a clean identity path through the residual stream and trains stably at great depth.

import torch.nn as nn

conv_block = nn.Sequential(
    nn.Conv2d(64, 128, kernel_size=3, padding=1, bias=False),  # bias redundant before BN
    nn.BatchNorm2d(128),
    nn.ReLU(inplace=True),
)

class PreNormBlock(nn.Module):              # Transformer-style residual sublayer
    def __init__(self, d, hidden):
        super().__init__()
        self.norm = nn.LayerNorm(d)
        self.ff = nn.Sequential(nn.Linear(d, hidden), nn.GELU(), nn.Linear(hidden, d))
    def forward(self, x):                   # x: (batch, seq, d)
        return x + self.ff(self.norm(x))

Vanishing and exploding gradients, and residual connections

Backpropagation multiplies local derivatives layer by layer. In a network of depth N, the gradient reaching the first layer is roughly a product of N factors (weight matrices times activation derivatives). If those factors are typically below 1, the product shrinks exponentially (vanishing gradients): early layers stop learning. If they are above 1, it grows exponentially (exploding gradients): updates become huge, the loss spikes, and weights overflow to NaN.

The arithmetic

The sigmoid's derivative is at most 0.25. Chaining 10 sigmoid layers multiplies the gradient by at most 0.25¹⁰ ≈ 9.5 × 10⁻⁷; 30 layers gives about 10⁻¹⁸. Conversely, a per-layer factor of 1.5 gives 1.5²⁰ ≈ 3,300 and 1.5⁵⁰ ≈ 6 × 10⁸. This is a big part of why neural networks were stuck at a few layers for two decades. RNNs suffer worst because the same recurrent weight matrix is multiplied at every time step: over 400 time steps, even a factor of 0.99 per step leaves 0.018 of the signal.

The fixes that made deep learning work

Residual connections

Residual networks were motivated by the degradation problem: making a plain network deeper (say 56 layers instead of 20) made even the training error worse, which is not overfitting but an optimization failure. A deeper network should be at least as good, because extra layers could learn the identity, but learning an identity through a stack of non-linear layers is hard. With a skip connection, the block only needs to learn the residual F(x) = H(x) − x, and "do nothing" is simply F = 0. This allowed networks of 152 and even 1,000 layers. Residual connections are now everywhere: every Transformer block wraps attention and the feed-forward network in them. If the shapes differ (channels or spatial size change), the shortcut uses a 1×1 convolution with stride to match.

          x ------------------------------+
          |                               |  identity shortcut
     [conv 3x3, BN, ReLU]                 |  (1x1 conv if shapes differ)
          |                               |
     [conv 3x3, BN]                       |
          |                               |
          +------------ (+) <-------------+
                         |
                       ReLU
                         |
                         y = ReLU(F(x) + x)

Gradient clipping

loss.backward()
total_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)  # returns the pre-clip norm
optimizer.step()
# log total_norm: persistent spikes point to a too-high LR or bad data

Norm clipping rescales the entire gradient vector when its L2 norm exceeds the threshold, preserving direction. Value clipping (clamping each element) distorts direction and is rarely preferred. Clipping is effectively mandatory for RNNs and standard for Transformers.

Convolutional neural networks (CNNs)

CNNs are designed for grid-structured data, where every value has a fixed position and well-defined neighbours: images (height × width × color channels), audio spectrograms, video frames, even some time series. An MLP on a 224×224 RGB image would need about 150,000 weights per hidden neuron and would treat a cat in the top-left corner as unrelated to the same cat in the bottom-right. CNNs build in two assumptions that fix both problems: locality (nearby pixels are related, so each unit looks only at a small patch) and translation equivariance (a pattern means the same thing wherever it appears, so the same filter is reused at every position).

The convolution operation

Worked example. A 5×5 grayscale image with a bright left region (value 10) and dark right region (0), and a 3×3 vertical-edge kernel:

Input (5x5)                Kernel (3x3)          Output (3x3), stride 1, no padding

10 10 10  0  0             1  0 -1                0  30  30
10 10 10  0  0             1  0 -1                0  30  30
10 10 10  0  0      *      1  0 -1       =        0  30  30
10 10 10  0  0
10 10 10  0  0

Top-left output: each row gives 10*1 + 10*0 + 10*(-1) = 0, so 0 in total (flat region).
Next position:   each row gives 10*1 + 10*0 + 0*(-1) = 10, three rows = 30 (edge found).

The output (a feature map) is large exactly where the edge is. Nobody designs this kernel in a CNN: all kernel values start random and are learned by backpropagation, just like MLP weights. Filters in early layers usually end up looking like edge and color detectors; the network discovers them because they reduce the loss.

Padding, stride and output size

• Padding adds a border (usually zeros) so the kernel can be centered on edge pixels. Without it, every convolution shrinks the map and border pixels are under-used. "Same" padding (P = (K − 1)/2 for odd K at stride 1) keeps the size; "valid" means no padding. Padding changes the output size, never the kernel size.
• Stride is how far the kernel jumps. Stride 2 halves the resolution (downsampling), captures coarser features and cuts computation, at the cost of fine detail.
• Dilation spaces out kernel taps to enlarge the receptive field without extra parameters (used in segmentation and audio models).

Channels and multiple filters

A color image has 3 input channels. Each filter spans all input channels (shape C_in × k × k) and produces one output channel by summing over them. A layer with C_out filters produces C_out feature maps stacked along depth, like RGB but with 32, 64 or 512 channels, each detecting a different pattern. In PyTorch, nn.Conv2d(16, 32, kernel_size=3, padding=1) means 16 input channels and 32 output channels (filters), not image sizes. Tensors are laid out as [batch, channels, height, width], for example [1, 3, 224, 224].

Pooling

Pooling summarizes small regions to shrink the feature maps. Max pooling keeps the strongest response in each window ("is this feature present somewhere here?"); average pooling keeps a smoother summary. A 2×2 window with stride 2 halves height and width. Example: the 4×4 map [[1,3,2,4],[5,6,1,2],[7,2,8,3],[4,5,9,1]] becomes [[6,4],[7,9]]. Pooling has no learnable parameters, reduces computation, and gives some tolerance to small shifts. The trade-off: it discards exact positions and fine detail, which is why segmentation and medical-imaging models use skip connections (U-Net) or fewer pooling steps, and many modern networks replace pooling with strided convolutions that learn their own downsampling. Global average pooling averages each channel over the whole map to one number, replacing large flatten-plus-dense heads and making the network work with any input size.

Receptive field

The receptive field of a unit is the region of the input that can influence it. Stacking n 3×3 convolutions at stride 1 gives a (2n + 1)×(2n + 1) receptive field: two layers see 5×5, three see 7×7. Two stacked 3×3 layers use 18 weights per channel pair versus 25 for one 5×5, and add an extra non-linearity; this is why small kernels dominate. Strides and pooling grow the receptive field multiplicatively, so deep layers see most of the image, which is how "semantic" layers can respond to whole objects.

The typical CNN pipeline

image [3x32x32]
   |  Conv 3x3 (32 filters) + BN + ReLU      -> [32x32x32]   edges, colors
   |  Conv 3x3 (32) + BN + ReLU, MaxPool 2   -> [32x16x16]
   |  Conv 3x3 (64) + BN + ReLU, MaxPool 2   -> [64x8x8]     textures, parts
   |  Conv 3x3 (128) + BN + ReLU, MaxPool 2  -> [128x4x4]    object parts
   |  Global average pool                    -> [128]        image embedding
   |  Linear 128 -> 10                       -> [10] logits  classes
   v  (softmax inside the loss during training)
 Spatial size shrinks, channel depth grows: less "where", more "what".

Flattening turns the final feature maps into a vector without losing information; each neuron of the following dense layer connects to every element. That vector (or the global-pooled one) is effectively an embedding of the image, task-specific unless the network was trained to produce general features. All parameters, convolution filters and dense weights alike, are trained together by one backward pass.

import torch
import torch.nn as nn

class SmallCNN(nn.Module):
    def __init__(self, n_classes=10):
        super().__init__()
        def block(cin, cout, pool=True):
            layers = [nn.Conv2d(cin, cout, 3, padding=1, bias=False),
                      nn.BatchNorm2d(cout), nn.ReLU(inplace=True)]
            if pool:
                layers.append(nn.MaxPool2d(2))
            return layers
        self.features = nn.Sequential(*block(3, 32, pool=False), *block(32, 32),
                                      *block(32, 64), *block(64, 128))
        self.head = nn.Sequential(nn.AdaptiveAvgPool2d(1), nn.Flatten(),
                                  nn.Dropout(0.2), nn.Linear(128, n_classes))

    def forward(self, x):                   # x: [B, 3, 32, 32]
        return self.head(self.features(x))  # [B, n_classes] raw logits

x = torch.randn(8, 3, 32, 32)
print(SmallCNN()(x).shape)                  # torch.Size([8, 10])

Invariance, equivariance and beyond images

Convolution is translation equivariant (shift the input, the feature map shifts the same way). Pooling and global averaging add approximate translation invariance (the final prediction barely changes). It is not perfect: strides and pooling cause aliasing, so small shifts can change outputs, and CNNs are not naturally invariant to rotation or scale, which is why augmentation is used. CNNs also work on 1D data (audio, sensor signals, text via 1D convolutions), 3D data (video as frames × height × width, volumetric medical scans), and video with CNN-plus-RNN or CNN-plus-Transformer hybrids that combine spatial features per frame with temporal modelling across frames.

Classic CNN architectures and transfer learning

The history of CNN architectures is a history of ideas that are still used everywhere: deeper stacks of small kernels, multi-branch blocks, 1×1 bottlenecks, residual shortcuts, and efficient depthwise convolutions. Knowing why each design appeared is more valuable in interviews than memorizing layer counts.

Key building blocks to explain

• 1×1 convolution: a per-pixel dense layer across channels. It mixes channel information and changes channel count cheaply, used as a bottleneck to reduce channels before an expensive 3×3 convolution.
• Depthwise separable convolution: split a standard convolution into a depthwise step (one k×k filter per input channel) and a pointwise 1×1 step. Remaining cost is about (1/C_out + 1/k²) of a dense convolution, roughly 8–9× cheaper for 3×3 kernels with wide output.
• Bottleneck block: 1×1 reduce → 3×3 → 1×1 expand, wrapped in a residual connection.
• Squeeze-and-excitation: global-pool each channel, pass through a tiny MLP, and rescale channels by the result: channel-wise attention.
• Compound scaling: scaling only depth or only width gives diminishing returns; scaling depth, width and resolution together is more efficient.

Beyond classification

The same backbones power other vision tasks: object detection (predict boxes and classes; two-stage R-CNN family versus single-stage YOLO/SSD/RetinaNet, which introduced focal loss), semantic segmentation (a class per pixel; encoder-decoder U-Net with skip connections, dilated convolutions), instance segmentation (Mask R-CNN), and pose estimation. For deployment on phones and embedded boards see Edge AI.

Transfer learning and fine-tuning

Features learned on a large dataset such as ImageNet transfer surprisingly well: early layers learn generic edges and textures, later layers more task-specific parts. Instead of training from scratch, start from pretrained weights and adapt them.

1. Replace the head Swap the final layer for one with your number of classes.
2. Match preprocessing Use the same input size and normalization statistics as pretraining.
3. Train the head first With the backbone frozen, a few epochs at a normal learning rate so random head gradients do not wreck pretrained features.
4. Unfreeze gradually Unfreeze deeper blocks with a 10-100x smaller learning rate; discriminative learning rates (smaller for earlier layers) help.
5. Watch BatchNorm With small batches, keep BatchNorm layers in eval mode (frozen statistics).

import torch.nn as nn
from torchvision import models

model = models.resnet50(weights=models.ResNet50_Weights.IMAGENET1K_V2)
for p in model.parameters():
    p.requires_grad = False                           # freeze backbone
model.fc = nn.Linear(model.fc.in_features, 5)         # new head, trainable by default

# later: unfreeze the last stage with a smaller learning rate
for p in model.layer4.parameters():
    p.requires_grad = True
optimizer = torch.optim.AdamW([
    {"params": model.fc.parameters(),     "lr": 1e-3},
    {"params": model.layer4.parameters(), "lr": 1e-4},
], weight_decay=1e-4)

Recurrent networks: RNN, LSTM, GRU

Sequences such as text, speech, sensor readings and prices have an order, variable length, and dependencies that can span long distances ("The keys that I left on the kitchen table yesterday are missing"). An MLP takes a fixed-size input and has no notion of order; a recurrent neural network processes one element per time step while carrying a hidden state, a vector that summarizes everything seen so far.

The vanilla RNN

Folded                          Unrolled through time (same weights at every step)

   +------+                     h0 -->[RNN]--h1-->[RNN]--h2-->[RNN]--h3-->[RNN]--h4
   |      v                             ^        |       ^        |      ^        |
 x -->[ RNN ]--> y                      x1       y1      x2       y2     x3       y3 ...
   ^      |
   +--h---+                     An RNN has one (or a few) layers, run across many time
                                steps; it is deep in time, not necessarily in layers.

Input-output patterns

For tagging, the output at step t conditions on inputs x₁..x_t (the current input included). For generation or forecasting, the model predicts step t from x₁..x_t−1, then feeds its own prediction back in.

Backpropagation through time (BPTT) and its problem

Training unrolls the network over the sequence, computes the loss (summed over time steps or at the end), and backpropagates through the unrolled graph. Weights are usually updated once per sequence or batch, not after every word. The number of backward steps equals the sequence length; truncated BPTT limits it to, say, 100 steps for long streams. Because the gradient flowing from step t back to step k contains the product of (t − k) Jacobians involving W_hh and tanh', it vanishes when the largest singular value of W_hh is below about 1 and explodes when it is above. In practice vanilla RNNs struggle to learn dependencies beyond 10-20 steps.

LSTM: long short-term memory

All gates are computed in parallel from the same inputs at each step; there is no "input first, forget second" ordering. The forget gate is not redundant with the input gate: the input gate only filters what enters now and has no control over what is already stored, so without a forget gate the memory would accumulate stale information forever. If the forget gate wrongly deletes something needed later, the loss increases and backpropagation adjusts its weights.

Why LSTMs help with vanishing gradients: the cell-state update is additive. The gradient of c_t with respect to c_t−1 is just f_t (element-wise), not a product with a weight matrix and a squashing derivative. When the forget gate stays near 1, gradients flow back many steps almost unchanged, the same idea as a residual connection but through time. It reduces, but does not eliminate, vanishing gradients.

Parameter count: an LSTM layer has 4 gate-sized weight blocks: 4 × (h × (h + x) + h). With input size 100 and hidden size 128: 4 × (128 × 228 + 128) = 117,248 parameters. (PyTorch keeps two bias vectors per gate, adding another 4h.)

GRU: gated recurrent unit

Bidirectional and stacked RNNs

A standard LSTM reads left to right, so the representation of a word knows only the past. A bidirectional RNN runs a second RNN right to left and concatenates both hidden states at each position, so each word sees context from both sides ("the film was not as bad as critics claimed"). Each direction still needs its own memory and gates. Bidirectional models suit tasks where the whole input is available (classification, tagging, the encoder of a translator) but not real-time generation, which cannot see the future. Stacked RNNs feed one layer's hidden-state sequence into another for more capacity; 2-4 layers with dropout between them is typical.

import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence

class SentimentLSTM(nn.Module):
    def __init__(self, vocab_size, emb_dim=128, hidden=128, n_classes=2, pad_idx=0):
        super().__init__()
        self.emb = nn.Embedding(vocab_size, emb_dim, padding_idx=pad_idx)
        self.lstm = nn.LSTM(emb_dim, hidden, num_layers=2, batch_first=True,
                            bidirectional=True, dropout=0.3)
        self.head = nn.Linear(2 * hidden, n_classes)

    def forward(self, tokens, lengths):              # tokens: [B, T] int ids
        x = self.emb(tokens)                         # [B, T, emb_dim]
        packed = pack_padded_sequence(x, lengths.cpu(), batch_first=True,
                                      enforce_sorted=False)   # skip padding
        _, (h_n, _) = self.lstm(packed)              # h_n: [layers*2, B, hidden]
        h = torch.cat([h_n[-2], h_n[-1]], dim=1)     # last layer, both directions
        return self.head(h)                          # [B, n_classes]

# in the training loop, always clip for RNNs:
# torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)

Case study: sentiment on long movie reviews

A useful real-world lesson comes from training several models on 50,000 movie reviews (balanced, about 230 words each, truncated to 400 tokens):

Takeaways: a strong classical baseline is essential; gradient clipping and shorter sequences are non-optional for RNNs on long text; pretrained embeddings matter enormously; and low training loss is necessary but not sufficient.

Where RNNs still make sense

Transformers dominate NLP because they parallelize across the sequence and capture long-range dependencies directly. RNNs remain useful for streaming and low-latency tasks (live speech recognition, keyword spotting, sensor processing on microcontrollers), where inputs arrive one step at a time and the model needs constant memory per step rather than a growing attention cache. Modern state-space models and linear-attention RNN variants revive this idea at scale. RNNs are architectures, not learning paradigms: they can be used supervised, self-supervised or unsupervised. Non-sequential data (customer age and salary) should not be forced into an RNN; sequences with missing values need imputation or masking first.

Sequence-to-sequence models and the attention idea

Translation and summarization map a sequence to another sequence of a different length. The encoder-decoder (seq2seq) design solves this with two RNNs: the encoder reads the input and compresses it into a vector; the decoder generates the output one token at a time, conditioned on that vector and on the tokens generated so far.

Encoder (reads source)                        Decoder (writes target)

 "the"  "cat"  "sat"                         <s>     "le"    "chat"
   |      |      |                            |       |        |
 [enc]->[enc]->[enc]--- context vector c --->[dec]-->[dec]-->[dec]--> ...
   h1     h2     h3                           |       |        |
                                             "le"   "chat"   "s'est"
 Bottleneck: the whole source squeezed into one fixed-size vector c.

 With attention: at every decoder step t
   score(s_t, h_i) for every source position i  ->  softmax  ->  weights a_ti
   context c_t = sum_i a_ti * h_i  (a different context for every output word)

Training and decoding

• Teacher forcing: during training the decoder receives the true previous token rather than its own prediction. It is fast and stable but creates exposure bias: at inference the model sees its own (possibly wrong) outputs, a distribution it never trained on. Scheduled sampling mixes the two.
• Decoding: greedy (pick the most likely token), beam search (keep the k best partial sequences), or sampling with temperature, top-k or top-p. These strategies carry over to modern language models; see Transformers & LLMs.
• Rare words: a fixed vocabulary maps unseen names and numbers to an unknown token, so the model cannot reproduce them. Pointer-generator networks add a copy mechanism that can copy source tokens directly; subword tokenization (BPE, WordPiece) largely solves the problem in modern systems.

Attention, in general form

Attention solved the bottleneck (the decoder can access every encoder state), gave direct short gradient paths between any output and any input position, and produced interpretable alignments (which source word each target word used). The next conceptual leap was to ask: if attention is this good, do we need recurrence at all? Self-attention lets every position of one sequence attend to every other position of the same sequence, in parallel, with no recurrence. Stacking self-attention with feed-forward layers, residual connections, LayerNorm and positional information gives the Transformer, the basis of BERT, GPT and every modern large language model.

Embeddings: learned dense representations

Neural networks need numbers. Categorical things (words, product IDs, users, tokens) have no natural numeric form. An embedding maps each item to a dense vector of, say, 64-4,096 real numbers, learned so that items that behave similarly end up close together in that vector space. Embeddings are how deep learning represents words, tokens, images, users and documents, and they power semantic search, recommendations and retrieval-augmented generation.

From one-hot to dense vectors

An embedding layer is mathematically a one-hot vector multiplied by a weight matrix E of shape (vocabulary size × dimension), which simply selects one row, so frameworks implement it as a table lookup. The rows start random and are trained by backpropagation like any other weights. The dimension (300 was a popular choice for word vectors, a few hundred to a few thousand for modern models) is a hyperparameter: too small cannot capture enough relationships, too large wastes compute and can overfit.

Distributional semantics and word2vec

"You shall know a word by the company it keeps": words appearing in similar contexts have similar meanings. Word2vec turns this into a self-supervised prediction task on raw text, with no labels needed.

• CBOW (continuous bag of words): predict the centre word from the surrounding context words within a window (order ignored). Faster; good for frequent words.
• Skip-gram: predict each context word from the centre word. Slower; better for rare words and fine relationships. You choose one of the two, not both.
• Every word takes a turn as the centre word as the window slides over the corpus.
• Negative sampling: instead of a softmax over the whole vocabulary, train a binary classifier to distinguish real (centre, context) pairs from random "negative" pairs; much cheaper.
• Window size: small windows (2-5) capture syntactic similarity; larger (5-10+) capture topical similarity. Tune it as a hyperparameter.

The trained input weight matrix is the embedding table. The famous result is that directions carry meaning: vec("king") − vec("man") + vec("woman") ≈ vec("queen"). GloVe learns similar vectors from global co-occurrence counts; fastText adds character n-grams so it can build vectors for misspelled or unseen words.

Static versus contextual embeddings

Word2vec gives one vector per word regardless of context, so "bank" in "river bank" and "bank account" share a vector: polysemy is not handled, and words not seen in training have no vector at all (out-of-vocabulary, which is a representation limit, not overfitting). Contextual embeddings compute a different vector for each occurrence based on the whole sentence. ELMo did this with deep bidirectional LSTMs (slower, heavier); BERT and GPT-style Transformers do it with self-attention, combining token embeddings with positional information. Chat assistants use exactly these contextual, Transformer-based dense representations over subword tokens.

Embeddings beyond words

• Images: the pooled output of a CNN or vision Transformer; trained contrastively with text (CLIP-style) so images and captions share one space.
• Sentences and documents: sentence-embedding models trained with contrastive losses; the backbone of semantic search and RAG.
• Users and items: recommender systems learn an embedding per user and per item; the dot product predicts affinity (matrix factorization is the linear special case).
• Categorical features in tabular models: embedding tables for high-cardinality columns (zip code, product ID) instead of huge one-hot vectors.

import torch, torch.nn as nn, torch.nn.functional as F

emb = nn.Embedding(num_embeddings=10_000, embedding_dim=300, padding_idx=0)
ids = torch.tensor([[12, 845, 3, 0, 0]])      # a padded sentence of token ids
vectors = emb(ids)                            # [1, 5, 300]

# load pretrained vectors and optionally freeze them
pretrained = torch.randn(10_000, 300)         # stand-in for word2vec/fastText
emb = nn.Embedding.from_pretrained(pretrained, freeze=True, padding_idx=0)

a, b = emb(torch.tensor(12)), emb(torch.tensor(845))
print(F.cosine_similarity(a, b, dim=0))

Generative models: autoencoders, VAEs, GANs and diffusion

Discriminative models learn p(label | input): "is this a cat?". Generative models learn the data distribution p(x) itself, so they can produce new samples: images, audio, molecules, text. Four families matter for interviews, and the modern language models on the Transformers page are a fifth (autoregressive models that generate one token at a time).

Autoencoders

An autoencoder has an encoder that compresses the input x into a low-dimensional latent code z, and a decoder that reconstructs x̂ from z. It is trained to minimize reconstruction loss (MSE or BCE) and needs no labels. The narrow bottleneck forces it to keep only the most important structure, like a non-linear PCA.

x (784 pixels) -> [Encoder: 784 -> 256 -> 32] -> z (32 numbers) -> [Decoder: 32 -> 256 -> 784] -> x_hat
                                   loss = || x - x_hat ||^2

• Uses: dimensionality reduction, pretraining, anomaly detection (train on normal data; high reconstruction error flags anomalies), and compression.
• Denoising autoencoder: corrupt the input and reconstruct the clean version, which learns robust features (a precursor of masked-prediction pretraining).
• Sparse autoencoder: penalize latent activations to learn interpretable features.
• Limitation for generation: a plain autoencoder's latent space has holes; decoding a random z gives garbage.

Variational autoencoders (VAEs)

A VAE makes the latent space smooth and sampleable. The encoder outputs the parameters of a distribution, a mean μ(x) and standard deviation σ(x), and a latent z is sampled from it. The loss adds a term pulling each encoded distribution toward a standard normal prior.

The reparameterization trick: sampling is not differentiable, so write z = μ + σ ⊙ ε with ε ~ N(0, I) drawn outside the network. Now gradients flow through μ and σ. VAEs give smooth interpolation and a principled likelihood, but their samples tend to be blurry. Their encoders are used as the compressor in latent diffusion: an image of 512×512×3 becomes a 64×64×4 latent before diffusion runs, which is much cheaper.

class VAE(nn.Module):
    def __init__(self, d_in=784, d_lat=16):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(d_in, 256), nn.ReLU())
        self.mu, self.logvar = nn.Linear(256, d_lat), nn.Linear(256, d_lat)
        self.dec = nn.Sequential(nn.Linear(d_lat, 256), nn.ReLU(), nn.Linear(256, d_in))

    def forward(self, x):
        h = self.enc(x)
        mu, logvar = self.mu(h), self.logvar(h)
        z = mu + torch.exp(0.5 * logvar) * torch.randn_like(mu)   # reparameterization
        return self.dec(z), mu, logvar

def vae_loss(x_logits, x, mu, logvar):
    rec = F.binary_cross_entropy_with_logits(x_logits, x, reduction="sum")
    kl = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return (rec + kl) / x.size(0)

Generative adversarial networks (GANs)

A generator G maps random noise z to a fake sample; a discriminator D outputs the probability that a sample is real. They are trained in alternation as a two-player minimax game:

GANs produce sharp samples and are fast at inference (one forward pass), which keeps them popular for super-resolution, face editing, style transfer and real-time applications. For general image generation, diffusion models have largely replaced them because they are more stable and diverse. Sample quality is measured with FID (distance between feature statistics of real and generated images; lower is better) and Inception Score.

Diffusion models (intuition)

Diffusion models learn to reverse a gradual noising process.

1. Forward process (fixed, no learning) Add a little Gaussian noise to an image over T steps (for example 1,000) until it is pure noise. A closed form lets you jump to any step directly: x_t = √ᾱ_t x₀ + √(1 − ᾱ_t) ε.
2. Training Pick a random image, a random step t and random noise ε; create x_t; train a network (a U-Net or a diffusion Transformer) to predict the noise ε from (x_t, t). The loss is plain MSE between true and predicted noise.
3. Sampling Start from pure noise and repeatedly subtract the predicted noise (plus a little fresh noise), step by step, until a clean image emerges. Faster samplers (DDIM and successors) need 20-50 steps; distilled and consistency models need 1-4.
4. Conditioning Feed a text embedding through cross-attention so the denoiser follows a prompt. Classifier-free guidance runs the model with and without the prompt and extrapolates toward the prompted prediction; a guidance scale of about 7 trades diversity for prompt faithfulness.
5. Latent diffusion Run the whole process in a VAE's compressed latent space instead of pixel space, which cuts compute by an order of magnitude or more.

Training at scale: batches, GPUs, mixed precision and parallelism

Modern models are limited by memory and throughput as much as by ideas. This section covers what uses GPU memory, how to fit bigger models and batches, and how training is spread across many GPUs.

Why GPUs

Neural network compute is dominated by dense matrix multiplications and convolutions, which decompose into millions of independent multiply-adds. CPUs have a few powerful cores optimized for sequential, branchy code; GPUs have thousands of simpler cores plus specialized tensor cores for low-precision matrix math and very high memory bandwidth. The practical rule: keep the GPU fed (fast data loading, large enough batches) and avoid moving tensors between CPU and GPU inside the loop.

What uses memory during training

Rule of thumb for mixed-precision Adam: about 16 bytes per parameter before activations (2 weights + 2 gradients + 4 master weights + 8 Adam state). A 1B-parameter model needs about 16 GB just for these, which explains why fine-tuning large models needs sharding or parameter-efficient methods such as LoRA.

Batch size

• Larger batches give more accurate gradient estimates and better hardware utilization, but each example contributes less and very large batches can generalize worse unless the learning rate is scaled and warmed up.
• Smaller batches add gradient noise, which regularizes and can help escape sharp minima, but underuse the hardware.
• BatchNorm needs reasonably large per-GPU batches; LayerNorm does not care.
• Practical approach: pick the largest batch that fits (or a known-good value), scale the learning rate accordingly, and use gradient accumulation to reach a target effective batch size.

Mixed precision

Automatic mixed precision runs matrix multiplications and convolutions in FP16/BF16 (roughly 2x faster, half the activation memory) while keeping numerically sensitive operations (softmax, normalization, loss, weight updates) in FP32. With FP16, loss scaling multiplies the loss by a large factor before backward so small gradients do not underflow, then unscales before the optimizer step, skipping steps whose gradients overflow to infinity.

scaler = torch.amp.GradScaler("cuda")         # only needed for float16
for x, y in loader:
    x, y = x.to(device, non_blocking=True), y.to(device, non_blocking=True)
    optimizer.zero_grad(set_to_none=True)
    with torch.autocast(device_type="cuda", dtype=torch.float16):
        loss = loss_fn(model(x), y)
    scaler.scale(loss).backward()
    scaler.unscale_(optimizer)                # so clipping sees true gradient values
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    scaler.step(optimizer)                    # skips the step if grads are inf/NaN
    scaler.update()
# with bfloat16: use dtype=torch.bfloat16 and drop the scaler entirely

Gradient accumulation

If a batch of 256 does not fit, run 8 micro-batches of 32, call backward on each (gradients add up in .grad), and step the optimizer once. Divide each micro-batch loss by the number of accumulation steps so the gradient equals the average over the effective batch. The math matches a true batch of 256, except for BatchNorm statistics, which still see only 32 examples.

accum = 8
optimizer.zero_grad(set_to_none=True)
for i, (x, y) in enumerate(loader):
    loss = loss_fn(model(x.to(device)), y.to(device)) / accum
    loss.backward()
    if (i + 1) % accum == 0:
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        optimizer.step()
        scheduler.step()
        optimizer.zero_grad(set_to_none=True)

Activation (gradient) checkpointing

Instead of storing every activation for the backward pass, store only some (for example at block boundaries) and recompute the rest during backward. This trades roughly 30% extra compute for a large activation-memory reduction, allowing longer sequences, bigger batches or deeper models (torch.utils.checkpoint).

Distributed training

Data parallel (DDP)                         Pipeline parallel (4 stages)

 GPU0: full model  <- batch slice 0         GPU0: layers 1-8   ---> activations
 GPU1: full model  <- batch slice 1         GPU1: layers 9-16  --->
 GPU2: full model  <- batch slice 2         GPU2: layers 17-24 --->
 GPU3: full model  <- batch slice 3         GPU3: layers 25-32 ---> loss
     \___ all-reduce gradients ___/         micro-batches keep all stages busy
 every GPU applies the same update

With DDP, use a DistributedSampler so each process sees a different shard of the data, and remember the effective batch size is per-GPU batch × number of GPUs (× accumulation steps). Throughput is often limited by the data pipeline (decoding images, tokenizing text); use multiple DataLoader workers, pinned memory and pre-processed data formats.

Debugging training: loss curves and a systematic checklist

Neural networks fail silently. Code with a bug often still runs, and the loss still goes down a bit, just not as far as it should. Experienced practitioners follow a routine: start simple, verify each piece, and change one thing at a time.

Step 1: overfit a tiny batch

Take one batch of 2-32 examples, turn off regularization (dropout, weight decay, augmentation) and train on it repeatedly. A correct model and pipeline should drive the loss to nearly zero and reach 100% accuracy on that batch within a few hundred steps. If it cannot, the problem is a bug, not a lack of data or capacity: wrong labels paired with inputs, a loss applied to the wrong tensors, a softmax applied twice, a detached graph, gradients not flowing to some layers, or a learning rate far off.

xb, yb = next(iter(train_loader))
xb, yb = xb[:16].to(device), yb[:16].to(device)
model.train()
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for step in range(300):
    opt.zero_grad()
    loss = loss_fn(model(xb), yb)
    loss.backward()
    opt.step()
    if step % 50 == 0:
        print(step, round(loss.item(), 4))   # should approach 0

Step 2: read the curves

Step 3: the full checklist

1. Look at the data Visualize inputs after preprocessing and augmentation, with their labels. Check shapes, dtypes, value ranges, class balance, duplicates between splits.
2. Establish baselines Majority class, a linear model, a published or pretrained model. Know what "good" looks like.
3. Check the initial loss About ln(K) for K balanced classes; much larger means bad initialization or output scaling.
4. Overfit one batch Proves the model, loss and optimizer are wired correctly.
5. Monitor gradients Per-layer gradient norms; zero means no flow (detached tensors, dead units, frozen layers); huge means explosion.
6. Monitor activations Fraction of dead ReLUs, saturated sigmoids, activation statistics per layer.
7. Check modes model.train() for training, model.eval() plus torch.no_grad() for evaluation.
8. Scale up gradually Add data, capacity and regularization one step at a time, and keep a log of experiments with fixed random seeds.

NaN hunting

• Check the data first: torch.isfinite(x).all() on inputs and targets.
• Use numerically stable losses (CrossEntropyLoss, BCEWithLogitsLoss), add ε inside logs, square roots and divisions.
• Lower the learning rate and add warmup; clip gradients; log the gradient norm to find the step where it spikes.
• In FP16, use a GradScaler or switch to BF16.
• torch.autograd.set_detect_anomaly(True) pinpoints the operation that produced a NaN in backward (slow; debug only).

A complete PyTorch training loop

Every PyTorch training loop, from a tiny classifier to large language model fine-tuning, follows the same five-beat rhythm per batch: zero the gradients, forward, loss, backward, step. The rest is bookkeeping: data loading, device placement, evaluation, schedules, mixed precision, early stopping and checkpointing. The example below trains the CNN from earlier on CIFAR-10 (32×32 color images, 10 classes) and is a solid template for most supervised tasks.

import math, time, torch, torch.nn as nn
from torch.utils.data import DataLoader, random_split
from torchvision import datasets, transforms as T

torch.manual_seed(42)
device = "cuda" if torch.cuda.is_available() else "cpu"

# ---------- data ----------
mean, std = (0.4914, 0.4822, 0.4465), (0.2470, 0.2435, 0.2616)
train_tf = T.Compose([T.RandomCrop(32, padding=4), T.RandomHorizontalFlip(),
                      T.ToTensor(), T.Normalize(mean, std)])
eval_tf = T.Compose([T.ToTensor(), T.Normalize(mean, std)])

full_train = datasets.CIFAR10("data", train=True, download=True, transform=train_tf)
train_set, val_idx = random_split(full_train, [45_000, 5_000],
                                  generator=torch.Generator().manual_seed(0))
val_set = torch.utils.data.Subset(
    datasets.CIFAR10("data", train=True, transform=eval_tf), val_idx.indices)  # no augmentation
test_set = datasets.CIFAR10("data", train=False, download=True, transform=eval_tf)

train_loader = DataLoader(train_set, batch_size=128, shuffle=True, num_workers=4,
                          pin_memory=True, drop_last=True)
val_loader = DataLoader(val_set, batch_size=256, num_workers=4, pin_memory=True)
test_loader = DataLoader(test_set, batch_size=256, num_workers=4, pin_memory=True)

# ---------- model, loss, optimizer, schedule ----------
model = SmallCNN(n_classes=10).to(device)          # defined in the CNN section
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)
optimizer = torch.optim.AdamW(model.parameters(), lr=3e-3, weight_decay=5e-4)

epochs = 30
total_steps = epochs * len(train_loader)
warmup = len(train_loader)                         # one epoch of warmup
def lr_lambda(step):
    if step < warmup:
        return (step + 1) / warmup
    return 0.5 * (1 + math.cos(math.pi * (step - warmup) / (total_steps - warmup)))
scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda)

use_amp = device == "cuda"
amp_dtype = torch.bfloat16 if use_amp and torch.cuda.is_bf16_supported() else torch.float16
scaler = torch.amp.GradScaler("cuda", enabled=use_amp and amp_dtype == torch.float16)

# ---------- one epoch of training ----------
def train_one_epoch():
    model.train()                                  # dropout on, BN uses batch stats
    total, correct, running = 0, 0, 0.0
    for x, y in train_loader:
        x, y = x.to(device, non_blocking=True), y.to(device, non_blocking=True)
        optimizer.zero_grad(set_to_none=True)      # 1. clear old gradients
        with torch.autocast(device_type="cuda", dtype=amp_dtype, enabled=use_amp):
            logits = model(x)                      # 2. forward
            loss = loss_fn(logits, y)              # 3. loss
        scaler.scale(loss).backward()              # 4. backward
        scaler.unscale_(optimizer)
        grad_norm = nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
        scaler.step(optimizer)                     # 5. update
        scaler.update()
        scheduler.step()                           # per-step schedule
        running += loss.item() * y.size(0)
        correct += (logits.argmax(1) == y).sum().item()
        total += y.size(0)
    return running / total, correct / total, grad_norm.item()

# ---------- evaluation ----------
@torch.no_grad()                                   # no graph, less memory, faster
def evaluate(loader):
    model.eval()                                   # dropout off, BN running stats
    total, correct, running = 0, 0, 0.0
    for x, y in loader:
        x, y = x.to(device), y.to(device)
        logits = model(x)
        running += nn.functional.cross_entropy(logits, y, reduction="sum").item()
        correct += (logits.argmax(1) == y).sum().item()
        total += y.size(0)
    return running / total, correct / total

# ---------- main loop with early stopping and checkpointing ----------
best_val, patience, bad_epochs = float("inf"), 5, 0
for epoch in range(epochs):
    t0 = time.time()
    tr_loss, tr_acc, gnorm = train_one_epoch()
    va_loss, va_acc = evaluate(val_loader)
    print(f"epoch {epoch:02d} | train {tr_loss:.3f}/{tr_acc:.3f} | "
          f"val {va_loss:.3f}/{va_acc:.3f} | lr {scheduler.get_last_lr()[0]:.2e} | "
          f"grad {gnorm:.2f} | {time.time() - t0:.0f}s")
    if va_loss < best_val - 1e-4:
        best_val, bad_epochs = va_loss, 0
        torch.save({"model": model.state_dict(), "optimizer": optimizer.state_dict(),
                    "scheduler": scheduler.state_dict(), "epoch": epoch}, "best.pt")
    else:
        bad_epochs += 1
        if bad_epochs >= patience:
            print("early stopping")
            break

# ---------- final test, once ----------
model.load_state_dict(torch.load("best.pt", map_location=device)["model"])
test_loss, test_acc = evaluate(test_loader)
print(f"test loss {test_loss:.3f}  test acc {test_acc:.3f}")

# ---------- inference on new data ----------
model.eval()
with torch.no_grad():
    probs = torch.softmax(model(x_new.to(device)), dim=1)   # probabilities only at inference
    pred = probs.argmax(1)

What each piece is for

Custom datasets and saving for deployment

from torch.utils.data import Dataset

class TabularDataset(Dataset):
    def __init__(self, X, y):
        self.X = torch.as_tensor(X, dtype=torch.float32)
        self.y = torch.as_tensor(y, dtype=torch.long)
    def __len__(self):
        return len(self.y)
    def __getitem__(self, i):
        return self.X[i], self.y[i]

# export for serving or mobile runtimes
model.eval()
example = torch.randn(1, 3, 32, 32, device=device)
torch.onnx.export(model, example, "model.onnx", input_names=["image"],
                  output_names=["logits"], dynamic_axes={"image": {0: "batch"}})

Keras (TensorFlow) wraps the same loop in model.compile(optimizer, loss, metrics) and model.fit(x, y, validation_data=..., callbacks=[EarlyStopping(monitor="val_loss", patience=5, restore_best_weights=True)]). The concepts are identical; PyTorch just makes the loop explicit. For deploying the trained model on phones and embedded hardware (quantization, runtimes, NPUs) see Edge AI.

Quick revision

• Deep learning is machine learning with multi-layer neural networks that learn their own features from raw data (representation learning).
• It took off because of large datasets, GPUs, and algorithmic fixes: ReLU, good initialization, normalization, residual connections, dropout and Adam.
• A neuron computes z = w·x + b and outputs f(z); weights live on connections, activations live in nodes.
• A single perceptron only separates linearly separable data; XOR needs a hidden layer.
• Without non-linear activations, any stack of layers collapses into one linear layer.
• The universal approximation theorem guarantees existence of a one-hidden-layer approximator, not that training finds it; depth is more parameter-efficient.
• Dense layer parameters = inputs × outputs + outputs.
• Sigmoid saturates and its derivative is at most 0.25; tanh is zero-centered; ReLU has derivative 1 for positive inputs but can die.
• GELU and SiLU are smooth ReLU-like activations used in Transformers and modern LLMs.
• Output activations follow the task: identity for regression, sigmoid for binary or multi-label, softmax for single-label multi-class.
• Loss is per example, cost is the average (plus regularization); libraries call both "loss".
• MSE penalizes large errors quadratically and is outlier-sensitive; MAE is robust; Huber mixes both.
• Cross-entropy is negative log-likelihood; confident wrong predictions are punished hardest (−ln 0.05 ≈ 3.0).
• Softmax plus cross-entropy has the gradient p − y with respect to logits.
• Feed raw logits to CrossEntropyLoss and BCEWithLogitsLoss; never softmax twice.
• Focal loss and class weights handle imbalance; label smoothing and temperature scaling improve calibration.
• Gradient descent: θ ← θ − η∇L; mini-batch SGD is the practical default.
• An epoch is one pass over the data; an iteration is one update on one mini-batch.
• Backpropagation is reverse-mode chain rule; it computes all gradients in about the cost of one or two extra forward passes but must store activations.
• Backprop computes gradients; the optimizer applies them; both happen only in training.
• Momentum smooths and accelerates; RMSProp adapts per-parameter step sizes; Adam combines both with bias correction.
• AdamW decouples weight decay from the adaptive update and is the default for Transformers.
• Learning rate is the most important hyperparameter; too high diverges, too low crawls.
• Warmup stabilizes early training; cosine decay is the common schedule; scale the learning rate with batch size.
• Zero or equal initialization leaves neurons symmetric; Xavier suits tanh/sigmoid, He (variance 2/fan_in) suits ReLU.
• Overfitting: low training loss, high validation loss; underfitting: both high.
• Regularizers: weight decay, dropout, early stopping, data augmentation, label smoothing, more data, transfer learning.
• Inverted dropout scales kept units by 1/(1 − p) in training and is disabled at inference.
• BatchNorm normalizes over the batch per channel and uses running statistics at inference; the forward pass uses biased batch variance, the running variance is updated unbiased; LayerNorm normalizes each example and suits Transformers; RMSNorm drops the mean.
• Label smoothing in PyTorch is y' = (1 − ε)y + ε/K, not (1 − ε) and ε/(K − 1).
• Vanishing or exploding gradients come from multiplying many factors below or above 1 through depth or time.
• Residual connections (y = F(x) + x) give gradients a direct path and solved the degradation problem. The residual stream is the vector every block reads and adds to.
• Pre-LN (x + F(Norm(x))) keeps a clean residual path and is the modern default; Post-LN (Norm(x + F(x))) needs warmup.
• SwiGLU is W₂(SiLU(W₁x) ⊙ W₃x); hidden width ≈ 8d/3 keeps the parameter count equal to a 4d MLP.
• Clip gradients by norm (typically 1.0) for RNNs and Transformers.
• CNNs exploit locality and parameter sharing; conv output size = ⌊(W + 2P − K)/S⌋ + 1.
• Conv parameters = (C_in × k × k + 1) × C_out, independent of image size. Dilated output size uses effective kernel 1 + D(K − 1).
• Pooling downsamples and adds shift tolerance but discards fine positional detail.
• Stacked 3×3 convolutions grow the receptive field cheaply; 1×1 convolutions mix channels.
• Architecture lineage: LeNet, AlexNet, VGG, Inception, ResNet, DenseNet, MobileNet, EfficientNet, ViT.
• Transfer learning: replace the head, freeze then gradually unfreeze, use a much smaller learning rate.
• RNNs share weights across time and carry a hidden state; BPTT unrolls them and suffers vanishing gradients.
• LSTM gates (forget, input, output) control an additively updated cell state; GRU merges them into update and reset gates.
• Bidirectional RNNs see both past and future context and cannot be used for streaming generation.
• Seq2seq compresses the source into a context vector; attention removes this bottleneck and leads to Transformers.
• Embeddings are learned dense vectors; word2vec (CBOW, skip-gram) learns them from context; contextual models give per-occurrence vectors.
• Autoencoders compress and reconstruct; VAEs add a KL term and the reparameterization trick to make the latent space sampleable.
• GANs train a generator against a discriminator and risk mode collapse; diffusion models learn to predict added noise and denoise step by step.
• Mixed-precision Adam needs about 16 bytes per parameter before activations; BF16 avoids FP16's loss scaling.
• Gradient accumulation simulates big batches; activation checkpointing trades compute for memory.
• DDP replicates the model and all-reduces gradients; FSDP/ZeRO shard states; tensor and pipeline parallelism split the model.
• Debug by overfitting one small batch, checking the initial loss (about ln K), and watching loss, learning rate and gradient norms.
• The PyTorch rhythm per batch: zero_grad, forward, loss, backward, step; use train/eval modes and no_grad for evaluation.

Glossary

Activation

The output value of a neuron after its activation function; also used for the function itself.

Adam

Adaptive optimizer combining momentum with per-parameter scaling by the running root-mean-square of gradients, plus bias correction.

AdamW

Adam with weight decay applied directly to weights instead of through the gradient.

Attention

A mechanism that computes a weighted average of values, with weights from query-key similarity, letting a model focus on relevant positions.

Autoencoder

Network trained to reconstruct its input through a low-dimensional bottleneck.

Autograd

Automatic differentiation: the framework records operations and computes gradients via the chain rule.

Backpropagation

Algorithm that computes the gradient of the loss with respect to every parameter by applying the chain rule from output to input.

Batch normalization

Normalizes each channel using mini-batch statistics, then applies learned scale and shift; uses running statistics at inference.

Batch size

Number of examples processed before one parameter update.

BF16

16-bit floating-point format with FP32's exponent range and reduced precision; preferred for mixed-precision training.

Bias (parameter)

Learnable offset added to a neuron's weighted sum, shifting its activation threshold.

Bidirectional RNN

Two RNNs reading a sequence in opposite directions, with states concatenated per position.

BPTT

Backpropagation through time: backprop on an RNN unrolled over the sequence.

Calibration

How well predicted probabilities match observed frequencies of being correct.

CBOW

Continuous bag of words: word2vec variant that predicts a centre word from its context words.

Cell state

The LSTM's long-term memory vector, updated additively under gate control.

Convolution

Sliding a small learnable kernel over an input and computing a dot product at each position.

Cosine similarity

Dot product of two vectors divided by the product of their lengths; measures directional similarity.

Cross-entropy

Loss equal to the negative log-probability assigned to the correct class; negative log-likelihood.

Data augmentation

Creating label-preserving variations of training examples to improve generalization.

Data parallelism

Each device holds a model copy and processes a different data slice; gradients are averaged across devices.

Depthwise separable convolution

A per-channel spatial convolution followed by a 1×1 convolution; far cheaper than a standard convolution.

Diffusion model

Generative model trained to reverse a gradual noising process by predicting the added noise.

Dropout

Regularizer that randomly zeroes activations during training and is disabled at inference.

Dying ReLU

A ReLU unit whose input is always negative, so it outputs zero and receives no gradient.

Early stopping

Halting training when validation performance stops improving, keeping the best checkpoint.

Embedding

A learned dense vector representing a discrete item such as a word, token, user or product.

Encoder-decoder

Architecture where an encoder builds a representation of the input and a decoder generates the output from it.

Epoch

One complete pass through the training dataset.

Exploding gradient

Gradients growing exponentially through depth or time, causing unstable updates or NaN.

Feature map

The 2D output of applying one convolutional filter across an input.

Fine-tuning

Continuing training of a pretrained model on a new task, usually with a small learning rate.

Focal loss

Cross-entropy scaled by (1 − p_t)^γ to focus training on hard, misclassified examples.

FSDP / ZeRO

Sharded data parallelism that splits parameters, gradients and optimizer state across devices.

GAN

Generative adversarial network: a generator and a discriminator trained against each other.

GELU

Smooth activation x·Φ(x), standard in Transformers.

Gradient

Vector of partial derivatives of the loss with respect to the parameters; points toward steepest increase.

Gradient accumulation

Summing gradients over several micro-batches before one optimizer step to simulate a larger batch.

Gradient clipping

Rescaling gradients whose norm exceeds a threshold to prevent explosions.

GRU

Gated recurrent unit: an RNN cell with update and reset gates and a single state.

He initialization

Random weights with variance 2/fan_in, designed for ReLU networks.

Hidden state

The vector an RNN carries between time steps, summarizing the sequence so far.

Hyperparameter

A setting chosen before training (learning rate, batch size, depth) rather than learned.

Label smoothing

Training toward softened targets y' = (1 − ε)y + ε/K instead of a hard one-hot, to reduce over-confidence.

Layer normalization

Normalizes across the features of each example independently of the batch.

Learning rate

Step size multiplying the gradient in each parameter update.

Logits

Raw, unnormalized scores output by a model before sigmoid or softmax.

LSTM

Long short-term memory: an RNN cell with forget, input and output gates and a cell state.

Mixed precision

Training with 16-bit arithmetic for speed and memory while keeping sensitive parts in FP32.

MLP

Multi-layer perceptron: stacked fully connected layers with non-linear activations.

Mode collapse

GAN failure where the generator produces only a few kinds of outputs.

Momentum

Optimizer technique that accumulates a moving average of gradients to accelerate and smooth updates.

Overfitting

Fitting training data (including noise) so closely that performance on new data suffers.

Padding

Adding border values (usually zeros) around an input so convolutions can cover edges and control output size.

Parameter sharing

Reusing the same weights at multiple positions or time steps, as in CNNs and RNNs.

Perceptron

A single neuron with a step activation; a linear binary classifier.

Pooling

Downsampling by taking the max or average over small windows.

Pre-LN / Post-LN

Where normalization sits relative to the residual add: inside the branch (Pre-LN, x + F(Norm(x))) or on the residual path (Post-LN, Norm(x + F(x))).

Receptive field

The region of the input that can influence a given unit.

ReLU

Rectified linear unit, max(0, x); the default hidden activation for CNNs and MLPs.

Reparameterization trick

Writing a sample as z = μ + σε so gradients can flow through a random sampling step.

Residual connection

A shortcut that adds a block's input to its output (y = F(x) + x).

Residual stream

The per-example (or per-token) vector that every residual block reads from and writes an additive update into.

RMSNorm

Normalization dividing by the root-mean-square of features without subtracting the mean.

RNN

Recurrent neural network: processes sequences step by step with a hidden state and shared weights.

Saddle point

A point with zero gradient that is a minimum in some directions and a maximum in others.

Seq2seq

Encoder-decoder model mapping an input sequence to an output sequence of possibly different length.

Sigmoid

Activation 1/(1 + e^−x) mapping to (0, 1); used for binary outputs and gates.

Skip-gram

Word2vec variant that predicts context words from a centre word.

Softmax

Function that turns a vector of logits into a probability distribution summing to 1.

Stride

Number of positions a convolution or pooling window moves at each step.

SwiGLU

Gated feed-forward activation W₂(SiLU(W₁x) ⊙ W₃x); hidden width is set near 8d/3 to keep the parameter count equal to a 4d MLP.

Teacher forcing

Feeding the true previous token to a decoder during training.

Transfer learning

Reusing a model trained on one task as the starting point for another.

Underfitting

A model too simple or undertrained to capture the patterns, with high training and validation error.

Universal approximation theorem

Result that a sufficiently wide one-hidden-layer network can approximate any continuous function on a bounded domain.

VAE

Variational autoencoder: an autoencoder with a probabilistic latent space regularized toward a standard normal.

Vanishing gradient

Gradients shrinking exponentially toward early layers or time steps, stalling their learning.

Warmup

Gradually increasing the learning rate at the start of training for stability.

Weight decay

Shrinking weights toward zero each step; equivalent to L2 regularization for plain SGD.

Word2vec

Self-supervised method that learns word embeddings by predicting words from context or context from words.

Xavier initialization

Random weights with variance 2/(fan_in + fan_out), designed for tanh and sigmoid networks.

Interview questions

model.train()
for x, y in loader:
    x, y = x.to(device), y.to(device)
    optimizer.zero_grad()          # clear accumulated gradients
    logits = model(x)              # forward pass (raw scores)
    loss = loss_fn(logits, y)      # e.g. CrossEntropyLoss on logits
    loss.backward()                # backprop: fill .grad for all parameters
    optimizer.step()               # update weights using the gradients