Fine-tuning a Pretrained Model: From Raw Data to the Training Loop

The first part opened up a Transformer from the inside: tokens, attention, residuals, the FFN, the three architecture families. The second part went from the outside in: pipeline(), the tokenizer, task heads, then the KV cache and the engines that serve inference.

Both of those posts are about a model that already knows something. This one is about changing what it knows.

Fine-tuning is the most familiar technique when working with modern language models. Instead of training from scratch on billions of samples, you start from a model that has been pretrained on a very large dataset, then keep training it on a smaller set that is close to your real problem.

The idea sounds simple. But to actually do it, you need to understand a chain of mechanisms that plug into each other:

Data
  ↓
Tokenizer
  ↓
Tensor
  ↓
Batch
  ↓
Model
  ↓
Logits
  ↓
Loss
  ↓
Backpropagation
  ↓
Gradient
  ↓
Optimizer
  ↓
New weights

This post walks the whole chain, with the focus on supervised fine-tuning for a classification task on a BERT-style Transformer encoder. Most of the principles here carry over to many other architectures and tasks.

The questions this post answers:

• Where exactly does fine-tuning differ from training from scratch, and why does that difference matter so much?
• Why is the quality of the data split part of the quality of the model, not a side chore?
• How do attention_mask and padding solve two different problems?
• Why is the classification head always randomly initialized, and what does that mean for the learning rate?
• What does cross-entropy see that accuracy does not?
• Where do gradients come from, and what does AdamW do differently from plain gradient descent?
• Why is forgetting zero_grad() a silent bug?
• Looking at a learning curve, how do you know whether to raise the learning rate or stop early?

Versions: the snippets in this post are excerpts from a PyTorch training loop, not standalone scripts: they assume a model, optimizer and data loaders already exist. The APIs they use were checked against Python 3.12, torch 2.14.1 and transformers 5.18.0, October 2026.

01What does fine-tuning actually change?

A pretrained model is a model that has been trained on a large amount of data to learn general patterns: relationships between words, syntax, semantics, context, and the recurring structures of language.

Very large corpus
    ↓
Pretraining
    ↓
Pretrained model
    ↓
General language representations

When you need to solve a specific task - sentiment classification, spam detection, intent classification, paraphrase detection, natural language inference - you do not need to retrain the whole thing from scratch:

Pretrained model
      +
Task dataset
      ↓
Fine-tuning
      ↓
Specialized model

1.1 The difference lies in where the parameters start

This is the crux, and it comes down to a single difference:

In the second column, the model does not start from a “knows nothing” state. Its inner layers already carry many useful representations.

That leads to a practical consequence this post will come back to again and again: the biggest risk in fine-tuning is not learning too slowly, it is learning too hard. A learning rate that is too high, training for too long, or a dataset that is too small can all wipe out the most valuable knowledge pretraining left behind.

1.2 Three levels of fine-tuning

Fine-tuning does not mean “retrain the entire model”. There are three common levels, which differ in which layers are allowed to receive gradients:

The intuition behind this ordering: the lower layers of a Transformer learn very general features, while the upper layers are closer to the task. The smaller the dataset, the more you should keep the general part frozen and only adjust the part close to the task.

Section 12 comes back to choosing a level, once we have enough tools to measure.

02The four layers of a fine-tuning pipeline

Let’s pick one concrete task to follow throughout the post: paraphrase detection, i.e. deciding whether two sentences mean the same thing.

Sentence A:
"The company released its quarterly report."

Sentence B:
"The firm's quarterly results were published."

Label:
0 → not equivalent
1 → equivalent

A model does not learn directly from character strings. It learns from numeric tensors. So a real training pipeline always has at least four layers:

This split is not just a way to organize knowledge. It is a diagnostic tool: when something goes wrong, the first question should be “which layer does this symptom belong to?” before you swap the model or tweak hyperparameters.

What this diagram adds over a sequential list: the bottom row is not an accessory. Every box in the bottom row is a decision you make before training starts, and each of those decisions constrains everything in the top row. Split the data wrong and no number downstream means anything. Pick the wrong learning rate and even the best backbone gets wrecked.

03The data pipeline and data leakage

A strong model cannot make up for a weak data pipeline.

In supervised learning, a dataset is usually split into three parts, and those three parts have three roles that are not interchangeable:

The validation set answers questions the training loss cannot:

• does the model generalize?
• is the learning rate appropriate?
• has it started to overfit?
• which checkpoint is best?
• should training stop?

The test set is a different animal. It is only worth anything if it has never influenced a single decision. If you look at the test score after every hyperparameter change and pick the configuration by it, you are “learning” from the test set through your own brain. The final number is then no longer an unbiased estimate - it is just a second validation score dressed up under another name.

3.1 Data leakage: dangerous, and very easy to commit

Data leakage happens when information from validation or test accidentally flows into training. The simplest example:

Train:
"This product is excellent"

Validation:
"This product is excellent"

The same sample sits in two splits. The model only has to memorize, not understand, and the validation score will look great.

Common forms of leakage:

What all five have in common: they push the validation score up, not down. That is exactly why they are dangerous. A bug that makes the score worse gets caught right away; a bug that makes the score look better usually gets celebrated.

So: split quality is part of model quality, not a cleanup step before the real work.

04Representing input for the encoder

A neural network does not understand character strings directly. It needs a sequence of numbers:

"Transformers are powerful."
        ↓
[101, 19081, 2024, 3928, 102]

A modern tokenizer does three things: splits text into tokens or subwords, maps tokens to integer IDs, and adds the necessary special tokens.

4.1 Why subwords

If every word were its own token, the vocabulary would explode, and the model would have to learn a separate representation for every variant:

run
runs
running
runner
runners

Subword tokenization breaks them into shared pieces:

running → run + ##ning

In return you get: a smaller vocabulary, better handling of rare words, better handling of never-seen words, and shared representations between words with the same root.

4.2 Special tokens and the structure of a sentence pair

With a BERT-style encoder, a single sentence is laid out as:

[CLS] sentence [SEP]

And a sentence pair - exactly our paraphrase detection task:

[CLS] sentence A [SEP] sentence B [SEP]

The role of each token:

4.3 Input IDs are just indices

After tokenization:

[CLS] hello world [SEP]
        ↓
[101, 7592, 2088, 102]

These numbers are the input_ids. The important point: the model does not treat them as arithmetic quantities. Token 2000 is not “twice as big” as token 1000. An ID is just a row index into the embedding table:

E = W_embed[id]

Concretely:

token id = 7592
       ↓
embedding table lookup
       ↓
[0.12, -0.34, 0.08, ..., 0.91]

4.4 The attention mask does not go through the embedding

When padding is added:

[CLS] hello world [SEP] [PAD] [PAD]

the model needs to know which tokens are real data. The attention mask records exactly that:

1 1 1 1 0 0

1 → real token
0 → padding

But it does not work by “removing tokens from the input”. The mask is applied to the attention scores, right before the softmax. Recall the formula from the first part:

S = (Q K^T) / sqrt(d_k)

The mask adds a matrix M to it:

S_masked = S + M

M = 0    at valid positions
M = -inf at masked positions

After the softmax:

softmax(-inf) ≈ 0

so [PAD] contributes almost nothing to attention.

This is a very easy place to get confused, so let’s be blunt: padding and the attention mask solve two different problems. Padding gives the tensor the right shape. The attention mask gives the computation the right meaning. With padding but no mask, the code still runs, the shapes are still right, and self-attention still looks at the empty tokens.

4.5 Segments: which sentence a token belongs to

When the model receives two sentences at once, it needs to know where the boundary is:

[CLS] sentence A [SEP] sentence B [SEP]
  0     0    0     0     1    1     1

This information becomes the segment embedding. The final representation of a token is the sum of three components:

x_i = E_token(i) + E_position(i) + E_segment(i)

The token embedding encodes what the token is, the position embedding encodes where it sits, and the segment embedding encodes which sentence it belongs to. Not every architecture uses segment embeddings, but they are the cleanest way to understand how an encoder handles a sentence pair.

05Padding, batching and the real cost

GPUs are efficient when they process many samples at once. Instead of feeding samples to the model one by one, we group them into a batch:

[sample 1, sample 2, sample 3]
             ↓
           model

But sentences usually have different lengths, while a batch tensor has to be a rectangle:

[batch_size, sequence_length]

So we have to pad.

5.1 Global padding and dynamic padding

Global padding pads every sequence to a fixed length:

max_length = 512

A 40-token sentence becomes 40 real tokens plus 472 empty ones. Easy to implement, but wasteful.

Dynamic padding only pads up to the longest sample in that particular batch:

Batch 1: 32, 45, 39, 50   → pad to 50
Batch 2: 71, 63, 69       → pad to 71

The total number of fake tokens to process drops considerably.

5.2 Why padding still costs resources even with a mask

This question is worth pausing on. The attention mask stops [PAD] from contributing meaning, but it does not make the tensor any smaller.

With a batch of 32, a sequence length of 512 and a hidden size of 768, the activation tensor is roughly:

32 × 512 × 768

If real sequences average only 100 tokens, most of that tensor exists purely because of padding - and it still takes up GPU memory and still goes through the matrix multiplications.

Worse, self-attention has a complexity of roughly:

O(n^2)

in the sequence length n. Going from 128 to 512:

512^2 / 128^2 = 16

The attention part has to handle 16 times as many token pairs. That is why “just set max_length=512 to be safe” is one of the most expensive decisions you can make, and also one of the easiest to fall into.

5.3 The collate function

In a data loader, the collate function is the step that groups individual samples into a batch tensor:

individual samples
      ↓
   collate
      ↓
one batch tensor

It typically handles: padding sequences, stacking tensors, building the attention mask, handling labels, converting dtypes.

The practical rule: defer padding until the batch is built. If you pad while tokenizing the whole dataset, you have locked yourself into global padding. If you pad in the collate step, you get dynamic padding almost for free.

06From hidden state to logits

A pretrained encoder produces a representation for every token:

Input tokens
     ↓
Transformer encoder
     ↓
Hidden states

If the sequence has length n and the hidden dimension is d:

H has shape [n, d]

To classify the whole sequence, we need to turn that entire tensor into a single representative vector. The common approach with a BERT-style encoder is to take the hidden state of the first token:

h_cls has shape [d]

Then pass it through a linear layer:

z = W · h_cls + b

where W has shape [C, d] with C the number of classes, and z are the logits. For binary classification:

hidden vector (768 dims)
        ↓
   Linear(768 → 2)
        ↓
[logit_class_0, logit_class_1]

This linear layer is called the classification head.

6.1 Why the head is always randomly initialized

The pretrained backbone has learned general representations. But a new task has a new label space:

Task A: positive / negative                   → 2 outputs
Task B: entailment / neutral / contradiction  → 3 outputs

The output dimensions differ, so there are no weights to reuse. The head has to be created fresh:

Pretrained backbone
        ↓
Randomly initialized head

This has a very practical consequence. During the first training steps the head is still random, so the gradients flowing back into the backbone are very noisy. If the learning rate is large from the very first step, those noisy gradients can damage exactly the representations we want to keep. This is why the warmup technique in section 9 exists.

Fine-tuning therefore does two things at once: it learns a head for the new task, and it adjusts the backbone so its representations fit that task better.

6.2 Logits are not yet probabilities

Suppose the output is:

logits = [-1.2, 2.8]

Softmax turns them into a probability distribution:

p_i = exp(z_i) / sum_j exp(z_j)

Concretely:

logits:  [-1.2, 2.8]
softmax: [0.018, 0.982]
argmax → class 1

But during training, the loss function usually takes raw logits; you do not apply softmax yourself first. The reason is that the loss implementation fuses log-softmax and negative log likelihood into a single operation that is more numerically stable. Applying softmax yourself and then taking the log is the fastest way to hit a NaN once a probability gets close to 0.

07Cross-entropy and how a model knows it is wrong

For multi-class classification, the usual loss is cross-entropy. If the correct class is y:

L = -log(p_y)

A few values are enough to see its character:

Cross-entropy does not only care about right or wrong. It cares about how confident the model is. Being wrong and confident is punished very hard.

This is also why the loss can keep falling while accuracy has not moved.

7.1 Loss and accuracy measure two different things

Suppose the correct label is class 1.

Model A: P(class 1) = 0.40 → predicts class 0
Model B: P(class 1) = 0.49 → predicts class 0

Both are wrong, so both have an accuracy of 0. But model B is much closer to the decision boundary, and cross-entropy sees that:

Loss is a continuous quantity; accuracy is a discrete metric. A small change in the weights almost always changes the loss, but is usually not enough to flip a prediction.

That is exactly the division of labor: loss is the objective you optimize, accuracy is the metric you report. Gradient descent needs a smooth surface to slide down, and accuracy cannot provide one.

7.2 When accuracy lies

Accuracy has one more weakness: it collapses when classes are imbalanced.

95% negative
5% positive

A model that always predicts negative hits 95% accuracy while being completely useless. So you need three more metrics:

Precision = TP / (TP + FP)
Recall    = TP / (TP + FN)
F1        = 2 · (Precision · Recall) / (Precision + Recall)

08Backpropagation and the optimizer

A training step starts with the forward pass:

Input → Embedding → Transformer layers → Classifier → Logits → Loss

During the forward pass, the framework quietly builds a computation graph: every operation records its mathematical relationship to the operation before it.

Take the simplest possible example:

z = w · x
L = (z - y)^2

We want to know how the loss changes when w changes. The chain rule gives:

dL/dw = (dL/dz) · (dz/dw)

In a real Transformer the graph has millions of operations, but the principle is the same. backward() runs automatic differentiation to compute the gradient for every trainable parameter.

8.1 What the gradient tells the optimizer

The gradient ∇L points in the direction in which the loss increases fastest. To reduce the loss, you go the other way:

θ ← θ - η · ∇L

where θ are the parameters and η is the learning rate. That is all there is to gradient descent.

8.2 What AdamW does on top of that

Plain gradient descent works, but deep learning usually uses adaptive optimizers. Adam keeps two running statistics over time:

m_t = β1 · m_{t-1} + (1 - β1) · g_t        (moving average of the gradient)
v_t = β2 · v_{t-1} + (1 - β2) · g_t^2      (moving average of the squared gradient)

After bias correction, the update is scaled by:

m_hat / (sqrt(v_hat) + ε)

What this means: every parameter gets its own effective learning rate, adapted to the history of its own gradients. A parameter whose gradient swings wildly takes shorter steps; a parameter with a stable gradient takes longer ones.

This also explains an important detail in section 10: the optimizer has state. m_t and v_t are what it accumulates over thousands of steps, and losing them is not like losing the weights - but it is not nothing either.

8.3 Where Adam and AdamW differ

Weight decay is a regularization technique that discourages the weights from growing large. The simple intuition:

L_total = L_task + λ · ||θ||^2

The old way was to add the L2 penalty straight into the gradient and let Adam handle that gradient. The problem: Adam’s adaptive mechanism divides the update by sqrt(v_hat), so it accidentally divides the weight-decay part too - and the actual amount of decay ends up differing between parameters in a way nobody intended.

AdamW performs decoupled weight decay, separating weight decay from the adaptive update:

θ ← θ - η · AdamUpdate - η · λ · θ

The decay is now applied uniformly to every parameter. This is why AdamW is almost always a sensible default when fine-tuning a Transformer.

8.4 Gradient clipping

Sometimes the gradient norm spikes, setting off a chain of failures:

large gradient → large update → loss spike → unstable training

Gradient clipping puts a ceiling on the gradient norm:

torch.nn.utils.clip_grad_norm_(
    model.parameters(),
    max_norm=1.0,
)

If ||g|| > max_norm, the whole gradient is scaled down - keeping its direction, only reducing its magnitude.

Clipping does not replace tuning the learning rate. It is a safety net for abnormal steps, not a fix for a wrong learning rate.

09Learning rate, batch size and the other knobs

The learning rate decides how big each update step is, and it is the hyperparameter with the strongest effect on fine-tuning.

A visual illustration:

Fine-tuning almost always uses a smaller learning rate than training from scratch. The reason goes back to section 1: we do not want to wreck the representations already learned. Somewhere between 2e-5 and 5e-5 is the familiar starting point for a BERT-sized encoder.

9.1 Schedulers and warmup

The learning rate does not have to be fixed. Two common schedules:

But before decaying, many setups first ramp up at the start - that is warmup:

Warmup solves exactly the problem raised in section 6.1. In the first steps:

• the classification head is still random, so the gradients are noisy;
• the optimizer’s moments m_t and v_t have not accumulated enough to give reliable estimates;
• an overly strong update at this point can damage the backbone before the head has learned anything.

Letting the learning rate climb from 0 over the first few hundred steps is the cheapest way to avoid that.

9.2 Batch size and effective batch size

Batch size is the number of samples used for one forward/backward pass. Each batch gives an estimate of the gradient:

g ≈ (1/B) · sum_{i=1..B} ∇L_i

9.3 Gradient accumulation

If your GPU can only fit a batch of 4 but you want an effective batch of 16, you do not need new hardware - you just delay optimizer.step():

micro batch 1 → backward
micro batch 2 → backward
micro batch 3 → backward
micro batch 4 → backward
                    ↓
            optimizer.step()
            optimizer.zero_grad()

effective batch = 4 × 4 = 16

This technique works precisely because PyTorch accumulates gradients by default - the very property that section 10 will show is the source of a silent bug when you are not using it on purpose.

One easy-to-miss detail: the loss needs to be divided by the number of accumulation steps, otherwise the gradient will be 4 times larger than expected and the effective learning rate will be off.

9.4 Mixed precision

Traditional training uses FP32 - each number takes 32 bits. Mixed precision moves most computation to FP16 or BF16, keeping a few critical steps at higher precision.

The benefits: less activation memory, less bandwidth, higher throughput on hardware with tensor cores.

The risk is in FP16: its dynamic range is much narrower than FP32’s, so very small gradients can underflow to 0 - vanishing entirely instead of just losing precision. The standard countermeasure is loss scaling:

L' = s · L

Backpropagating through L' produces gradients s times larger, enough to sit inside FP16’s representable range. Before the optimizer updates, the gradients are divided by s again.

BF16 has an exponent range closer to FP32’s, so it is usually more stable than FP16 and rarely needs loss scaling - if your hardware supports it.

10The complete training loop

A minimal PyTorch training loop:

model.train()

for batch in train_loader:
    input_ids = batch["input_ids"].to(device)
    attention_mask = batch["attention_mask"].to(device)
    labels = batch["labels"].to(device)

    logits = model(
        input_ids=input_ids,
        attention_mask=attention_mask,
    )

    loss = criterion(logits, labels)

    loss.backward()

    optimizer.step()
    scheduler.step()

    optimizer.zero_grad()

The thing worth noticing here is not the syntax but the order. Each call depends on the result of the one before it, and swapping the order breaks things in ways that do not always raise an error.

10.1 Why zero_grad() is required

In PyTorch, gradients accumulate into .grad by default. If you do not clear them after each batch:

gradient stored in .grad = g1 + g2 + g3 + ...

The model will update according to the sum of the gradients of every batch seen so far, with ever-growing magnitude. No exception is thrown. The loss just behaves strangely.

As section 9.3 said, this very property is what makes gradient accumulation possible. The only difference between “a useful technique” and “a bug” here is whether you meant to do it.

10.2 model.train(), model.eval() and dropout

A model can behave differently in training and evaluation. model.train() turns on training behavior; model.eval() switches modules like dropout into inference mode.

Dropout randomly zeroes out part of the activations during training:

Before: [0.2,  0.7, -0.1,  0.9, 0.4]
After:  [0.2,  0.0, -0.1,  0.0, 0.4]

The goal is to stop the model from relying too heavily on a particular group of features. At inference time, dropout is turned off.

Forgetting model.eval() during validation leads to: random-looking predictions, noisy validation metrics, and two evaluations on the same data giving two different results. Very hard to diagnose if you do not know where to look.

10.3 torch.no_grad() solves a different problem

model.eval()

with torch.no_grad():
    for batch in validation_loader:
        ...

Keep these clearly separate:

Two mechanisms, two problems. Evaluation needs both.

10.4 A fuller loop

Adding mixed precision, gradient clipping and evaluation:

scaler = torch.amp.GradScaler("cuda")   # mixed precision: FP16 on a CUDA GPU

for epoch in range(num_epochs):
    model.train()

    for batch in train_loader:
        input_ids = batch["input_ids"].to(device)
        attention_mask = batch["attention_mask"].to(device)
        labels = batch["labels"].to(device)

        optimizer.zero_grad()

        with torch.autocast("cuda", dtype=torch.float16):
            logits = model(
                input_ids=input_ids,
                attention_mask=attention_mask,
            )
            loss = criterion(logits, labels)

        scaler.scale(loss).backward()
        scaler.unscale_(optimizer)

        torch.nn.utils.clip_grad_norm_(
            model.parameters(),
            max_norm=1.0,
        )

        scaler.step(optimizer)
        scaler.update()
        scheduler.step()

    model.eval()

    all_predictions = []
    all_labels = []

    with torch.no_grad():
        for batch in validation_loader:
            input_ids = batch["input_ids"].to(device)
            attention_mask = batch["attention_mask"].to(device)

            logits = model(
                input_ids=input_ids,
                attention_mask=attention_mask,
            )

            all_predictions.append(logits.argmax(dim=-1).cpu())
            all_labels.append(batch["labels"])

    # Compute validation metrics
    # Save a checkpoint if it improved
    # Apply early stopping if needed

A detail worth noting: scaler.unscale_(optimizer) has to run before clipping. If you clip gradients that are still scaled up by a factor of s, max_norm=1.0 no longer means anything at all.

10.5 A checkpoint is more than weights

A complete checkpoint should contain:

model weights
optimizer state (m_t, v_t)
scheduler state
current epoch
current global step
random states

Why does optimizer state matter? As section 8.2 said, Adam accumulates m_t and v_t over thousands of steps. If you only load the weights and create a fresh optimizer, “resuming” really means restarting the optimization from a different point - both moments go back to 0, and the first steps after resuming behave like the first steps of a brand-new run.

A proper checkpoint lets you: resume when a job crashes, pick the best model, roll back, and compare different stages of training.

11Learning curves and diagnosis

Training running through all its epochs does not mean training succeeded. What you need to look at are the learning curves, and all four of them: training loss, validation loss, training metric, validation metric.

A healthy learning process looks like this:

Good signs: both fall together, the gap between them is reasonable and does not keep widening, the curves gradually flatten, and the metrics rise and then plateau.

11.1 Overfitting

Overfitting happens when the model keeps getting better at the training data while generalization gets worse:

Signs: training loss ↓ while validation loss ↑, or training accuracy rising sharply while validation accuracy plateaus and then drops.

Common causes: a model too large for the dataset, training too long, a small dataset, noisy labels, a narrow training distribution, weak regularization, or a bad split.

Ways to reduce it, ordered by how invasive they are:

11.2 Early stopping and the best checkpoint

Suppose the validation loss over the epochs is:

Epoch 1 → 0.54
Epoch 2 → 0.43
Epoch 3 → 0.37   ← best
Epoch 4 → 0.39
Epoch 5 → 0.42
Epoch 6 → 0.45

The best model is at epoch 3, not epoch 6. In other words:

Last checkpoint ≠ Best checkpoint

Early stopping usually has a patience parameter. With patience = 3, training stops when the validation metric has not improved for 3 consecutive evaluations - patient enough to ignore a single wobble, but without dragging on pointlessly.

What matters even more than stopping: save checkpoints by the validation metric, rather than taking the last model.

11.3 Underfitting

Underfitting is when the model cannot even learn the training data:

training loss   = high
validation loss = high
training metric   = low
validation metric = low

Possible causes: a learning rate that is too low, too few epochs, too little model capacity, too many frozen layers, wrong preprocessing, problematic labels, or input that simply does not contain enough information to solve the task.

The simplest differential diagnosis in this whole post:

Good training + Poor validation  → Overfitting
Poor training + Poor validation  → Underfitting

These two cases lead to two opposite sets of actions. Confusing them means turning the knob the wrong way: adding regularization to a model that is underfitting will only make it worse.

11.4 Oscillating curves

Some runs do not go down smoothly:

Possible causes: a learning rate that is too high, batches that are too small, gradient explosion, very noisy data, inconsistent preprocessing, unstable mixed precision, or an unsuitable scheduler.

What to try, in this order:

↓ learning rate
↑ batch size (or gradient accumulation)
enable gradient clipping
check the data and labels
check for NaN / Inf
revisit the scheduler

11.5 Converged does not mean good

A model is considered close to converged when training stops improving meaningfully:

0.80 → 0.55 → 0.41 → 0.35 → 0.33 → 0.329 → 0.328

But convergence only speaks about the optimization process, not about quality. An underfitting model still converges - it just converges at a high loss.

Optimization convergence ≠ Good generalization

Conclusions about quality always need a validation metric, never just a flat training loss.

12From a single run to a process

A single training run is rarely enough. This final section is about turning fine-tuning from a lucky one-off run into a repeatable process.

12.1 Unfreeze gradually, not all at once

Go back to the three strategies from section 1. What works well in practice is to climb up from the cheapest level, instead of starting with the heaviest configuration:

This approach is valuable in two ways. First, it avoids paying a high price when you do not need to. Second - and more importantly - it tells you which level actually produces the improvement. If you jump straight into full fine-tuning and the result is good, you do not know whether training just the head would have got you the same result at a fraction of the cost.

When to choose what, in short:

12.2 Experiment tracking

We usually try many configurations:

Run A: lr = 5e-5, batch = 16
Run B: lr = 2e-5, batch = 16
Run C: lr = 2e-5, batch = 32
Run D: lr = 2e-5, batch = 32, weight_decay = 0.01

Without a record, it is very hard to know which run was good and why. An experiment log should store at least:

model version          scheduler
dataset version        warmup ratio
random seed            weight decay
learning rate          number of epochs
batch size             validation metric
optimizer              best checkpoint
                       training time + hardware

The accompanying rule: change one thing at a time. If Run D changes both the batch size and the weight decay compared with Run B, a better result does not tell you which one helped.

12.3 Reproducibility

Deep learning has many sources of randomness: parameter initialization, data shuffling, dropout, augmentation, and even some GPU kernels.

import random
import numpy as np
import torch

seed = 42

random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)

if torch.cuda.is_available():
    torch.cuda.manual_seed_all(seed)

It is worth being honest about the limits: setting a seed reduces variance but does not eliminate it, because some GPU operations are inherently non-deterministic. So if the variance between runs is significant, report results across several runs rather than a single number.

12.4 A lookup table of common mistakes

Conclusion

Fine-tuning a pretrained Transformer is not calling a train() function. Behind it is a chain of tightly connected mechanisms, and this post has walked the whole chain: the data is split so nothing leaks, the tokenizer turns text into three number sequences, the collate step groups them into tensors, the encoder and head produce logits, cross-entropy turns the error into a single number, autograd turns that number into gradients, and AdamW turns the gradients into new weights.

But the thing most worth keeping is not that chain. It is the fact that there are two loops, not one:

Inner loop (every batch)    Outer loop (every evaluation)

forward                     evaluate
  → backward                  → diagnose
  → update                    → tune hyperparameters
                              → run again

The inner loop is what the code does. The outer loop is what you do, and that is where it gets decided how good the final model will be.

A common mistake is to think that the lower the loss, the better the model. Training loss only says how well the model fits the training data. The real goal is generalization - performing well on data it has never seen - and a good model has to balance four things at once: optimization, generalization, robustness and efficiency.

Once you understand how data flows through the pipeline, how a loss becomes gradients, how the optimizer updates weights, and what a learning curve reflects, training is no longer a black box. That is the foundation before going further into parameter-efficient fine-tuning, instruction tuning, multi-task learning, distributed training and preference optimization.

In this post, we took one step entirely for granted: the tokenizer turned text into input_ids, and we simply used the result. But that step decides more than it seems: how long the context window really is, how much each request costs, and why the same sentence takes more tokens in one language than in another. The next part of the series goes right back to that step: tokenization - normalization, pre-tokenization, subword segmentation, vocabulary and special tokens.

References

Papers

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018) - the encoder this post fine-tunes, including the [CLS]/[SEP] input format and segment embeddings.
• Universal Language Model Fine-tuning for Text Classification (2018) - ULMFiT, the origin of gradual unfreezing and discriminative learning rates.
• Adam: A Method for Stochastic Optimization (2014) - the Adam optimizer.
• Decoupled Weight Decay Regularization (2017) - AdamW and why weight decay should be decoupled from the gradient update.
• On the difficulty of training Recurrent Neural Networks (2012) - exploding gradients and gradient norm clipping.
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour (2017) - learning-rate warmup and scaling the learning rate with batch size.
• Mixed Precision Training (2017) - FP16 training with loss scaling.
• Dropout: A Simple Way to Prevent Neural Networks from Overfitting (2014) - dropout, and why model.train() and model.eval() behave differently.
• LoRA: Low-Rank Adaptation of Large Language Models (2021) - a starting point for the parameter-efficient fine-tuning mentioned in the conclusion.

Documentation

• Hugging Face LLM Course, chapter 3: Fine-tuning a pretrained model - the hands-on version of the same pipeline with datasets and transformers.
• Transformers: Fine-tuning - fine-tuning with Trainer and with a plain PyTorch loop.
• Transformers: Data Collator - DataCollatorWithPadding and dynamic padding.
• Transformers: Optimization - learning-rate schedules with warmup.
• PyTorch: AdamW and clip_grad_norm_ - the APIs used in sections 8 and 10.
• PyTorch: Automatic Mixed Precision - torch.autocast and GradScaler.
• PyTorch: Saving and loading models - checkpoints that include the optimizer state.
• PyTorch: Reproducibility - seeds and the sources of non-determinism that seeds cannot remove.