Fine-tuning: from raw data to the best checkpoint

Fine-tuning: from raw data to the best checkpoint A data-flow diagram generated by Archify. 01 / Data 02 / Representation 03 / Model 04 / Optimization 05 / Evaluation Train split · text + labels · 01 / Data · the model learns here Train split text + labels the model learns here Data split · train / val / test · 01 / Data · prevents leakage Data split train / val / test prevents leakage input_ids + mask · [batch, seq_len] · 02 / Representation · model takes numbers only input_ids + mask [batch, seq_len] model takes numbers only Collate + padding · pad within each batch · 02 / Representation · drives cost Collate + padding pad within each batch drives cost Encoder + head · hidden state → logits · 03 / Model · forward pass Encoder + head hidden state → logits forward pass Freeze strategy · head / partial / full · 03 / Model · chosen before training Freeze strategy head / partial / full chosen before training Cross-entropy · loss → gradient · 04 / Optimization · backward() Cross-entropy loss → gradient backward() AdamW + scheduler · per-step learning rate · 04 / Optimization · optimizer.step() AdamW + scheduler per-step learning rate optimizer.step() Validation · metric + learning curve · 05 / Evaluation · no weight updates Validation metric + learning curve no weight updates Best checkpoint · by validation metric · 05 / Evaluation · not the last epoch Best checkpoint by validation metric not the last epoch only train is learned data tokenize encoding group into batches batching forward compute which layers get gradients config logits vs labels objective gradient backward end of epoch evaluation save on improvement checkpoint Legend primary data policy / PII async batch data store data flow