Reading learning curves and picking the right fix

Reading learning curves and picking the right fix A workflow diagram generated by Archify. 01 / Observe at each evaluation EX / Good train, poor validation EX / Both are poor EX / Curve oscillates wildly Look at both curves Name the symptom Fix, then re-measure Two loss curves · train and validation per step · Observe at each evaluation › Look at both curves Two loss curves train and validation per step Compare the curves · trend and gap · Observe at each evaluation › Look at both curves · ask before tuning Compare the curves trend and gap ask before tuning Healthy curve · both fall, then flatten · Observe at each evaluation › Fix, then re-measure Healthy curve both fall, then flatten Re-run · one change at a time · Observe at each evaluation › Fix, then re-measure · log the config Re-run one change at a time log the config Overfitting · train falls, validation rises · Good train, poor validation › Name the symptom Overfitting train falls, validation rises Reduce memorization · early stopping, weight decay · Good train, poor validation › Name the symptom Reduce memorization early stopping, weight decay Underfitting · both losses still high · Both are poor › Name the symptom Underfitting both losses still high Increase capacity · raise LR, more epochs, unfreeze · Both are poor › Name the symptom Increase capacity raise LR, more epochs, unfreeze Erratic · loss jumps between steps · Curve oscillates wildly › Name the symptom Erratic loss jumps between steps Stabilize updates · cut LR, bigger batch, clip grad · Curve oscillates wildly › Name the symptom Stabilize updates cut LR, bigger batch, clip grad the gap widens not learning yet no clear trend stop earlier learn harder steadier steps both curves fall keep best checkpoint Legend Agent logic Policy Cloud service External system