Drills: Fine-Tuning.
Five questions at a time from the whole volume, until a fine-tuning recipe reads like a checklist.
This plate draws rounds of five from the questions and predictions of the nine plates before it, with the reasoning and source plate after each answer.
The volume’s thread is that fine-tuning shapes behaviour that pre-training made possible: it teaches a format cheaply, learns whatever its examples say, right or wrong, and pays for each step in what it moves the model away from.
Below: a round of five. Answer each, read the explanation, and press “Next round” for another five.
- Explain what SFT changes and what it does not.
- Design fine-tuning data and loss.
- Count LoRA and QLoRA memory.
- Anticipate forgetting and distillation limits.
What to carry forward
A base model continues text; fine-tuning teaches it to answer, drawing on knowledge from pre-training (plate XVII.1), in a template it must be prompted with (plate XVII.2), with the loss on the answers (plate XVII.3). Model-written data needs a filter (plate XVII.4), and a few right examples beat many wrong ones (plate XVII.5).
LoRA trains a thin low-rank correction (plate XVII.6) and QLoRA puts it on a 4-bit base (plate XVII.7); every fine-tune forgets in proportion to how far it goes (plate XVII.8), and a distilled student inherits its teacher’s errors (plate XVII.9). Vol. XVIII turns from demonstrations to preferences.
Terms
- SFT
- Supervised fine-tuning on demonstrations.
- Loss masking
- Loss on the answer only.
- LoRA
- ΔW = BA beside frozen weights.
- Distillation
- Training on a teacher’s outputs.