Fine-Tuning: SFT, LoRA/QLoRA, RLHF and DPO

Fine-tuning is the most over-reached-for and under-reached-for lever in AI engineering, often on the same team in the same quarter. Engineers who have not shipped an LLM feature before reach for it too early — "the model keeps getting the format wrong, let's fine-tune it" — before they have exhausted prompting, few-shot examples and structured-output constraints.

Engineers who have been burned reach for it too late, or not at all, and instead pile increasingly baroque instructions into a prompt to fight a behaviour problem that a few thousand labelled examples would fix cleanly and cheaply. Interviewers who ask about fine-tuning are almost never testing whether you can recite what LoRA stands for; they are testing whether you know when not to reach for it, and whether you can defend the decision with data, cost and an evaluation plan.

The core discipline this subject teaches is a single distinction, applied rigorously: fine-tuning changes behaviour — format, tone, style, domain dialect, tool-call conventions, refusal patterns, latency-driven model-size reduction. It does not reliably change knowledge. A model fine-tuned on your support transcripts will start sounding like your support team; it will not reliably learn that your refund window is 14 days unless "14 days" appears, verbatim and often enough, in the training data, and even then it cannot cite where that fact came from or update when the policy changes next month without a full retrain. That gap between "learned a pattern" and "learned a fact" is where most fine-tuning post-mortems live, and it is the gap this subject keeps returning to.

This subject un-compresses the "Fine-Tuning Options" section of llm-application-system-design into a full standalone treatment: the mechanics of each technique, what data each needs and where it comes from, a cost model for training and serving, an evaluation plan that catches regressions before they ship, and a worked scenario you can reuse as an interview answer shape. Read prompt-engineering first in this track — it is the "try this first" step for nearly every problem people bring to fine-tuning — and see model-training-and-experimentation for the infrastructure (experiment tracking, distributed training, checkpointing) that sits underneath everything described here.