training methods

SFT (Supervised Fine-Tuning)

The first stage of instruction-tuning, where a base model is trained on human-written demonstrations of ideal responses.

Supervised Fine-Tuning (SFT) is the first and arguably most critical step in transforming a raw, pre-trained base model into a usable conversational assistant. While a base model is trained simply to predict the next token across billions of internet documents, it lacks the concept of 'answering a question' or 'following a command'. SFT solves this by training the model on tens of thousands of high-quality, human-written prompt-and-response pairs.

During SFT, the model learns the structural format of a conversation (e.g., User: [Prompt], Assistant: [Response]). More importantly, it learns the expected tone, formatting, and helpfulness that humans desire. If the SFT dataset contains polite, well-structured, and perfectly factual answers, the model adopts that persona. If the SFT dataset is sloppy or toxic, the model's behavior degrades proportionally.

Because SFT requires human experts to write the ideal response from scratch rather than just comparing two pre-generated options, it is significantly more expensive and time-consuming per data point than preference labeling. However, a high-quality SFT phase is essential; subsequent reinforcement learning steps (like RLHF or DPO) can only optimize behaviors that the model already learned during SFT. You cannot reinforce a skill the model never acquired.

What this means for trainers

When you are asked to 'write the ideal response' to a prompt rather than comparing two existing responses, you are doing SFT work. This is the hardest and highest-paying tier of AI training. Your exact words, formatting, and tone will be absorbed directly into the model's weights, serving as the gold standard for how the AI should behave in future interactions.