Train your own model

What you'll see in this lesson

1. Before training: do you really need it?

   Most problems are solved with a good prompt, a few examples or RAG; fine-tuning a model is the last step, not the first.

2. LoRA and QLoRA: training just a thin layer

   LoRA freezes the model and trains a few small matrices; QLoRA does the same with the model in 4 bits and fits on a regular GPU.

3. Unsloth, Axolotl and TRL: which one to use

   All three are free and open source; they differ in convenience and in the project size they're designed for.

4. The data: how many examples and how good

   A hundred good examples already give results; over a thousand is ideal. Quality and variety matter more than quantity.

5. DPO: teaching through preferences

   With DPO you give the model pairs of answers, one chosen and one rejected, and it learns to resemble the good one.

6. GRPO: learning by reinforcement with rewards

   GRPO generates several answers to each question, scores them with a reward function and reinforces the best ones.

7. What it costs: your GPU or a rented one

   A small QLoRA run fits on a 16 GB GPU or costs a few dollars in the cloud; long reinforcement runs are the expensive part.

8. Evaluate before using it, then deploy

   Fine-tuning is only worth it if it beats the original model with a good prompt on your real cases. Measure it before going to production.

9. Summary: train only when it pays off

   Prompt and RAG first; if that's not enough, QLoRA with good data, preferences or reinforcement, and always evaluate against the original.