The Fine-Tuning Trap
"Fine-tune the model on your data, and it will know your domain perfectly." Sounds logical. Sounds surgical. Except most of the time it isn't.
Companies spend weeks and thousands of dollars fine-tuning a model — only to find that a better prompt with better retrieval outperforms it in every way.
The trap: Fine-tuning feels like real engineering. It looks sophisticated. But it's often the expensive detour when the direct path was shorter.
What Fine-Tuning Actually Changes
Can do: Shift output style and tone. Teach domain-specific formatting. Improve consistency on specific task types.
Cannot do: Inject new factual knowledge reliably. Fix retrieval failures. Override hallucination patterns. Replace up-to-date information.
If your retrieval is broken, fine-tuning won't fix it.
Fine-tuning is the wrong answer when you:
- Want facts about your domain → Wrong. Use RAG.
- Want consistent formatting → Wrong. Use output parsing + prompts.
- Your prompt isn't working → Wrong. Iterate the prompt first.
- Think it will reduce hallucinations → Wrong. It may make them more confident and less detectable.
The litmus test: Before fine-tuning, try solving the problem with a $50 prompt engineering session and a smarter retrieval strategy.
The Surgical Rule
If you can't articulate exactly what weights you're changing and why — you shouldn't be fine-tuning. Fine-tune only after you've exhausted prompt engineering and retrieval optimization — and only when you can measure the ROI.