Fine-Tuning, PEFT, and Distillation

Key jargon

Term	Plain-language meaning
Fine-tuning	Continuing model training on task- or domain-specific examples.
Parameter-efficient fine-tuning (PEFT)	Adapting a model by training a small subset or added parameters.
LoRA	A PEFT method that learns low-rank weight updates.
Distillation	Training a smaller student model to reproduce selected behavior of a larger teacher.

Key concepts

Use tuning to change repeatable behavior after proving prompts and retrieval are insufficient.
Compare the tuned model with a baseline on held-out quality, safety, and regression datasets.

Concept map

flowchart LR
    A["Identify stable behavior gap"] --> B["Prepare train validation test data"]
    B --> C["Tune adapter or student"]
    C --> D["Evaluate and version artifact"]

Supervised fine-tuning: optimize on input/output examples.
Preference optimization: train behavior from ranked or comparative feedback.
PEFT/LoRA: update a small parameter subset or adapters.
Distillation: train a student to reproduce useful teacher behavior.
Continued pretraining: extend training on domain text; not the same as instruction tuning.

Before adapting

Establish a prompt/retrieval baseline, held-out task set, safety set, data rights, privacy process, contamination check, and rollback artifact. Adaptation can improve target behavior while degrading unrelated capabilities.

Exercise

Design a 100-example dataset for structured support-ticket classification. Specify splits, difficult negatives, provenance, label rubric, baseline, and acceptance thresholds. Do not train a model yet.

Decision test

If the requirement is current factual recall, record deletion, or access control, use a knowledge/data layer rather than attempting to encode records into weights.