Memory, RAG, CAG, and Fine-Tuning

Key jargon

Term	Plain-language meaning
Working memory	Short-lived state used during the current task or conversation.
Long-term memory	Selected information persisted for future interactions.
Cache-augmented generation (CAG)	Reusing preloaded or cached context rather than retrieving it for every request.
Fine-tuning	Updating model parameters using additional training examples.

Key concepts

Choose storage by change rate and purpose: state for current work, retrieval for attributable changing knowledge, cache for repeated context, and fine-tuning for learned behavior.
Persisting everything as memory creates privacy, relevance, and poisoning risks.

Concept map

flowchart LR
    A["Classify information need"] --> B["Choose state memory retrieval cache or tuning"]
    B --> C["Apply retention and trust rules"]
    C --> D["Evaluate behavior"]

Need	Prefer
Current authoritative facts	Retrieval/RAG
Small stable reference corpus reused frequently	Cached context/CAG where supported
User preference or workflow state	Explicit application memory/state store
Consistent style, format, or task behavior	Prompting first; fine-tuning if evals justify it
New domain vocabulary or behavior at scale	Task-specific fine-tuning with held-out evals
Deterministic business rule	Code/policy engine

Fine-tuning does not provide reliable record-level lookup, easy deletion, current citations, or permission filtering. RAG does not change the model’s underlying behavior. Memory should not silently turn every conversation into permanent user profiling.

Exercise

Classify ten requirements as prompt, retrieval, memory, fine-tuning, or deterministic code. Defend each choice using freshness, privacy, update, and evaluation needs.