Generation, Decoding, and Hallucinations

#ai #decoding #hallucinations #reliability

Generation, Decoding, and Hallucinations

Key jargon

Term	Plain-language meaning
Logit	An unnormalized score the model assigns to a possible next token.
Temperature	A setting that reshapes the token probability distribution before sampling.
Top-p sampling	Sampling from the smallest high-probability token set whose cumulative mass reaches a threshold.
Hallucination	A fluent output that is false, fabricated, or unsupported by the available evidence.

Key concepts

Generation is an iterative next-token process; decoding policy changes which continuation is selected.
Grounding and verification reduce unsupported answers, but prompting alone cannot guarantee truth.

Concept map

flowchart LR
    A["Model emits logits"] --> B["Decoder forms probabilities"]
    B --> C["Select next token"]
    C --> D["Verify claims and repeat"]

Mental model

A language model produces a probability distribution over possible next tokens. A decoding strategy selects one, appends it, and repeats. Greedy decoding, temperature, top-p sampling, beam search, and constrained decoding trade diversity, determinism, and computational cost.

A hallucination is an unsupported or false output presented as though grounded. It is not a single bug with one universal fix. Causes include absent knowledge, ambiguous prompts, misleading context, retrieval failure, pressure to answer, or an evaluation that rewards style over evidence.

Reliability ladder

Ask whether generation is needed at all.
Supply relevant, authoritative context.
Require structured output when software will consume it.
Validate types, citations, calculations, and permissions outside the model.
Let the model abstain.
Evaluate on representative failures.
Require human approval for consequential decisions or effects.

Exercise

Ask the same factual question five times at two temperature settings. Compare wording variability with factual support. Then provide a short source passage and require sentence-level citations.

Misconception

Temperature zero improves repeatability but does not make unsupported statements true.