Model Selection, Cost, and Privacy

Key jargon

Term	Plain-language meaning
Capability	The kinds and difficulty of tasks a model can perform reliably.
Latency	Elapsed time from request to usable response.
Token cost	Usage charges or compute consumption associated with input and output tokens.
Data residency	The geographic or administrative location where data is stored or processed.

Key concepts

Choose a model against measured task requirements, not brand reputation or benchmark rank alone.
The cheapest model per token may be expensive overall if retries, failures, or oversized context increase total work.

Concept map

flowchart LR
    A["Define quality privacy latency budget"] --> B["Test candidate models"]
    B --> C["Measure total task cost"]
    C --> D["Route to best acceptable option"]

Selection dimensions

Task quality on your own examples.
Input/output modality and context needs.
Latency, concurrency, and availability.
Input, cached-input, output, tool, and hosting cost.
Data retention, training use, residency, and contractual controls.
Structured-output and tool-use reliability.
Model/version stability and deprecation policy.
Local hardware and operational burden.

Routing pattern

Use deterministic code for deterministic work; a small model for classification or simple transformation; a stronger model for ambiguous reasoning; and a human for decisions whose consequences exceed the evidence.

Privacy questions

What data leaves the device or organization?
Is it retained, logged, or used to improve a service?
Can administrators or tool providers access it?
Does the output reveal source data?
Can the task be completed using redacted, synthetic, or local data?

Exercise

Create a three-model bake-off using 20 representative tasks. Record quality, latency, tokens, estimated cost, and privacy constraints. Pick per task class rather than one universal winner.

Staleness warning

Never preserve prices, model aliases, limits, or benchmark rankings as timeless facts. Date every comparison.