Prompt Injection and Untrusted Content

Key jargon

Term	Plain-language meaning
Prompt injection	Content crafted to redirect model behavior away from the application's intended instructions.
Direct injection	Malicious instructions supplied directly through a user-controlled prompt.
Indirect injection	Malicious instructions embedded in content the system retrieves or processes.
Instruction-data separation	Treating external content as untrusted data while enforcing authority outside the model context.

Key concepts

A model cannot reliably make hostile content safe by being told to ignore it.
Limit consequences with least-privileged tools, validation, confirmation, provenance, and architectural isolation.

Concept map

flowchart LR
    A["Receive untrusted content"] --> B["Label and isolate as data"]
    B --> C["Model proposes bounded result"]
    C --> D["Policy blocks unauthorized effect"]

Prompt injection occurs when crafted input changes model behavior contrary to the application’s intended instruction hierarchy. Indirect injection arrives through documents, webpages, email, tool results, images, or retrieved content.

Because models process instructions and data through the same language channel, delimiter wording alone is not a complete defense.

Control strategy

Minimize tools and data available for the task.
Keep authorization outside model judgment.
Label and isolate untrusted content.
Avoid placing secrets in model context.
Validate tool arguments and outputs.
Require approval for consequential effects.
Test realistic injected content and monitor policy decisions.

Exercise

In a synthetic document, include text asking the assistant to ignore its task. Verify that a read-only summarizer reports document content without treating it as authority or calling tools.

Rule

The goal is not to prove the model can never be influenced; it is to ensure influenced output cannot cross a consequential trust boundary unchecked.