Excessive Agency and Tool Risk

Key jargon

Term	Plain-language meaning
Excessive agency	Giving an AI system more functions, permissions, autonomy, or impact than its task requires.
Authorization	The decision that a specific identity may perform a specific action on a specific resource.
Confused deputy	A privileged component tricked into misusing its authority for another party.
Reversibility	How easily an action and its consequences can be undone.

Key concepts

Model intent is not authorization; every consequential tool request needs independent policy enforcement.
Reduce risk by limiting available tools, arguments, identities, duration, rate, scope, and transaction value.

Concept map

flowchart LR
    A["Model proposes action"] --> B["Bind user identity and scope"]
    B --> C["Authorize and confirm consequence"]
    C --> D["Execute minimally and record"]

Agency risk grows with tool breadth, credential privilege, data reach, action irreversibility, autonomy duration, environmental uncertainty, and weak supervision.

Reduce agency

Give task-specific tools rather than a shell or universal API.
Use short-lived, scoped credentials bound to user/tenant.
Separate read, propose, approve, and commit.
Constrain targets and arguments with policy.
Sandbox generated code and network access.
Set step, cost, and effect budgets.
Preserve idempotency and outcome reconciliation.
Provide an independent kill switch.

Exercise

Replace a hypothetical run_any_command tool with five narrow capabilities. Define identity, resource constraints, and approval for each.

Source

OWASP Top 10 for LLM Applications