AI as a Glass Box — Observe the Complete Run

Learning promise

After this lesson, you should be able to turn one AI interaction into a traceable run: reconstruct what entered the system, what context and tools it used, what configuration controlled it, what it returned, and which claims the evidence does not justify.

You do not need model weights, training data, source code, activations, gradients, or private chain-of-thought. You need observable events, stable identifiers, controlled experiments, and careful boundaries between evidence and inference.

Why a glass box belongs between black box and white box

A hosted model can be opaque internally while the surrounding application is highly observable. Conversely, owning an open-weight model does not automatically give you a good record of which prompt version, retrieved documents, tool results, or policy decisions produced a particular answer.

Glass-box rule: record the observable path, then make only the claims that path supports.

The observable run

flowchart LR
    A["User request"] --> B["Instruction and context assembly"]
    B --> C["Retrieval"]
    C --> D["Model call"]
    D --> E["Tool proposal"]
    E --> F["Authorization and execution"]
    F --> D
    D --> G["Validation"]
    G --> H["Final response"]

    B -. "trace" .-> I["Run record"]
    C -. "provenance" .-> I
    D -. "model config and usage" .-> I
    F -. "tool result" .-> I
    G -. "checks" .-> I
    H -. "outcome" .-> I

OpenTelemetry models a trace as connected spans and defines developing semantic conventions for generative-AI operations. W3C PROV supplies a stable model for describing entities, activities, agents, and their lineage. Together they provide useful vocabulary for a glass-box run record. S1

Three evidence windows

1. Before the model call: what was assembled?

Inspect the actual request sent to the model, not only the text typed by the user.

The important artifact is the assembled context. A polished answer cannot tell you afterward which hidden instruction, stale document, or missing message shaped it.

2. During the run: what path did the system take?

Represent each meaningful step as an event or span with a parent, start and end time, status, and sanitized attributes. A run may contain retrieval, reranking, several model calls, validation, human approval, and tools. OpenTelemetry’s GenAI conventions cover model and agent operations, but the specification warns that the conventions are still evolving. S1

For each tool call, separate:

1. What the model proposed.
2. What the application validated and authorized.
3. What code actually executed.
4. What result or error was returned to the model.
5. What user-visible effect was committed.

This distinction prevents a tool-shaped model output from being mistaken for a completed action.

3. After the call: what evidence supports the result?

Record the final response, finish reason, usage, latency, validation results, citations, human decisions, and user-visible outcome. When an answer cites sources, verify that each source exists and supports the nearby claim; a retrieval score or citation string does not prove correctness.

The record should let you answer:

• Which exact configuration and context produced this response?
• Which documents and tool results were available to the model?
• Did an action execute, fail, retry, or remain only a proposal?
• Which checks passed or failed?
• What changed between this run and another run?

Minimum glass-box run record

The representation can be JSON, database rows, or trace spans. The conceptual fields matter more than this exact schema.

{
  "run_id": "run-0187",
  "started_at": "2026-09-09T14:32:10Z",
  "task": "answer_with_sources",
  "versions": {
    "workflow": "sha256:...",
    "instructions": "v4",
    "model": "provider/model-version",
    "retrieval_index": "2026-09-08"
  },
  "input": {
    "content_ref": "protected://inputs/0187",
    "token_count": 842
  },
  "retrieval": [
    {"document_id": "doc-42", "version": "7", "chunk_id": "c3", "score": 0.81}
  ],
  "steps": [
    {"span_id": "s1", "type": "model", "status": "ok", "latency_ms": 920},
    {"span_id": "s2", "type": "tool", "name": "search", "status": "ok"}
  ],
  "validation": {
    "schema": "pass",
    "citation_support": "manual-review"
  },
  "output": {
    "content_ref": "protected://outputs/0187",
    "token_count": 311,
    "finish_reason": "stop"
  }
}

Use stable run, trace, span, document, tool-call, and effect identifiers so records can be connected without copying sensitive payloads into every log. W3C PROV’s entity/activity/agent relationships are useful when lineage must cross systems or organizations. S2

Observation is not explanation

A trace narrows hypotheses. A controlled experiment tests them.

Controlled glass-box experiments

Change one observable variable at a time and repeat trials when generation is stochastic.

Do not read causality from a single before/after example when random sampling, service changes, retries, or hidden provider behavior could also differ. Record what you controlled, what you could not control, and how many trials you ran.

Five-session agenda

Use 45–75 minutes per session. Any model interface is suitable if you can wrap it with your own recorder; optional provider telemetry can add evidence but is not required.

Hands-on lab: explain two different answers

1. Choose a harmless question with a short, authoritative source document.
2. Create a run ID and record the instruction version, model identifier, configuration, timestamps, and assembled context.
3. Run the request once with the source document and once without it. Keep every other visible setting fixed.
4. Record retrieval provenance, model usage, latency, final answer, citations, and validation results.
5. Compare the runs claim by claim. Label each difference as observed, inferred, or unresolved.
6. Repeat each condition at least three times if sampling is enabled.
7. Introduce one failed tool call and verify that the trace distinguishes proposal, attempted execution, error, and fallback.
8. Produce a one-paragraph incident summary using only the run record.
9. Remove or restrict raw payloads and confirm that the remaining metadata still supports your analysis.

The lab is complete when another person can reconstruct the observable path and reproduce the comparison without relying on your memory.

Privacy and security boundary

Observability creates another sensitive data system. Prompts, retrieved text, tool arguments, outputs, and even URLs may contain personal data, credentials, proprietary content, or adversarial text. NIST’s Generative AI Profile treats privacy, information security, confabulation, and related risks as lifecycle concerns rather than model-only problems. S3

Apply these defaults:

• Collect the minimum content needed for a stated purpose.
• Prefer protected payload storage plus references in general traces.
• Redact secrets and personal data before ordinary logging.
• Do not use an unsalted hash as protection for guessable sensitive text; use access-controlled storage or a keyed construction where matching is required.
• Separate production access from authorized debugging access.
• Set retention and deletion rules.
• Record policy and schema versions.
• Test the redaction path itself.
• Never treat hidden chain-of-thought as required telemetry; record observable decisions, tool proposals, validations, and outcomes.

Common misconceptions

• “Glass box means open weights.” Open weights belong to white-box access; a glass box observes the run boundary and system path.
• “A trace is the truth.” A trace is evidence produced by instrumentation, which can be incomplete, misconfigured, or wrong.
• “Logging the user prompt is enough.” The model may receive additional instructions, history, retrieved data, tool definitions, and prior results.
• “Retrieved means used.” Retrieval proves availability, not faithful influence.
• “Tool call means action.” A proposal, authorization, execution, and committed effect are different events.
• “More logging is always better.” Excess collection increases privacy, security, retention, and access risk. S3
• “One changed output explains the cause.” Stochastic generation and uncontrolled variables require repeated, bounded experiments.
• “The trace reveals the model’s thoughts.” It exposes recorded operations and artifacts, not a privileged transcript of internal cognition.

Exit check

You are ready for the white-box lesson when you can:

• Draw the observable path from user request to final response.
• Reconstruct one run using identifiers, versions, timestamps, and provenance.
• Distinguish model input from user input.
• Distinguish tool proposal, authorization, execution, result, and effect.
• Explain what a retrieval record and a citation do—and do not—prove.
• Design a one-variable experiment with repeated trials.
• Diagnose a difference between two runs without inventing hidden causes.
• Protect sensitive content while preserving enough evidence to debug.
• State which questions require white-box access instead.

Source map

Next

Continue to AI as a White Box to open the model itself: tensors, attention, logits, loss, gradients, optimization, and autoregressive generation.