AI Observability and Incident Response

Architecture

A telemetry and response architecture for tracing model, retrieval, tool, policy, quality, cost, and user-outcome failures.

The architecture separates intent, policy, state, execution, evidence, and response. Each boundary has one accountable owner and an observable contract. Probabilistic components may propose or classify; deterministic software owns permissions, budgets, state transitions, and release conditions.

Decisions

| Decision | Required evidence | Review trigger | |---|---|---| | Trace stable identifiers and versions without logging sensitive content by default. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Join service indicators with evaluation and outcome evidence. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Define containment and rollback before a model or prompt release. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change | | Preserve evidence before changing a failing system. | Versioned test, trace, or architecture record | Material workload, provider, policy, or data change |

Alternatives and trade-offs

A centralized control plane simplifies policy and evidence collection but can become a latency and availability dependency. Distributed enforcement reduces central bottlenecks but requires consistent identity, versions, and audit semantics. Prefer the smallest boundary that can enforce the invariant and prove the outcome.

Failure modes

• Prompts are logged indiscriminately.
• Provider status is treated as complete service health.
• Quality incidents have no severity or owner.
• Rollback destroys the evidence needed to explain the incident.

Operational checklist

• [ ] Trace IDs connect retrieval, model, tool, and outcome events.
• [ ] Sensitive fields are redacted or represented by hashes and labels.
• [ ] SLOs have error-budget response policies.
• [ ] Kill switches and fallbacks are exercised.
• [ ] Post-incident actions update tests and controls.

Connected practice

• Labs: /labs/cost-latency-reliability, /labs/evaluation-metric-comparison
• Projects: /projects/system-design-casebook, /projects/evaluation-control-system-project
• System breakdowns: /systems/inside-vllm

Sources

• otel-genai
• opentelemetry-traces
• sre-book
• nist-ai-rmf