Security and Privacy for LLM Systems

Mental model

An LLM is an untrusted probabilistic component processing both instructions and data. Security comes from system boundaries, least privilege, validation, and data governance, not from asking the model to behave.

Theory

Threats include direct and indirect prompt injection, sensitive information disclosure, insecure output handling, poisoned knowledge, excessive tool agency, denial of service, and supply-chain compromise. Treat external content as untrusted, isolate tenants, minimize retained data, authorize every action outside the model, sanitize rendering, and maintain audit and incident paths.

Alternatives and trade-offs

Provider filters can complement controls, sandboxing contains execution, allowlisted tools reduce authority, and human approval addresses consequential actions. No single layer is sufficient.

Failure modes and misconceptions

Prompt instructions are not access control, redaction after logging may be too late, RAG sources can contain attacks, and output schemas do not establish authorization.

Knowledge check

Decision scenario

A support agent retrieves only documents the signed-in user may access, strips active content, uses read-only scoped tools, and requires approval for account changes.

Learning outcomes

• Explain Security and Privacy for LLM Systems as a system mechanism rather than a slogan.
• Compare its alternatives, trade-offs, and production failure modes.
• Apply the concept to a decision and identify evidence that would validate it.

Trade-offs

Using Security and Privacy for LLM Systems can improve capability or control, but it also introduces cost, latency, complexity, and failure modes that must be measured against an explicit objective.