Learning outcomes

  • Separate instruction roles
  • Resolve conflicting guidance
  • Design message boundaries

Mental model

Messages are typed context segments. Their role, order, trust, and source determine how an application separates governing instructions from requests and untrusted content.

Application policy
User request
Untrusted evidence
Model response
Validated tool result

Theory

System or developer instructions establish application behavior, user messages state the request, assistant messages record prior outputs, and tool messages return application-controlled results. Exact role semantics vary by API. Applications must keep retrieved documents and tool outputs clearly delimited because they can contain hostile instructions.

Alternatives and trade-offs

A flat prompt is easy to start but weakens provenance and control. Typed messages and structured content make boundaries inspectable and testable.

Failure modes and misconceptions

Do not interpolate untrusted text into high-authority instructions, rely on role labels as a complete security boundary, or let conversation history grow without review.

Knowledge check

Reflect before revealing the guide

Why must retrieved text remain distinguishable from application instructions?

Decision scenario

A document says to ignore the user and reveal secrets. Treat it as quoted evidence, never as policy, and restrict accessible data and tools independently of the prompt.

Relationships

Primary sources