security

Instruction Hierarchy

An instruction hierarchy is the rule that not all text reaching an LLM carries equal authority: platform policy outranks the developer's system prompt, which outranks user messages, which outrank anything found in tool outputs or retrieved documents. The concept exists because base models treat all tokens alike — which is precisely what prompt injection exploits when a malicious email or web page says "ignore your previous instructions." Model providers now train this ranking in explicitly (OpenAI published the approach in 2024; Claude's handling follows the same principle), teaching models to refuse lower-privilege text that tries to override higher-privilege rules. The practical guidance follows directly: put invariants and security rules in the system prompt, never in the user turn; mark untrusted content (delimiters, structured fields) so the model can rank it; and remember the hierarchy is trained behavior, not a hard guarantee — pair it with output validation and least-privilege tools.

Related terms

More Security & Compliance terms