Prompt Logging

Prompt logging is the practice of storing the inputs sent to a model and the outputs it returned, so that behaviour can be debugged, evaluated and audited after the fact. It is close to unavoidable in a serious AI system — without it, a complaint about a wrong answer cannot be investigated, an evaluation set cannot be built from real traffic, and a cost or latency regression cannot be traced to what changed. It is also the point at which an AI feature quietly becomes a store of sensitive data. Prompts contain whatever the user or the retrieval layer put in them: customer records, support tickets, documents, credentials pasted by mistake. A log of prompts is therefore a copy of some of the most sensitive text in the product, held in a system that was designed for engineers rather than for data subjects, and often replicated onward into an observability vendor. The controls that make it survivable are ordinary ones applied deliberately. Redact before the record is written rather than after, because a redaction step that runs later still leaves the raw text in transit and at rest. Set a retention period short enough to be defensible and long enough to be useful, and enforce it by deletion rather than by policy text. Restrict access with the same rigour as the underlying data — a support agent who cannot open a customer's file should not be able to read a prompt containing it. Decide separately what is logged in full and what is logged as metadata. Token counts, model version, latency, prompt version, tool calls and a trace identifier answer most operational questions and carry little personal data. Full prompt and output text is what evaluation and complaint handling need, and it is the part that should be sampled, redacted, and kept for the shortest period the use case tolerates. Say all of this where users and buyers can read it, and check the provider side too: whether a model vendor retains prompts, for how long, and whether they are used for training is a contractual question with a zero-retention answer available from most, and it is separate from what your own logs do.