Model Exfiltration (Model Extraction)

Model exfiltration — also called model extraction or model stealing — is an attack that reconstructs a proprietary model, or a close-enough copy, by systematically querying it and learning from the outputs. An attacker with API access sends large volumes of carefully chosen prompts, records the responses (and any confidence scores or logits you expose), and trains a cheaper substitute model that mimics your behavior. Related attacks try to recover training data or the system prompt itself. For SaaS builders, the exposed asset is often not the base LLM — which you rent — but the fine-tuning, prompt engineering, and proprietary data that make your feature valuable. Practical note: rate-limit and monitor per-key query patterns, avoid returning raw logits or verbose confidence data you don't need to, watermark or fingerprint outputs where feasible, and keep genuinely sensitive logic server-side behind business rules rather than encoding it entirely in a promptable model.