Data Sampling

Data sampling is when an analytics tool answers a question from a subset of the data and scales the result up, rather than reading every row. It exists because scanning everything is expensive, and it is usually a reasonable trade: the shape of a trend, the rough size of a segment and the direction of a change all survive sampling comfortably. What does not survive is anything rare or exact. A sampled report is a poor way to count events that happen to a small number of accounts, because the sample may contain very few of them or none, and the fewer there are the more violently the estimate swings. It is a worse way to produce a number that has to be exactly right — anything that feeds billing, a contractual commitment, a board pack or a regulatory filing should come from a complete count, since a sampled figure that is later restated is far more damaging than a slow query. The trap that catches people is compounding filters. A sample drawn across all traffic is generous, but each filter you add narrows it, and by the time a query reaches one plan tier in one country on one device the estimate may rest on a handful of underlying records while the interface still shows a confident round number. The practical defence is to know, for each tool in your stack, whether it samples at all, what triggers it, and whether the report tells you when it happened, because a tool that samples silently will teach a team to trust numbers it should be interrogating. Where the answer must be exact, run it against the warehouse and accept the wait. Where it only has to be directionally right, sampling is fine and arguing about it wastes the time it was meant to save. Two related habits belong here as well: state in the report which numbers are estimates, and never mix a sampled figure and a complete one in the same comparison, because the difference between them will be read as a change.