On-Call Rotation

An on-call rotation is the published schedule that says which named person is responsible for responding to a production problem at any given hour, and who is called if that person does not answer. It is the mechanism that converts a system somebody built into a service somebody operates, and its absence is usually invisible until an incident, at which point the first fifteen minutes are spent working out whose problem this is. The rotation is not really about waking people up. It is about making responsibility unambiguous in advance, at a moment when the alternative is a group chat where everyone assumes somebody more senior is already looking. What makes a rotation work is mostly what surrounds it. The person on call needs the access to fix things, not merely to observe them, because a rota that can only escalate is a slower version of no rota at all. They need a written procedure for the situations that recur, so that competence at three in the morning does not depend on having built the system personally. They need alerts that are worth waking for, since a rotation that pages several times a night for conditions nobody acts on trains its members to acknowledge and go back to sleep, which is the failure that matters. And there has to be a second name, reachable, for the ordinary case where the first person is asleep, on a train, or looking at the wrong dashboard. The load is also worth measuring honestly. Count the pages per shift and how many required action, review them regularly, and treat a rotation that keeps someone awake as a defect in the system rather than a fact of life, because the same alert firing every night is telling you something that a person should not be the fix for. One more question is worth asking before a rotation is created rather than after: what is this person expected to do that an automated response could not. If the honest answer for most pages is restart something or wait, the rotation is standing in for work that has not been done, and the fix is in the system rather than in the schedule.