The People Who Designed It Should Be on the Pager

Share
The People Who Designed It Should Be on the Pager. Abstract opinion illustration in orange and dark grey on debugly.dev

Here is the thesis. A team's escalation path is its learning loop, and most escalation paths are designed to protect senior engineers from interruptions, which is to say they are designed to protect the system from being improved. The people with the context to recognise a design flaw are the last to see the incident that proves it, and by the time the pattern reaches them it has been summarised twice and lost its texture.

I am not arguing that seniors should absorb every page. I am arguing that the page is the most honest design feedback a system produces, and routing it exclusively to the people with the least context guarantees the feedback is never acted on at the level that matters.

What the pager actually teaches

An incident is a forced, high attention reading of the system as it is, not as it was designed. The person on the pager learns three things that no architecture document contains.

Where the system lies. The runbook said restart the worker, and restarting the worker did not help, because the worker is fine and the queue is the problem. That gap is only visible to the person holding the incident.

Which failures are frequent. A one off bug is noise. The same page every Tuesday is a design property. Frequency is only visible to someone who sees many incidents, and juniors see each one as novel because they have not yet seen the repeat.

Where the pain is. The ergonomic truth, that this alert is unactionable, that this dashboard lies, that this deploy is scary, is felt by the person doing the recovery. It is not visible from a design review.

All three are design inputs, and all three are collected by the pager and discarded by any escalation model that keeps seniors away from it.

The counterargument, fairly

The defence of the current model is real and I want to state it honestly. Senior engineers are the scarce resource for deep work, and a page at three in the morning taxes the next day's design work. Protecting their focus is not vanity, it is capacity planning.

There is also the growth argument. Juniors need incident experience to grow, and handing every incident up denies them the very learning the pager provides. A team where seniors take everything is a team where nobody else ever learns to hold the incident.

Both are true. And both are arguments about allocation, not about absence. The defect is not that seniors sometimes sleep. It is that the escalation path makes senior exposure to incidents rare, accidental and always in the direction of being avoided, so the learning loop that should feed design instead drains into a postmortem folder.

What the good versions look like

The teams that get this right share practices, and none of them amount to "seniors take every page".

Seniors carry a regular, bounded share of the rotation. Not every night, but a real share, often a secondary or a dedicated week, so that incident exposure is scheduled rather than accidental. The boundedness answers the focus argument. The regularity answers the absence argument.

Escalation is for help, not for handoff. The junior owns the incident and calls the senior in for a specific question, and afterwards the senior writes down what they noticed about the system, not just the fix. The distinction matters. Handoff removes the junior and the learning. A question keeps both and still transfers the context.

The senior reads the incident stream even when not paged. A weekly pass over every incident, ten minutes each, looking for the repeat and the lie, is cheap and produces the design backlog that the escalation model suppresses. This is the feedback loop reinstated without the three in the morning cost.

Postmortems are owned by the person who can change the design. A postmortem whose action items land on someone with no power to change the architecture is a ritual. Pairing the incident owner with the design owner closes the loop.

The uncomfortable conclusion

A system that pages the same alert monthly for a year is not failing its on call. It is succeeding at an escalation design that never let the monthly pattern reach anyone who could delete the alert's cause. The repetition is the output of the org chart, not of the software.

I have watched teams add tooling, dashboards and runbooks to reduce pages, and watched the pages continue, because the pages were a design property and the interventions were operational. The only intervention that removed the recurring page was a senior engineer sitting inside the incident long enough to recognise it as a design flaw and having the authority to change the design.

The rule of thumb

Route incidents toward context, not away from it. Give seniors a bounded but real share of the pager, make escalation a question rather than a handoff, and let the person who can change the design read the incident stream every week.

The page is the user feedback of the internal system, and a team that insulates its architects from it is a team that has chosen to keep its surprises. The same loop, applied to the first minutes of the incident itself, is the first ten minutes of an incident.

The junior side of the bargain

This argument is only fair if the junior side is handled with care, and I want to state that half explicitly. Putting seniors nearer the pager must not become a habit of seniors overriding juniors during incidents, which would teach the opposite lesson and destroy the confidence the rotation is meant to build. The senior's role in a junior owned incident is to answer the specific question, name the system property they noticed, and then hand the incident back. The junior owns the recovery and the narrative. The senior owns the follow up design question. When that division holds, the pager becomes the apprenticeship it was always supposed to be, and the feedback loop feeds both the system and the next generation of people who can hold it.

Read more