A reader asked what a small company should build for incident management on day one. Here is the entire policy.
Incident Policy, v1
When to declare. Declare an incident when a problem needs more attention than you can give it alone. The question is never how bad it is. The question is whether you need people. If unsure, declare. Nobody will ever be criticised for declaring an incident that turned out to be nothing.
What happens when you declare. For the people involved, normal work is suspended. One channel per incident. One person drives: they say who does what, and they say when it’s over. Whoever declares, drives, until they hand it to someone else.
Who is on call. One person, one week at a time, on a roster we build together. If you’re not on the roster this week, you’re off. Off means off.
Communication. The driver, or someone they pick, posts what we know, what we’re doing, and when the next update lands. Guesses are labelled as guesses. If customers are affected, tell them in plain words before they find out for themselves.
Afterwards. Within a week, the driver, or someone they pick, writes a short record. What happened, what we learned, what we’ll change. Anything we’ll change goes in the backlog like any other work, or it will not happen.
The count. Every incident goes on the list. Date, duration, what broke, what fixed it. A spreadsheet is fine.
Amendments
None yet.
If your first reaction is that this cannot be enough, good. Hold onto that.
Every incident process you have ever worked under is a record of wounds. Each rule in the big binder at the big company exists because one day, somewhere, its absence hurt. The severity matrix followed a fight. The commander role followed a shambles. The comms template followed a customer finding out the hard way. Process is what an organisation writes down after it bleeds, and a company that has not bled yet has almost nothing to write. Copy the binder anyway and you inherit its injuries without its history. The document above is short because your scars are few. It will not stay that way.
Most of the page explains itself. Three lines do not.
Declare an incident when a problem needs more attention than you can give it alone.
A declaration is not a verdict on severity. It is a mode switch. The moment someone declares, the rules of normal work suspend for the people involved: one channel, one driver, updates on a clock. That is all an incident is at this size. A signal that says stop what you are doing, I need you.
Which is why declaring has to be cheap. The engineer who declares at 2am over something that resolves itself by 2:15 has not cried wolf. They have tested the machinery, and the machinery worked. Say so in public. The alternative is a team that hesitates before pulling the handle, and hesitation is the most expensive failure mode available to you right now.
One person drives: they say who does what, and they say when it’s over.
Note what this line does not say. It does not say incident commander. There is no role here, no training, no rotation of certified humans. There is a rule that at any moment exactly one person is driving, and everyone knows who. The named role comes later, and you will find it below, filed with the other things you do not need yet. The rule and the role are different sizes. Confusing them is how a fifteen-person company ends up with a forty-slide onboarding deck for a job nobody holds.
If you’re not on the roster this week, you’re off. Off means off.
The roster looks like the part where a burden gets imposed. It is the opposite. At fifteen people, everyone is already on call, all the time, informally and without end. The pager in everyone’s head never stops. A roster does not create the on state. It creates the off state. Its product is the fourteen people allowed to sleep tonight because it is not their week.
This is also why the policy says a roster we build together, not a roster I have built. Write the thing with the team in the room, not for them. People defend rules they watched get made, and an on-call roster runs on that goodwill and nothing else.
Now for what is missing, which is most of the discipline. Everything absent from the policy is absent on purpose, and none of it is absent forever. Simple systems that work grow into complex systems that work; it does not run the other way. So every omission below carries an expiry date, written as an injury. When the absence of a thing has hurt you twice, it has earned its place in the document.
Severity. A severity matrix is a treaty, and treaties follow wars. Yours arrives the week two incidents both claim to be the most important thing in the company, or the month your services outgrow anyone’s ability to judge a response by feel.
Incident commander. Someone always drives; that rule is already on the page. The role gets a name the day two people both believe they are driving, or the day nobody does. Training and rotation follow the title, not the other way around.
Blameless. You do not legislate blamelessness at fifteen people. You model it. The written rule arrives the morning after somebody breaks the norm you never wrote down, and not a day before.
Tooling. The spreadsheet is the tool. Automation gets bought one manual step at a time, each purchase justified by the incident where that step cost minutes you did not have. When the spreadsheet starts lying to you, you may go shopping.
Reporting. Five incidents is a stack of anecdotes. Five hundred is a dataset. When the count can hold a trend, start reporting, because the people you report to fund trajectories, not moments. Until then, the count is not for them. It is a letter to the company you will be at fifty people.
Which leaves the last section, where two words are doing the most work on the page. None yet. Not none needed. Not none ever.
The amendments section is the only part of the document guaranteed to grow.
I love the ‘not yet’ approach to mature the guides / roles as need builds.
this weirdly made me want to be on call 😅