FoundersOnly

Running a small hosting company: what outage taught me about status pages

#1

Hostík runs 300 Czech shops. Worst outage: a fiber cut plus our failover script failing on a typo. Four hours, during Christmas market season. The technical fix took a week. The trust fix took a quarter.

What worked: a status page updated every 20 minutes even when the update was "still broken, next update 20:00". Customers quoted those updates back to us later as the reason they stayed. Silence is the churn driver, not downtime.

What we changed: status page on separate infrastructure (obviously, in retrospect), runbooks tested by the junior (if only seniors can fail over, you cannot fail over), and a credit policy written before the outage instead of during it.

#2

"Silence is the churn driver, not downtime" belongs on a wall. Our incident notes now include one human sentence before the technical paragraph. Complaints about tone went to zero.

#3

Separate-infrastructure status page: learned the same way, during the outage, like everyone. Ours is a static page on a different provider updated by a script. Costs cents, saves quarters.

#4

Runbooks tested by the junior is the line. If the runbook only works in the senior's head, it is not a runbook, it is a memory. Stealing for our maintenance notices.

Your perspective belongs here.

Register to reply