Security

Real offensive depth

Testing and defence led by a published security researcher with five CVEs, including a CVSS 9.1 Mirai botnet kill-switch.

All security →
Compliance

Audit-ready, fixed scope

SOC 2, ISO, and the Canadian privacy stack, run end to end with an independent auditor.

All frameworks →
Resources

Learn the space

Original research, free tools, and plain-language guides on security and compliance, from a published security researcher.

Read the blog →
Security

How to Run a Blameless Postmortem (With Template)

Your site was down for 47 minutes last Tuesday. Customers noticed. The CEO is asking what happened. Your instinct is to find who made the mistake and make sure they do not do it again. Resist that instinct. It will make your team afraid to take risks, hide mistakes, and avoid accountability.

Blameless postmortems are the alternative. They focus on systems, not people. They assume that the humans involved made reasonable decisions given the information they had at the time. They produce lasting improvements instead of lasting resentment. The postmortem is one piece of a wider incident process, which is what an incident response retainer puts in place.

The process

Step 1: Schedule the postmortem within 48 hours. Do it while the details are fresh. Invite everyone involved in the incident plus anyone who wants to learn from it. Keep the meeting to 60 minutes maximum.

Step 2: Prepare the timeline. Before the meeting, the incident commander (or whoever led the response) writes a factual timeline. No opinions, no blame. Just: at 2:14 PM, the deployment pipeline started. At 2:19 PM, error rates increased. At 2:23 PM, the on-call engineer was paged. And so on.

Step 3: Run the meeting. Walk through the timeline. At each step, ask: What did we know at this point? What decisions were made and why? What information would have led to a different decision?

The facilitator is job is to redirect blame into systemic questions. When someone says "Bob should have checked the logs," reframe it: "What would have made it easier for anyone in Bob is position to check the logs?" Maybe the answer is better monitoring. Maybe the logs were not accessible from the incident response channel. Maybe the runbook did not mention checking logs. These are system problems, not Bob problems.

Want this handled? Tell us what your buyer is asking for and we will tell you what the work involves, what it costs, and what you can do yourself. Talk to us

The template

Incident title: [Brief description, e.g., "Payment processing outage, March 15, 2025"]

Severity: [SEV1/SEV2/SEV3]

Duration: [Start time to resolution, e.g., "47 minutes (14:19 - 15:06 EST)"]

Impact: [Quantify the impact. e.g., "312 customers unable to complete checkout. Estimated revenue impact: $8,400."]

Timeline: [Chronological list of events with timestamps]

Root cause: [The systemic root cause, not "someone made a mistake." e.g., "Database migration was applied to production without first running on staging. The migration contained a breaking schema change that caused the payment service to fail."]

Contributing factors: [Other factors that made the incident worse. e.g., "No staging environment for the payment service. Migration script did not include a rollback procedure. Alerting delay of 4 minutes before the on-call engineer was paged."]

What went well: [Things that worked during the response. e.g., "Rollback was executed within 8 minutes of diagnosis. Customer communication was sent within 20 minutes."]

What can be improved: [Systemic improvements. e.g., "All migrations must be tested on staging first. All migration scripts must include rollback procedures. Alerting threshold should be reduced from 5 minutes to 2 minutes."]

Action items: [Specific, assignable tasks with owners and due dates]

  • [Action 1] - Owner: [name] - Due: [date]
  • [Action 2] - Owner: [name] - Due: [date]
  • [Action 3] - Owner: [name] - Due: [date]

Making postmortems stick

The most common failure mode is writing the postmortem, identifying action items, and then never completing them. Fix this by:

  • Adding action items to your sprint backlog, not a separate document that nobody looks at.
  • Reviewing open postmortem action items in your weekly engineering meeting.
  • Publishing postmortems internally (and selectively externally) to build a culture of transparency.
  • Celebrating well-run postmortems. They are a sign of a healthy engineering culture, not a sign of failure.

Need help building an incident management process?

traztech helps startups build incident response and postmortem processes that improve reliability over time. We set up the tools, train the team, and facilitate the first few postmortems.

Book a free strategy call

Define severity before you need it

Most of the arguments during an incident are not about the fix, they are about how seriously to take it. Settle that in advance with definitions that a tired engineer can apply at 3am without asking anyone. A workable set for a small SaaS company looks like this. SEV1 is customer-facing unavailability, data loss, or a confirmed security compromise, and it pages immediately and wakes the founder. SEV2 is significant degradation with a workaround, or failure of a single major feature, and it pages during extended hours. SEV3 is a defect with limited blast radius that can wait for the next working day. Write the definitions with examples drawn from your own product rather than generic categories, because "significant degradation" means nothing until someone has written down that checkout failing for 5% of customers counts and that the admin dashboard being slow does not.

Attach the process to the severity, not to the mood in the room. SEV1 and SEV2 always get a written postmortem. SEV3 gets a two-line note in the incident log. This removes the negotiation about whether an incident "deserves" a postmortem, which is the conversation where blame quietly re-enters through the side door.

Roles when the whole company is six people

The incident commander model reads as overkill at small scale until the first incident where three engineers are all typing into production at once. You do not need a rotation of trained commanders. You need one rule: whoever declares the incident is the commander until they explicitly hand it over, and the commander coordinates rather than fixes. Their job is to keep a running timeline, decide who is doing what, and own the decision to escalate or roll back. Everyone else does exactly one assigned thing and reports back.

The second role worth naming is the communications owner, and at a small company this is usually a founder. They handle the status page, the customer emails, and the account managers asking for updates, so that the people debugging are not also drafting copy. The third, optional, role is a scribe. If you cannot spare a person for it, have the commander paste decisions into the incident channel as they happen with timestamps. That channel export becomes the timeline, and it is significantly more accurate than anyone's memory two days later.

Security incidents are not reliability incidents

The blameless template above works well for outages. It is the wrong first move for a suspected compromise, and using it unchanged is one of the more expensive mistakes we see. The differences matter from minute one.

Preserve evidence before you restore service. The instinct during an outage is to restart the box, redeploy, and rotate everything. During a security incident that instinct destroys the only record of what the attacker did. Snapshot the volume, capture memory if you can, export the relevant logs to a separate account the suspected credentials cannot reach, and only then rebuild. If your log retention is 7 days and you discover the intrusion on day 12, you have no investigation, only speculation.

Start the legal clock consciously. Under PIPEDA, a breach of security safeguards involving personal information must be reported to the Privacy Commissioner and to affected individuals when it creates a real risk of significant harm, and you must keep records of every breach regardless of whether it meets that bar. Quebec's Law 25 carries its own notification duty for privacy incidents. Contracts with enterprise customers frequently impose tighter windows than the law does, often 24 to 72 hours from becoming aware. Someone needs to be reading those obligations while the technical response runs, and that person cannot be the person restoring the database. Retainer clients get this thread pulled by us rather than by the founder; it is the main reason an incident response retainer exists at all.

The write-up for a security incident carries extra sections: what data was accessible versus what was demonstrably accessed, the indicators of compromise you found, the credentials and keys rotated with timestamps, and the notification decisions with the reasoning behind them. Auditors and enterprise buyers both ask for this document later. Writing it once, properly, is far cheaper than reconstructing it during a due diligence review eighteen months on.

Where root cause analysis goes wrong

Two failure modes turn a postmortem into theater. The first is the singular root cause. Real incidents have a chain: a change was risky, review did not catch it, the test environment did not resemble production, the alert was slow, the rollback was manual. Picking one link and calling it the cause means the other four survive to combine differently next month. Write contributing factors as a list and let it be uncomfortable.

The second is the five whys terminating at a person. If your chain ends at "because the engineer forgot to check staging," you stopped one question early. The next question is why the system allowed a deploy that skipped staging, and the one after that is why nobody noticed for four minutes. Human error is where an investigation begins, not where it ends. A useful test before you publish: could this document be read aloud with every name replaced by a role, and still make sense? If removing the names removes the explanation, the analysis is about people rather than systems.

A third, quieter problem is counterfactual reasoning. Sentences of the form "if only we had done X" feel like insight but are unfair, because the person in the moment did not have the information you have now. Replace them with what information was available and what would have had to be visible for a different decision to be obvious.

Making action items survive contact with the roadmap

Separate corrections from corrective actions. A correction is what you did to end the incident: restarting the service, reverting the deploy, restoring from backup. A corrective action changes the likelihood or the impact of the same class of incident recurring. Postmortems that list only corrections look complete and change nothing.

Cap the list. Three to five corrective actions with named owners and dates beat fifteen aspirational ones, because a list of fifteen guarantees that the important two get lost among the trivia. Assign each one to a person rather than a team, put them in the same tracker as feature work with a label you can filter on, and review the open ones in the weekly engineering meeting. If an item has been open for two review cycles, either resource it or close it as declined with a written reason. An openly declined action item is honest. A permanently open one is a lie you are telling yourselves about your own risk.

Track two numbers across incidents rather than within them: time to detect and time to mitigate. A high time to detect points at monitoring rather than at response, and a high time to mitigate points at runbooks and rollback tooling.

When you should not run one, and when not to hire us

Not everything needs a postmortem. A single failed background job that retried successfully, a third-party outage you have no control over and no dependency choice about, or a known-issue recurrence with an existing tracked fix all deserve a line in the log and nothing more. Running a formal review on trivia burns the goodwill you need for the incidents that matter, and teams that over-apply the process end up skipping it exactly when the stakes are highest.

On outside help: the template in this article is the whole product for most companies. Copy it, run your next two incidents through it, and you will have a working practice without paying anyone. Facilitation is worth buying in a narrower set of situations. The first is when the incident involved a genuine security compromise and you need someone who has handled notification decisions and evidence preservation before, because that is where an inexperienced response gets expensive. The second is when previous postmortems have turned adversarial and an outside facilitator can hold the room in a way an internal manager cannot. The third is when a customer or regulator has asked for a formal written report and the standard of documentation is higher than your normal practice. If you are outside those three, keep the money. If you are inside them, say which one when you get in touch and we will scope to that rather than to a program.

Want this handled? Tell us what your buyer is asking for and we will tell you what the work involves, what it costs, and what you can do yourself.

Talk to usOr talk about a retainer

Before you go

Want the rest of this by email?

If this was useful, I send a few short notes on security posture. Unsubscribe in one click, and replies reach me directly.

From Jacob Masse, principal of traztech. No spam, unsubscribe in one click.

Want a second opinion on where you stand?

We run SOC 2, ISO 27001 and the rest of the compliance stack for startups and SMEs, and the security testing that sits behind it. The first call is free, and we will tell you if you are not ready to start yet.

Book a free call

Track record

Who is actually doing the work

5
Published CVEs, including a CVSS 9.1
76
Controls taken from nothing to a passed SOC 2 Type II
Zero
Exceptions on that Type II report
20+
Penetration testing engagements delivered

Published vulnerability research

Five published CVEs. CVE-2024-45163 (CVSS 9.1) is a flaw in the Mirai botnet itself, which gave defenders a way to shut down attacker infrastructure. CVE-2026-42626 takes HP ENVY 5000 printers offline from any unauthenticated device on the same network.

A SOC 2 Type II built from nothing

At Humera, a venture-backed US security company, Jacob built the compliance programme in-house from nothing: no report, no policies, no documented controls. It ended in a Type II attestation across 76 controls with zero exceptions, on a team of 15.