The more you launch, the more post-mortems you need

The more features your team launches, the more post-mortems you will need. AI writes a large share of the code now, so a team can ship a feature a week, and each of those features either changes something for your customers or leaves you something to explain later.

So look at your last few launches. Do you run a post-mortem after a launch at all? In the weeks after, do you see a rise in support tickets and bug reports, or does the feature just run flat on the usage page? A flat line feels safer than a pile of tickets, while it is the same waste: the team spent the weeks, the code needs maintaining and nobody's work got any easier.

If you have had a flat one, you know the feeling of opening the usage page a second time in case the first number was a mistake. Everybody worked hard and nobody did anything wrong, which is what makes it such a lonely week.

What the post-mortem looks like

Imagine a team that ships a reporting feature in nine days, which is possible now that AI writes a large share of the code. The launch email goes out. The announcement collects a few friendly reactions in the customer channel. Two weeks later the usage page shows the feature opened by the people who built it and a handful of curious customers who tried it once.

The post-mortem starts with the usual questions. Was the announcement clear? Was the button in the right place? Did we pick the wrong segment? Every answer is reasonable and every fix is cheap, which is the trap: the team builds the fix and never asks the other question.

The other question

Before the launch, the customers had a way of dealing with this problem and they still have it. Somebody exports the data and builds the report in a spreadsheet. Somebody asks an analyst. Somebody lives without it. Whatever they chose is what your feature has to replace and it has been working for them for a while [1] [2].

A flat launch is rarely a launch problem. It means the feature did not replace anything, and no announcement or button position fixes that. People switch away from something they already do when it stops being good enough, so a feature that arrives without a gap to fill gets a polite look and no change in habit.

Building has become cheap, so this will happen more often, not less. A team that ships a feature a week and keeps asking about the button will run the same post-mortem every week, while every one of those features stays in the product for somebody to maintain.

One question

Open the usage page for your last launch.

What did your customers do instead?

That one is for the next time you look at the number.

References

1. Moesta, B. with Engle, G. (2020). Demand-Side Sales 101: Stop Selling and Help Your Customers Make Progress. Lioncrest Publishing. https://www.porchlightbooks.com/products/demandside-sales-101-bob-moesta-9781544509983 (opens in a new tab)
2. Wikipedia. Status quo bias. https://en.wikipedia.org/wiki/Status_quo_bias (opens in a new tab)