Deployment despair

A repulsive example

Recently I pulled out a few strains of my hair out of despair when I opened a web app’s README and read the deployment section:

Ask around on Slack whether it is okay to deploy. Make sure that enough people and someone from (dev)ops are available to help in case of a necessary roll back. Watch at the CI deployment step to notice when something goes wrong, and be able to test the deployment once it’s done.

Nope, we’re not talking about a decade old legacy product. This was a two year old codebase using next.js, graphQL and a popular headless CMS.

The deployment process itself wasn’t that hard, but there was a climate of fear and a resistance to deployments. It’s not that the people running the project weren’t technically capable of implementing automated testing (there was an abandoned cypress flow, and a couple useless jest unit tests). The root cause was that team was mismanaged. The devs however were responsible for their own misery too: they were not self-protective enough to spend a couple of hours to evolve the deployment process.

Deploying should be as casual as lacing a shoe, a regular automatic action one can do without thinking, drunken in the middle of night.

Benefits of frequent deployments

Let’s re-state the obvious:

How to get there?

If you develop a web app, your goal needs to be that no team member is ever scared to deploy their changes.

In what situation is CI/CD bad advice?

The general gist of simplifying deployments, and being confident in deployments are always applicable. Thanks dear devops movement for making everyone aware!

If your product has a high risk profile, then you might not have any business deploying fast - which is part of the culture that makes us bad at software engineering! 😬 Let’s say your product is a nuclear plant. Or election computers. Really any critical infrastructure. You might want to have more safe-guards in place for software deployments. You might not want to allow individual contributors to deploy changes themselves. You might have technical reasons preventing frequent deployments, e.g. software development might not happen on-site with the deployment and the site might be disconnected from public networks for security reasons.

But besides those examples?

For regular consumer facing web based projects I’d rather “ask for forgiveness, not permission”. The former you’ll have to do for 1 out of 100 deployments, the latter a 100 out of a 100 times. I know which one seems the better trade-off to me.