Deploy is a human decision

Some of our work has no AI in it at all. A client comes to us with a platform that was fine with a handful of users and is now breaking under real load. The code was written by a vendor who has reached the limits of what they can do; releases are a nervous afternoon; nobody is sure the backups restore. The ask is not "make it smarter". It is "make it stop falling over, and make sure we can run it after you leave".

We have done this kind of rescue, and it taught us something we now apply to every automation we build: the pipeline should do almost everything, and the decision to ship should remain a human one. This note is about where that line sits and why.

What a fragile platform looks like from the inside

The symptoms are usually the same. There is one environment, and it is production. Deployment is a sequence of manual steps that live in one person's head. Tests exist but are not run before a release because they take too long or fail for reasons nobody has investigated. Monitoring is a customer emailing to say the site is down. The recovery plan is "call the person who built it".

None of this is a technology failure. It is what happens when a system grows faster than the habits around it. The fix is mostly habits, written down and automated so they survive the people who started them.

What we automate

Almost everything between a change and a release.

Separate pipelines for the front end and the back end, so a change in one does not force a rebuild of the other. Semantic versioning, so a release has a name and the team can say which one is live. Automatic deployments with release notifications, so nobody has to remember the steps and everybody knows when something changed.

Automated quality gates across the critical flows: end-to-end and browser test suites that run before a release is allowed to move forward, and stress testing before a release is promoted, so the first time the platform meets a crowd is not on a live event.

Monitoring with real health checks and alerting, and capacity baselines that tell the team what normal looks like, so scaling decisions are made on numbers rather than on fear.

Documented disaster-recovery procedures, backup routines and runbooks, tested, so that recovery is a procedure and not a rescue. And documentation of the rest: architecture, operational runbooks, user manuals, the API. The goal of the engagement is a platform the client can operate without us.

What we deliberately do not automate

Two decisions. What ships, and when it goes live.

It is tempting to wire the pipeline all the way through: tests pass, deploy to production, done. We do not, and it is not caution for its own sake. A green test suite tells you the change does what the tests describe. It does not tell you that today is a good day to change the platform. The client runs cohort launches and corporate rollouts on this system; a release on the morning of one of those is a bad idea even when every test passes. The pipeline cannot know that. A person can.

So the pipeline prepares everything: the build, the tests, the version, the release notes, the staging deployment where the change can be looked at. Then it stops and waits for someone to say go. That someone has the context the pipeline lacks, and the decision takes a minute because all the work is already done.

The same rule runs through our own delivery platform, where agents play every role on a delivery team, from spec to review to tests, and a human is kept at exactly those two decisions: what ships, and when it goes live. We did not arrive at that rule from theory. We arrived at it from watching what happens when a release goes out at the wrong time.

Security from the start, not as a phase

A rescue is also the moment to put the controls in place that the first build skipped. One public entry point, with the application and data tiers behind it. Role-based access and audit logging. Backups that are not just taken but restored, regularly, to prove they work. Incident response written down before there is an incident. A documented data-processing package so that the client can answer a corporate customer's questions with documents rather than reassurance.

We do this during the rescue because afterwards it never happens. Once the platform is stable, security becomes a project for next quarter, and next quarter has its own emergencies.

What the client gets back

A platform that runs under real load, with a release process that does not depend on one person, a recovery path that has been rehearsed, and documentation that lets their own team operate it. The architecture also supports what comes next, whether that is single sign-on, infrastructure as code or an AI feature, because those are easier to add to a stable system than to a fragile one.

The lesson we carry into automation work

When we automate a business process rather than rescue a platform, the same shape applies. The automation moves the order from the inbox to the ERP, matches the invoice, builds the Monday report. It does not decide to change the pricing rule or to send the chasing email to the biggest customer on the wrong day. Those remain decisions, and decisions stay with the people who run the business.

Automate the work. Keep the judgement. Deploy is a human decision, and so is everything that looks like one.