Supposed to Run Itself

Most agent demos end at the moment the agent says it succeeded. That’s where my work starts.

Last year, Helix was a few machines at home and one rule: if a routine task needs me to log in, it isn’t finished. I’ve spent the time since finding out how much of what passes for autonomy falls apart once you ask for proof.

I’m a registered polysomnographic technologist, which is probably why I got suspicious early. Much of that work is watching signals and deciding which ones are real. A patient rolling over can look alarming on the screen, while a real problem can look quiet. Sleep medicine has also had software scoring studies for years, and a person still reviews the result, because everyone in the field knows the software can be confidently wrong. The AI industry is learning that lesson in public and acting surprised.

Multi-agent is a concurrency problem

Early on, I measured progress by how much the agents could do, and they could do a lot. They also got in each other’s way. Two agents working in the same tree cost me a review gate in August and a file in September.

Nothing about that was an intelligence failure. Two writers shared one resource with no coordination, a bug older than most of the people now selling multi-agent frameworks. A better prompt wouldn’t have touched it, and I’ve come to think an instruction in a prompt is only a request. A rule is something the agent can’t get around.

So I wrote a boring one: code reaches the main server by one route, behind tests, and it rolls back if they fail. That route grew into a loop that runs on a schedule, finds failures that keep recurring, proposes fixes, tests them and ships them. A new check has to fail against the broken code before it can vouch for a fix, because I don’t trust a test I’ve never seen fail. Every change gets a versioned release note, and there have been 650 of those since version 1.0 went out on September 5.

Your dashboard is lying to you

The failures that taught me the most had nothing to do with how smart the models were.

In September, two servers went down. An outside watcher I’d built saw it and said nothing for six hours. I had shipped it muted so it wouldn’t nag me while it proved itself, and four separate settings kept it quiet. That one was entirely my fault, and I’d now call a muted monitor worse than none, since having it let me believe someone was watching. It ships armed now, and a muted watcher gets reported like any other problem.

A few days later, the account paying for one of our models hit zero, and every job that depended on it failed for days. Everything watching the fleet was checking whether machines and services were alive, and none of it could see a wallet. I’ve never seen a wallet on an agent architecture diagram, and I’d bet yours doesn’t have one either. I added a watcher for the balance.

Then there was the tool reporting “Logged in” on machines whose logins were dead. We stopped asking it, and now we find out who can do the work by having them try. I treat an agent’s “done” the same way. It’s a claim, and it’s worth about as much as that login status until something outside the agent confirms it.

Alerts are where people fool themselves most. Everyone adds them and almost nobody measures them. I tested mine against 1,924 real outputs. Tightening the rules took them from about 20 a day to about 6, and on a labelled test set they still caught all 49 cases they had to catch. If you can’t give me those numbers for your own alerts, you don’t know what they do.

I want less of my attention spent, and all of my authority kept

Full autonomy is the wrong thing to aim for, and I say that as someone whose rule is that the system shouldn’t need me. What I want is for it to need my attention rarely and my authority always.

I use agents for most of the routine work, more of it than ever, with more checks around them. “Done” has to mean a test passed or a real run happened, and the result gets written down. A model from a different vendor reviews what matters, since models share blind spots with themselves, and asking one to check its own work is mostly theater. Agreements with an agent go in a log so they survive the conversation. A small set of systems can’t be changed without my recorded approval, and the most serious approvals carry my signature.

I’d take a mediocre model inside that structure over the best model in the world without it. Most of the engineering is in that layer: deciding what an agent may do alone and what needs a person, and being able to tell afterward what really happened. It’s less fun to post about than a new model release, and it’s the only part of this that has kept paying off.

This site came out of the same process. I rebuilt its publishing desk with agents, and a review of the work caught a real security bug before anything went public. I don’t find that embarrassing. Agents write bugs and so do I, and what I care about is whether something was set up to catch them.

Things still break, and I still find gaps I hadn’t thought to check. When something goes wrong now, I can usually tell you what knew about it, when it knew, and why it said what it said, which I couldn’t have done a year ago. I’d ask that of anyone telling you their agents run themselves. The tagline here still says “supposed to,” and I’m keeping it.