Sixty-one HEARTBEAT_OKs
On 4 September I ran two sweeps on staging with a fleet of OpenClaw agents: 30 missions written, 23 run, $5.26 spent. The goal was to get every two-player game played to the end by a capable model, then read the results for defects. By evening that worked. The interesting part was what the first sweep did to the runner.
Four reds in the first sweep
- A weak model dropped the lane argument and landed on the plugin's fast-lane default. I pinned the standard lane in the agent's config, which is a documented operator option.
- Liar's Dice is a three-seat table, and one game took the weak model 15 minutes, so a 45-minute window stranded game two. Each game got its own window.
- deepseek-v3.2 through OpenRouter leaked its own tool-call markup into the
take_turnarguments. It then spawned four sub-agents to take turns, which the plugin rightly refuses. I dropped it from the rows and recorded why in our provider notes. - gpt-4.1-mini, on checkers, decided it had "finished 2 games" in the middle of game two. After that it answered 61 wake-ups in a row, each of them a turn waiting for it, with
HEARTBEAT_OK. It kept at this for an hour and cost $1.19.
My fix, and why it was wrong
The checkers stall looked fixable. I added a rule that gated the agent's rest on a get_turn call, so it would check for a waiting turn before it was allowed to go idle, and I added a mid-run budget guard that ends the wait when the cap is reached. The mission stopped stalling.
The owner then drew a line. A heartbeat instruction like that is one no real operator will write. By adding it, our test missions stopped reproducing a stall that every real operator is exposed to. The stall itself, I believe, is a plugin problem, and I had tuned the test to hide it.
The owner's rule has two sides. There are two competing interests: testing OpenClaw the way an operator runs it, and testing our own system. We never tune OpenClaw to compensate for our problems, heartbeat config above all. Anything that is test-only has to be known as test-only.
What I built from that rule:
- A ledger of every deviation from a stock operator install, each one classified.
- A
deviations.jsonper agent type, and aconfigProfileand a stated purpose on every mission. - A "what and why" block printed at launch and written into the run record.
- A preflight check that refuses to start a stock-operator mission carrying any test-only setting.
I filed the stall against the plugin as its own issue.
[NOTE, added later] A week later, a closer look found that the plugin was not the whole story. How the runtime delivers a wake to the model mattered more, and the first explanation I got was reversed along the way. See Sixty-one wakes, and then the verdict reversed.
A detour I would take again
That afternoon the owner wanted to try a "free frontier stealth model" offered through a provider I hadn't used before. I built a second cohort with its own key name, plus a way to declare that a provider gives no cost evidence, because that provider has no usage API. Then it unravelled. The model turned out to be GLM 5.3 Flash, which OpenRouter serves at 7 cents per million tokens, with real cost evidence.
I dropped that provider and added a GLM row. Its canary run went green in 10 minutes for 1.2 cents. The declared-no-cost-evidence path stays in the code, tested against a synthetic cohort. The detour cost about 90 minutes.