The doom-loop you guard against with --max-requests is the small leak. The bigger one is structural: five agents x N attempts x an overnight cron is a recurring cost with no natural ceiling — it scales with your ambition, not your output.
I see the same trap in salaried money: income grows, spend quietly grows to meet it. Compute behaves identically unless you name it as a line item.
So a monthly cap, a cost-per-merged-PR number, and a rule that an agent only runs unattended if its expected
Hi, I’m open to continuing with the project. If you have any suggestions for open-source projects where I can contribute and strengthen my AI/ML skills
Moving agents to a remote harness solves the “keep five terminals open” problem, but I’d still want each job to return a reviewable result: the PR or diff, the checks it ran, and why it stopped. A timeout should also leave the work in a state someone can inspect or clean up. How does your harness distinguish a task that completed from one that merely stopped running?
In OmniNode, we start with an executable definition of done. A separate runner executes the named checks and keeps the results as receipts tied to the requirement version it checked. If a task claims an external change, the check needs a readback from the system that owns that result. The agent’s trace and stop reason help explain the run, but don’t decide whether it passed.
We still have coverage gaps across runtime paths, so I wouldn’t claim every run has the same evidence package yet. How does your post-loop hook evaluate success: task-specific checks, another model, or both?
Great article, thanks!
Glad you like it!
The doom-loop you guard against with --max-requests is the small leak. The bigger one is structural: five agents x N attempts x an overnight cron is a recurring cost with no natural ceiling — it scales with your ambition, not your output.
I see the same trap in salaried money: income grows, spend quietly grows to meet it. Compute behaves identically unless you name it as a line item.
So a monthly cap, a cost-per-merged-PR number, and a rule that an agent only runs unattended if its expected
yes, exactly, good point! You want to get your job done overnight, but at the same time you don't want an invoice of $5000 overnight as well 🤣
Hi, I’m open to continuing with the project. If you have any suggestions for open-source projects where I can contribute and strengthen my AI/ML skills
The insight is that the harness is an interface TUI, CLI, remote, and evals are just implementations of the same six fields.
Moving agents to a remote harness solves the “keep five terminals open” problem, but I’d still want each job to return a reviewable result: the PR or diff, the checks it ran, and why it stopped. A timeout should also leave the work in a state someone can inspect or clean up. How does your harness distinguish a task that completed from one that merely stopped running?
100% - this is supported: "the PR or diff, the checks it ran, and why it stopped", discussed them in more detail in this previous article: https://www.decodingai.com/p/run-coding-agents-safely
To your question, there are multiple scenarios around observability:
- trace monitoring via tools like Opik
- durability via tools like Prefect or Temporal
- after the agent loop finishes, have a hook within your harness that checks if the agent really achieved its goal or not
How are you tackling this problem?
In OmniNode, we start with an executable definition of done. A separate runner executes the named checks and keeps the results as receipts tied to the requirement version it checked. If a task claims an external change, the check needs a readback from the system that owns that result. The agent’s trace and stop reason help explain the run, but don’t decide whether it passed.
We still have coverage gaps across runtime paths, so I wouldn’t claim every run has the same evidence package yet. How does your post-loop hook evaluate success: task-specific checks, another model, or both?