The permission layer is the most important section, and the tension you flag at the end is the whole point. There are two enforcement regimes here and only one is real. Deterministic and trustworthy: the allow/deny rules, the sandbox jail derived from them, and the monotonic scope-narrowing where a child can never out-permission its parent, which holds no matter what the model does.
Then plan mode, enforced prompt-side by a system reminder. That's not an exact control, it's a suggestion, and from a security view anything enforced by "we hope the model reads it" should be treated as already bypassed, since a confused model and an adversarial input land in the same place. The takeaway across your build/configure/use frame: any safety property you actually care about has to live in the deterministic layer, not the just the prompt. That or make sure to include human in the loop.
The harness is where most teams underinvest. They agonize over the model choice and ship a default loop, default tools, no memory. Then they blame the model when it fails. Build the moat parts, use the rest as-is, but know which is which.
Great article!
The permission layer is the most important section, and the tension you flag at the end is the whole point. There are two enforcement regimes here and only one is real. Deterministic and trustworthy: the allow/deny rules, the sandbox jail derived from them, and the monotonic scope-narrowing where a child can never out-permission its parent, which holds no matter what the model does.
Then plan mode, enforced prompt-side by a system reminder. That's not an exact control, it's a suggestion, and from a security view anything enforced by "we hope the model reads it" should be treated as already bypassed, since a confused model and an adversarial input land in the same place. The takeaway across your build/configure/use frame: any safety property you actually care about has to live in the deterministic layer, not the just the prompt. That or make sure to include human in the loop.
Exactly! You can add that custom business logic into hooks or a custom layer that sits between the serving layer and the harness
The harness is where most teams underinvest. They agonize over the model choice and ship a default loop, default tools, no memory. Then they blame the model when it fails. Build the moat parts, use the rest as-is, but know which is which.
Exactly!