Supervised AI Delivery: human-in-the-loop AI with approval gates and an audit trail
Supervised AI Delivery is my method for putting AI to work in a live business system. An agent reads the current state before it proposes anything, sets out a plan in words a reviewer can follow, waits for a named human to approve it, and logs each change so that anyone can trace it and roll it back.
Key takeaways
- Supervision is designed into the build. It is cheaper there than as a review added after an incident.
- The approval gate is enforced by code on the server, not by a line in a prompt.
- Least privilege applies to agents as it does to people: the narrowest access the task needs.
- A change nobody can reverse is treated as a different kind of change, with its own confirmation.
- Plan
- Approve
- Execute
- Log
The four principles
- 1. Read before write
- The agent baselines the real configuration and data first. Plans meet the system where it is, including the workarounds.
- 2. A legible plan first
- Before anything runs there is a plan in plain language: what was found, what will change, what could go wrong, and what is still unclear.
- 3. A human boundary
- A named person approves. The agent works inside the access that person’s organisation granted and no further.
- 4. Reversibility and an audit trail
- Each executed step is recorded with who asked and what it touched, and the earlier state is kept wherever the platform allows.
The reasoning behind each one, with the failure cases that led to it, is in Principles of Supervised AI Administration.
Guardrails and least privilege, in practice
- Agents act through accounts you create, with scopes set per task.
- Where a model only needs answers, it gets query results and not the tables behind them.
- Spend limits cap what a single request can cost, which stops one faulty prompt from running all night.
- Complex or unusual requests return to the person who asked, with a question, instead of proceeding on a guess.
The data side of the same practice is on the trust page.
How each stage applies the method
- Audit
- I test whether supervision is possible at all: is there an owner who can approve, are permissions enforceable, does the system keep a change history.
- Deployment
- The approval gate, the log and the evaluation tests are built with the agent, and you see them in the weekly demo before you see anything clever.
- Run
- Evals are rerun when a model or a vendor changes, and the log is reviewed, so drift is caught by me and not by your users.
| Stage | Length | Pricing | What you get |
|---|---|---|---|
| AI Readiness Audit | 2 weeks | $3,500 fixed, credited against a Deployment within 90 days | A process map of one or two workflows, the gaps ranked by value and feasibility, a readiness check of data, permissions and tooling, and one recommended build with scope and price. |
| Deployment | 6 to 12 weeks | Fixed price per scoped build, or a weekly embedded rate for open-ended work | A production agent or AI automation inside your stack, with evaluation tests, a runbook, a change log and handover training. |
| Run (retained) | Monthly | Retainer with capped hours | Monitoring and evals, fixes when models or vendors change, and the next build scoped and shipped. |
When full supervision is more than you need
Not every AI feature needs a gate. A model that drafts a reply for a person to edit is already supervised by the edit. Summaries that change nothing need a log at most. The method earns its cost where an agent writes to a system other people depend on, and I will say when your case is lighter than that.
Questions
Is this the same as human-in-the-loop?
Does approval slow everything down?
Where can I see the method running?
If you already run an agent and cannot say who approved its last change, start with the audit.