Run: a monthly retainer that keeps a shipped AI build correct
After a deployment, the build keeps working only if somebody keeps checking it. Run is a retainer, billed monthly with a cap on hours, in which I monitor the agent, rerun its evaluation tests, fix what a model or vendor change breaks, and scope whatever should be built next.
Key takeaways
- AI builds decay without anyone touching them, because the models and APIs underneath them change.
- The hours are capped, so the monthly cost is known before the month starts.
- Run is optional. A build handed over with its tests and runbook can be run by your own team.
What the retainer covers each month
- Monitoring. The agent’s log is reviewed for failed steps, rejected plans and unusual spend.
- Evals. The evaluation tests from the deployment are rerun on a schedule and after every model or vendor update.
- Fixes. When a provider changes a model, a price or an API, the build is adjusted before your team feels it.
- The next build. Small improvements ship inside the hours. Larger ones are scoped and priced like any deployment.
Why a finished AI build still needs attention
Ordinary software does the same thing until someone changes it. An agent depends on a model its vendor retires or updates on their own calendar, on prompts tuned to that model, and on your configuration, which your admins keep changing for good reasons. Any of the three can shift an answer from right to nearly right, and nearly right is the failure nobody reports.
The evals catch it. Without them, the first signal is a person noticing, weeks later, that the output stopped making sense.
Capped hours and what happens past the cap
The retainer buys a fixed number of hours a month. Work is prioritised with the process owner, and if a month needs more than the cap, I say so before the hours are spent, with the choice of deferring the item or approving extra time. Unused hours are not a goal on either side; in a quiet month the report says it was quiet.
When a retainer is the wrong arrangement
If you have an engineer who can read the runbook, rerun the tests and adjust a prompt, take the handover and run the build yourselves. A retainer also makes little sense for a build that changes nothing on its own, such as a drafting assistant whose output a person always edits.
Questions
What does Run cost?
Is there a minimum term?
Can Run cover a build someone else made?
Run follows a deployment. If you are not there yet, the audit is the place to begin.