Strategy · Guide
Decide when an AI pilot should stop
Stopping can be a successful decision when a bounded experiment answers its question.
Write stop criteria early
Before testing, state what would make continued use inappropriate. Examples include material unsupported outputs, review effort greater than the current process, or unresolved access issues. NIST’s Manage guidance includes decisions about whether deployment should proceed. Our recommendation is to translate that principle into a small set of operational conditions people can recognize, rather than waiting for a dramatic incident to justify pausing.
Distinguish stop from repair
A confusing label may deserve correction; a missing legal authorization may prevent the use entirely. Do not put every issue into an endless improvement backlog. Assign severity based on business consequences and available controls. In an illustrative summary pilot, a formatting issue and an invented payment term should not be treated as equally tolerable. The owner must know which findings pause the workflow immediately and which can wait.
Workflow at a glance
Preserve a working fallback
Stopping is only practical if the team can continue serving clients. Keep authoritative records outside an unproven workflow, document the manual route, and rehearse who takes over. Avoid deleting old templates or removing access before the pilot is accepted. A fallback should not require the unavailable developer to perform a rescue. It needs to be understood by the actual people on duty when an exception occurs.
Capture the learning
At review, record the failed assumption, evidence, remaining uncertainty, and whether a smaller use remains worth testing. Do not rewrite an unsuccessful experiment as success because the team learned something; explain the learning accurately. Equally, do not treat a decision not to deploy as wasted work. Preventing a costly rollout can be valuable when the evidence is clear and the decision is documented.
A pause rehearsal
Schedule a safe rehearsal in which the assisted workflow becomes unavailable. Ask the team to continue with the agreed fallback and record where they hesitate. Fix missing instructions, access, or ownership before expanding the pilot. A rehearsal is not proof that every incident is covered; it tests whether the basic stop condition has an operating meaning beyond the wording in a project document.
Sources and further reading
- AI RMF Playbook: ManageAI RMF Playbook: ManageNIST · Documentation ·
- NIST AI RMF Playbook FAQsNIST AI RMF Playbook FAQsNIST · Documentation ·
Editorial approach
AI-assisted original writing, checked against linked public sources. Practical recommendations and illustrative scenarios are editorial analysis, not reported client results. Source dates and product details may differ from this collection’s preparation date.