When Do We Stop? Writing the Kill Criteria Before You Write the Cheque
A clinic pilot with no stated ending does not end, it fades, and keeps billing while it fades. Deciding the number, the date, and who calls it costs nothing on day one and is nearly impossible to introduce in month nine.
What this settles
The number, the date, and the person that together mean this project ends, agreed before it starts.
In this article
Nine other decisions in this series are about starting well. This one is the cheapest of the ten and the one skipped most often, because on the day a clinic pilot begins nobody wants to talk about how it ends.
That is exactly when it costs nothing to say. In month nine it costs someone their reputation, which is why in month nine it does not get said at all.
Projects do not fail, they fade
A clinic AI pilot rarely produces a clean negative. It produces a partial: two clinicians use it, the number moved a little, flu season was unusual, and the next quarter would be a fairer test. Every one of those statements is usually true, and together they are indistinguishable from a project that is not working.
Without a written ending, continuing is the default and stopping requires an act of will from a named person who will be remembered as the one who ended it. Nobody volunteers for that, so the renewal quietly happens. The cost is not only the licence. It is that the next proposal arrives into a partners’ meeting that has learned these things do not conclude.
What drift actually costs
A pilot nobody ended
$28,800
$2,400 a month, month after month
The licence is the small half. The larger half is the attention of whoever keeps half-owning it, and the credibility the next project inherits.
A stop condition is three things
Not a feeling that it is not working. A sentence: the number, the level, the date.
“If notes are not being closed the same day by October 31, we stop.” That is checkable by anyone, requires no interpretation, and converts an uncomfortable judgement into arithmetic. It also needs a fourth element that most versions omit: who is allowed to call it. A criterion nobody owns is one everyone assumes somebody else is watching, which is the same failure as an AI system with no named owner arriving one level up.
This only works if the number was baselined before you started, which is the discipline a proof of value exists to enforce. Without a baseline there is nothing to measure against and every result is arguable, which is its own way of never having to stop.
The result is in. What does it mean?
Decision helper
The pilot is over. Which ending is this?
1. Did the number you baselined actually move?
2. Do you know why it behaved that way?
3. Who is using it without being reminded?
4. If you switched it off on Friday, what happens Monday?
My read
Notice that two of the three endings keep something. Stopping is not the opposite of succeeding, it is one of three ways a well-run experiment concludes, and the only genuinely bad outcome is the fourth one where nobody decides.
A project that cannot end cannot really be evaluated. Writing the ending is what makes the rest of it a decision rather than a commitment.
What to do this week
Write one sentence for the project you are running or about to approve: if this number is not at this level by this date, we stop. Send it to whoever owns the project, so it exists somewhere other than your head.
If you cannot fill in the number, you have found something more useful than a stop condition. You have found that the project does not yet have a definition of working, which is the same gap the first decision in this series was meant to close, and it is far cheaper to discover today than in month nine.
Paying for something you suspect should have ended already? Book a free consult and we will look at the number, the baseline, and the honest ending with you, including whether the right answer is to shrink it to the one clinician it works for rather than kill it. See more on Second Opinion.
The decision brief
One page for the person you are going to forward this to.
- The decision
- The number, the date, and the person that together mean this project ends, agreed before it starts.
- What getting it wrong costs
- Nothing dramatic happens, which is the problem. The pilot never fails, it just never quite succeeds, and it keeps its line in the overhead for a year because ending it requires a partner to volunteer as the person who was wrong. The next proposal then arrives into a partners' meeting that has learned these things do not conclude.
- What you can do this week
- For the AI project you are currently running or about to approve, write one sentence: if this number is not at this level by this date, we stop. Send it to the person who owns the project. If you cannot fill in the number, that is the finding, and it is a cheaper finding today than in month nine.
- What to tell your team
- If notes are not being closed the same day by October 31, we stop and we go back to how we worked before. That is not a threat to anyone, it is how we make sure this is worth your time.
Frequently asked questions
How do you decide when to stop an AI project?
By deciding before you start, and writing it down. A usable stop condition names three things: the number that matters, the level it has to reach, and the date it is measured. If notes are not being closed the same day by October 31, the project stops. Made in advance the decision is arithmetic. Made in month nine it is a judgement about whose fault it was, which is a much harder conversation and usually gets postponed instead.
What are kill criteria for a pilot project?
The conditions under which you will end it, agreed while nobody is invested. Good ones are specific and measurable, tied to the number you baselined at the start rather than to a general sense that everyone seems happier. They should also name who is allowed to call it, because a criterion with no owner is a criterion everyone assumes someone else is watching.
Why do failed AI pilots keep getting funded?
Because ending one requires a person to be visibly associated with a decision that did not work, and in a small partnership no such person volunteers. The licence is usually cheap enough that the renewal is easier than the conversation. This is ordinary sunk cost behaviour, and the reliable defence is procedural rather than personal: if the ending was written down in advance, stopping is somebody keeping their word instead of somebody admitting a mistake.
What should happen if an AI pilot does not work?
Three honest endings exist. Stop, when the number did not move and the reason is structural. Shrink, when it worked for one clinician or one appointment type but not broadly, which is the most common real answer. Scale, when the number moved and you understand the failure modes. What is not an ending is continuing unchanged while nobody decides, and that is the outcome the stop condition exists to prevent.
Before You Spend
All 10 decisionsThe decisions an owner or CEO makes before the first dollar of AI budget leaves the building. One decision per article, each ending in a page you can put in front of your leadership team.
Found this useful?
Share this with your network on LinkedIn, it helps more than you think.
Enjoyed this read? Get the next one in your inbox.
When there is something worth your time, you will be first to know. No spam, unsubscribe anytime.
Keep reading
Is Your Data Ready for AI? The Audit to Run Before Any Build
More clinic AI projects stall on data than on models, but 'get our data ready' does not mean what most practices fear. A six-question scorecard tells you whether the records behind your first workflow are ready, and what to fix if they are not.
Read articleWhere to Start With AI in a Clinic: How to Pick Your First Workflow
The hardest part of AI in a practice is not the model, it is choosing where to begin. A simple way to pick a first workflow that is high-value, low-risk, and proves itself fast, and the three candidates a clinic should never start with.
Read articleYour Patient Does Not Experience Your Org Chart
The front desk, the clinician and the biller can each hit their target while the patient has a terrible eleven days. You measure the boxes and they experience the lines between them, and buying clinic AI one department at a time usually makes those lines worse.
Read articleHave software that should be smarter?
Let’s map a free AI-transformation roadmap for your product.