The pilot that can't fail

The pilot started in March. The vendor's slide says 92% accuracy. The two estimators using it say it's fine, mostly, and they still check every line by hand before a quote goes out. Your IT lead thinks it's in production. The inside sales manager thinks it's still being evaluated.

They're both right. Nobody ever wrote down what "working" would look like, so there was never a day when the thing passed or failed. It just kept going.

Why a pilot like this can't fail

A pilot gets sold as the low-risk way to find out. Without a written pass mark it's the opposite: the cheapest way to never find out. It can't fail. It can only continue, and renew.

It's a common spot to be stuck in. Grant Thornton's 2026 AI Impact Survey polled 950 senior leaders, 100 of them in manufacturing, between February and March. In its manufacturing write-up, 48% of manufacturers were still just piloting AI, against 34% across industries. Manufacturing isn't behind on starting. It's behind on finishing.

What the stall actually costs

The license fee is the small number. The big one is the problem the pilot was supposed to fix, still sitting there a year later.

Run the math on your own numbers. Here's an example with round figures, so swap in yours:

• Average quoted order: $12,000
• Gross margin: 25%, so about $3,000 for every order you win
• A quote that goes out in two days instead of the same day loses some deals to whoever answered first. Win back one more of those a week and that's about $150,000 a year in gross margin.
• Win back ten more a month and it's about $360,000.

Those aren't benchmarks. They're the reason the pilot got approved in the first place. If it never cleared a test, you don't know whether you're getting any of that.

It also caps your growth. You approved the pilot because quoting was the step that breaks first when volume climbs. Volume climbed, and it broke. So you hired for it, or tried to and couldn't, and the pilot absorbed none of that growth. The next 20% of volume will cost you a hire exactly the way the last 20% did. Manufacturing had 584,000 open jobs in September, a 4.4% openings rate, in preliminary BLS figures. The estimator you'd be hiring is somewhere in that number.

What fixing it looks like

Write the pass mark before you sign anything. Not after the demo. Before.

An acceptance test is one sentence with no soft words in it: by this date, this named person does this task on this much real work, and this measured number clears this threshold. For the quoting tool, that might read: by March 31, both inside estimators send 80% of standard quotes the same day, starting from the tool's draft, with no more than one correction per quote. If the proposal can't fill every slot, the proposal isn't finished. That's the vendor's problem to fix, not yours.

Write two tests, not one:

1. The output is right. The draft matches what your best estimator would have sent.
2. A person actually used it with a real customer. The quote went out from the draft, not after a full rework.

A tool can pass the first and fail the second forever. Accurate drafts that everyone still re-checks line by line release no capacity at all, and that's the exact failure that keeps a pilot alive past its usefulness.

The rest is unglamorous:

• Match each line of the proposed scope to the evidence offered for that line. "The pilot was a success" often means one of five things got tested.
• Sort every number in the proposal into measured, self-reported, projected, or a quoted price.
• Name the person whose Monday changes, and what comes off their plate to make room.
• Measure your own baseline before the vendor conversation, and never ask the vendor for it. If they supply it, the number their claim gets measured against is theirs.

This is why a fixed-price prototype running on your own quotes, with the test agreed before work starts, beats an open-ended pilot. You find out in weeks whether it clears the bar, not at next year's renewal.

When this isn't worth the effort

• If the software is off the shelf and costs a few thousand a year, this is more rigor than the decision deserves. Buy it, run it for a quarter, and cancel it if it's bad.
• If you already know it works and you're negotiating price, skip to the negotiation.
• If nobody on your team can agree what a correct quote looks like, fix that first. No test survives that, and neither does any tool.

What the numbers say

Forecasts about AI travel further than their evidence. IDC's 2027 predictions, released October 27, say 60% of Global 2000 companies will run most of their business processes "AI-first" by 2029. The same release predicts that 60% of enterprises will fail to turn their data into measurable business value by 2028. Put side by side, they say most big companies will run on AI and most will struggle to show what it's worth. That's a reason to measure your own results, not to plan on someone else's forecast.

The same goes for the line that "95% of AI pilots fail." It traces to a July 2025 preliminary report from a group called Project NANDA, not to an MIT study, according to this analysis of where the number came from. Your own pilot's numbers are worth more than either. Which is the point: write them down.

Try this before Friday

Pull the last pilot or software proposal you approved. Find the sentence that says what pass looks like: a date, a person, a volume, a number. It takes about twenty minutes. If there isn't one, you've found why the thing is still in next year's budget.

One next step

If there's a proposal on your desk right now, or a pilot that's been running since spring, book a scoping call. Twenty minutes is enough to tell whether it can pass a real test, and what that test should be.