You're paying for AI licenses. Your team says they're using them. Someone mentioned it "saves a ton of time." And if your banker asked you tomorrow whether it's worth the money, you'd probably say "I think so" and change the subject.
That's not a failure on your part. Almost nobody writes down how long the work took before AI showed up, so there's nothing to compare against.
The fix is smaller than it sounds. You don't measure "AI." You measure one piece of work, before and after, with three numbers and one honest question. A month is enough to know.
TL;DR: Pick one task your team does every week. Write down how many, how long each takes, and how long until it's done, for one week. Run the same task with AI for three weeks and count the same things again. Then ask the person doing it what they'd do if you took it away. If the hours you got back went somewhere useful, it's paying off.
Measure one task, not the tool: proposals, meeting follow-ups, or invoice backup, not "how AI is going."
Write down the before: one week, three numbers, on a sticky note if that's what you've got.
Count the same three after: time per item, turnaround, and things that slipped.
Decide where the hours go: time saved that disappears into email isn't a return.
Ask the take-it-away question: the answer tells you more than any dashboard.
Why most owners can't answer the question
Most AI rollouts get judged on feel. The team likes it, or they don't. Someone had a great week with it, or a bad one.
Feel isn't useless, but it doesn't hold up when the renewal bill shows up. The common substitutes don't help much either. Seat counts and login reports tell you who opened the tool, not whether any work got faster. The number of chats tells you even less. A person can spend an hour in a chat and end up exactly where they started.
What you care about is the work. So that's what gets measured.
Step 1: Pick one task you can count
The right first task happens every week, takes real time, and has a clear "done." Good candidates for most 10 to 100 person companies:
Proposals or quotes: each one has a start and a send.
Meeting follow-ups: notes, action items, and the recap email.
Customer replies: the repeat questions your front office answers all day.
Month-end reports: invoice backup, owner or client updates, or status reports.
Pick one. Not three. The point is a clean comparison, and every task you add makes it muddier.
Pick a task one person owns, too. If five people do it five different ways, you'll be measuring the people, not the change.
Step 2: Write down the before
For one normal week, have the person who does the task jot down three things:
1. How many they did.
2. About how long each one took, start to finish, including the digging around.
3. How long until it was out the door, from the moment it was needed.
That's it. A sticky note on the monitor works. Rough is fine. "About two hours, sent the next afternoon" is a perfectly good data point.
If you've already rolled AI out, you can still do this. Ask the person to estimate how the task went before, then sanity-check it against a few old emails or files with timestamps. It won't be perfect. It'll be a lot better than "I think so."
Step 3: Run it with AI for three weeks
Same person, same task, now with AI taking the first draft. Three weeks, not one.
The first week is usually slower. People are learning what to ask for and fixing drafts that aren't quite right yet. If you judge on week one, you'll cancel something that was about to work. By week three you're seeing the real pace.
Make sure the setup gives it a fair shot. An AI that knows nothing about your business will write generic drafts that take as long to fix as to write. Give it your templates, a few good past examples, and the instructions you'd give a new hire. (Our post on the six layers of an AI setup covers what a fair setup looks like.)
Step 4: Count the same three things again
At the end of week four, write down the same three numbers: how many, how long each, how long until it went out.
Then add a fourth thing the "before" week couldn't show you: what stopped slipping. The follow-up that used to go out two days late. The quote that used to sit until Friday. The report that used to have a typo the client caught. Those don't always show up as hours, and they're often the part customers notice first.
Here's how the math works, with made-up round numbers just to show the shape. Say a proposal took three hours and now takes one, and you do four a week. That's eight hours a week back. Multiply by what that person's hour costs you, compare it to what you're paying for the tool and the setup, and you have your answer. Most owners can do that on the back of an envelope.
Step 5: Decide where the hours go
This is the step almost everyone skips, and it's the one that decides whether AI pays off.
Eight hours back a week is only a return if those hours go somewhere. If they drift into more email and longer meetings, you paid for AI and got nothing you can point to. Decide ahead of time where the time goes: more calls with customers, clearing the backlog, taking on the account you'd been turning away, or not posting that job opening yet.
That last one is where owners usually feel it. Not as a line on a time report. More like "we didn't have to hire for that role this year."
The take-it-away question
Once the numbers are in, ask the person doing the work one question: "If we turned this off Monday, what would you do?"
If the answer is "honestly, not much would change," you have your answer, and it's not a good one. If the answer is a long pause followed by "please don't," that tells you something the numbers can't. People don't protect tools that don't help them.
We'd trust that answer more than any usage report.
What can go wrong
Four things, usually.
Measuring too much at once: five tasks and three people, and nobody can tell what changed. Start with one.
Judging too early: week one is the learning curve. Give it three.
Blaming the tool for a setup problem: generic drafts usually mean the AI wasn't given your templates, examples, or instructions, not that you picked the wrong product.
Counting hours that went nowhere: if nobody decided where the time goes, it goes to email.
What if the answer is no?
Sometimes the honest answer is that it isn't paying off yet. That's useful to know, and it's usually one of three things.
The task was the wrong fit. Some work is too varied or too rare for AI to help much. Try a more repetitive one.
The setup was too thin. This is the most common one we see. The tool is fine. It just doesn't know enough about your business to be useful.
Nobody's really using it. If that's what you're seeing across the team, our post on why teams don't use their AI licenses covers what to do.
And sometimes the right answer is to stop paying for something that isn't working. We'd rather you do that than keep a subscription out of hope.
Could your business do this?
Yes, and you can do it on your own with this post and a sticky note. If you'd rather have someone count with you across a few roles at once, that's where we come in. Every engagement starts with a discovery call. From there we run a short team survey and a systems audit to find the tasks where AI is most likely to pay off, then measure them the same way. A Payoff Snapshot looks at one workflow. A full Payoff Audit looks across the whole business.
Where to start
This Friday, pick one task and write down three numbers for it: how many this week, about how long each took, and how long until each one was done. That's your before. Everything else builds on it.
FAQ
How long does it take to know if AI is paying off?
About a month for one task: one week to write down the before, then three weeks with AI. The first week with AI is usually slower while people learn, so don't judge on it.
What should I measure?
Three things for one task: how many, how long each takes, and how long until it's done. Then note what stopped slipping. Skip login counts and chat counts. They measure the tool, not the work.
What if I didn't measure before we started using AI?
Ask the person who does the task to estimate how it used to go, then check a few old emails or files with timestamps. Rough numbers are fine. They're better than none.
Is time saved the only thing that counts?
No. Faster turnaround, fewer things slipping, and work customers notice all count. So does not having to hire for a role you were about to post. Time is just the easiest one to write down.
What if it isn't paying off?
Check the task and the setup first. Most of the time the work was a poor fit or the AI didn't know enough about your business. If neither fixes it, stop paying for it.
Practical AI. No Hype. Just ROI.