How to measure AI visibility, and whether it brings customers

1. Is AI search bringing you customers?
2. Why AI visibility numbers are easy to misread
3. What our recorded answers show about measuring
4. How to measure AI visibility and the customers it brings
5. Questions owners ask
6. How we got these numbers
7. Sources
8. See it on your own website

Is AI search bringing you customers?

Homeowners now ask ChatGPT, Claude or Google’s AI Mode to find a contractor. Maybe an agency has offered you an “AI visibility score”, or you’ve tried a free checker. The real question underneath is simple: is AI search sending you work, and is that growing?

Answering it takes three separate records, because each one measures a different thing:

1. Named: how often an assistant names your business when asked the questions your customers ask. This is what AI visibility tools measure.
2. Visits and calls: how many people reach your website, call you or ask for directions after reading an AI answer. Google Analytics, Google Search Console and your Google Business Profile each show part of this.
3. Customers: how many of those people book a job. Only your own records, such as your calendar, invoices or call notes, can show this.

Each record is counted out of something different. Being named is counted out of answers. Visits are counted out of all your visits. Customers are counted out of inquiries. Blending them into one score hides what is actually happening.

There is no straight line between them, either. An assistant can name you and nobody clicks. A customer can find you in ChatGPT, search your name on Google a day later and call from your Business Profile. The rest of this guide explains why each number is easy to misread and what our recorded answers show about chance. Then it sets out simple records you can trust.

Why AI visibility numbers are easy to misread

One check is close to a coin toss

Assistants rarely give the same list twice. SparkToro had volunteers run the same prompts in ChatGPT, Claude and Google’s AI nearly 3,000 times. The lists almost never repeated, and the order repeated even less. The researchers concluded that the share of answers that include you is a reasonable measure, but a “rank” inside an AI answer is not ¹. One check on a Tuesday afternoon can’t tell you much either way.

Your test is not your customer’s test

When you ask ChatGPT yourself, the answer is shaped by you. ChatGPT may use an approximate location from your internet connection. With memory turned on, it may also use your saved memories when it writes its searches ². A homeowner across town gets their own version.

Tracking tools have the same problem in another form. Petra Labs, a company that sells AI visibility tracking, says most tools ask ChatGPT while signed out or through its developer access. It says none can ask as signed-in paying users at scale. In its test of one OpenAI model, a brand’s share of answers moved by more than 30 points depending only on the route the question took ³. Our own answers were also collected by software, each in a fresh conversation with no history.

The wording of the question moves the answer

Homeowners don’t all type the same words. “Who fixes tankless water heaters in Largo?” and “best plumber for a tankless heater near me” are the same need in different words. In a Petra Labs study of 36,000 runs, even a one-character typo shifted about a fifth of the sources an assistant cited. Rewrites that added context about the person asking moved far more ⁴. A tracker that asks one wording per job sees only a sliver of what customers see.

Small changes hide inside chance

Because answers vary, a share measured from a few answers is rough. In another Petra Labs study, six questions were each run 600 times in one day to see how much shares move by chance alone. With 100 runs, a real 5-point change was detected only 11% of the time ⁵. Checking by hand, you will usually run far fewer than 100.

Being named is not the same as a visit

Many people read the AI answer and stop. Pew Research Center tracked the real browsing of 900 U.S. adults in March 2025. Google users who saw an AI summary clicked a regular result in 8% of visits, against 15% when there was no summary. They clicked a link inside the summary in just 1% of visits ⁶. So more mentions may bring few extra website visits. Some customers will call straight from the answer, or look you up later by name.

Your reports don’t label AI traffic cleanly

Google counts visits from AI Overviews and AI Mode inside your ordinary Google search traffic in Search Console, in the Web search report ⁷. An AI Overview takes a single position on the results page, and every link in it gets that same position ⁸. Search Console shows your clicks from Google, but not which of them came from an AI answer.

Google Analytics now has an AI Assistant channel for visits from tools such as ChatGPT, Gemini and Copilot. It leaves out Google’s AI Overviews and AI Mode, which it counts as Organic Search ⁹. OpenAI says ChatGPT adds utm_source=chatgpt.com to the links it sends, a tag your analytics can read ¹⁰. Google’s list of examples doesn’t mention Claude. And a customer who types your web address after reading an answer shows up as a direct visit, with no sign of the assistant ⁹.

A visit is not a customer

A homeowner may visit your site from ChatGPT, leave, and call a week later from your Google Business Profile. Your analytics will never connect the two. Google’s count of Business Profile calls is the number of times someone tapped the call button there ¹¹. It can’t tell you who booked. Only asking customers, and writing the answer down, closes that gap.

What our recorded answers show about measuring

Between September 2 and October 14, 2026, we asked ChatGPT, Claude and Google AI Mode to recommend a local contractor 47,209 times: ChatGPT 19,577, Claude 16,358 and Google AI Mode 11,274. The questions covered HVAC, plumbing, electrical and roofing jobs in towns around Tampa Bay, Florida. Each was asked in a fresh conversation with web search on.

We counted a business as named when the answer set its name in bold, treating close spellings as the same business. Google AI Mode’s answers in our records don’t mark names that way, so the figures on names cover ChatGPT and Claude. These are observations of answers, not experiments.

The same question, two weeks later

Many identical questions were asked again, typically about 14 days later. On average, only 43% of the businesses ChatGPT named the first time were named again. For Claude it was 26%. If you checked once, changed nothing and checked again, most of the names you saw could be different.

The same need, in different words

We also asked general questions about the same trade and town in different words on the same day. On average, only 28% of the businesses ChatGPT named for one wording were named for the other. For Claude it was 26%. So for ChatGPT, rewording the question changed the list more than waiting two weeks did. For Claude, the two changed it about equally.

The comparison is rough, because the reworded pairs were general trade questions and most repeats were about one job. The lesson still holds: a fair test asks several wordings for each job you care about.

Visibility is a share, not a yes or no

For 990 pairings of a job (or a trade in general) and a town, we had at least 12 ChatGPT answers, typically 14. Each answer named about 3 businesses, but together those answers named 15 different ones. The most-named business appeared in 71% of the answers, not all of them. About 45% of the businesses were named only once.

Claude spread its picks wider. Across 423 such pairings, each answer named about 4 businesses and the answers together named 25. The most-named business appeared in 58% of them, and about 54% were named only once.

So a business that shows up once may rarely show up again, and even the most-named one is missing from some answers. The useful number is the share of answers that name you, counted out of every answer to the same questions.

How much the share moves by chance

To see what chance alone does, we split each of those sets of answers into two halves at random. Nothing set the halves apart except luck. We then compared the share of answers naming each business in one half with the other, for every business named in at least 25% of the answers.

• For ChatGPT, with about 7 answers in each half, the share typically differed by 17 points. One time in ten it differed by 43 points or more.
• For Claude, with about 6 answers in each half, the typical difference was 17 points, and one time in ten it was 50 points or more.

When we split the same answers by date instead, earlier against later, ChatGPT’s gap stayed at 17 points. Claude’s grew to 24 points, which suggests something beyond chance shifted in Claude’s answers over those weeks. In our records, two different models answered as ChatGPT, and two as Claude. A model update can move your share while nothing about your business changes.

Standard statistics say the same thing. With 10 answers, a business named in half of them could really be named anywhere within about 26 points either side of that. With 30 answers the range is about 17 points, and with 100 answers about 10 points. A change smaller than that range may be luck.

Bigger samples make bigger counts

We didn’t collect the same number of answers every week. In our five full weeks, the weekly total ranged from 6,689 to 8,592, a difference of 28%. A report that counts raw mentions would show growth in the busier week with nothing else changing. That is why every count needs the number of answers behind it.

Where the clicks would go

Of the 31,742 ChatGPT and Claude answers about specific jobs, 98% linked to at least one contractor’s own website, and 53% to a directory or review site. A customer who clicks from those answers can land on your site, where analytics can count the visit.

Google AI Mode works differently. In 74% of its 9,819 job answers, the links opened Google’s own listing for a business, not a website. A customer there may call or ask for directions from your Business Profile and never visit your site. For AI Mode, your Business Profile numbers matter as much as your website analytics.

How to measure AI visibility and the customers it brings

You don’t need an expensive tool to start. You need the same questions, asked the same way, many times, and honest records of what happened next. Here is a setup a busy owner can keep up.

1. Write down the questions your customers ask

List the jobs you most want more of, and the towns or ZIP codes you serve. For each job, write three or four questions in a homeowner’s words: one naming a town, one with a ZIP code and one with a time, such as “tonight”. Ten jobs give you thirty to forty questions. Save the list with today’s date, and don’t change it between checks.

2. Ask each question more than once, in each assistant

• Ask in ChatGPT, Claude and Google AI Mode, each in a new conversation. Where you can, use a private window or turn memory off, so your own history doesn’t shape the answer ².
• Repeat the whole list on several different days each month.
• For each answer, record the date, the assistant, the exact question and whether your business was named. Don’t bother recording your place in the list ¹.
• Open the sources each answer cites. Claude, for example, includes citations in every answer that uses web search ¹². Note whether they lead to your site, a directory or a Google listing.

3. Report a share, with its count

Write “named in 9 of 40 answers in October”, not “named 9 times”. The share lets you compare months with different numbers of checks. Keep each assistant separate, since they pick businesses so differently. With a few dozen answers a month, a move of 10 or 15 points can easily be luck. Treat a change as real only when it is larger than that and lasts more than one month.

4. If you pay for a tool, ask how it counts

AI visibility tools ask a list of questions on a schedule and count how often you are named. That can save time, but a tool is only as good as its method. Before paying, ask:

• Which questions does it ask, and can I see and edit them? Are they worded the way my customers talk, with my towns?
• How many times is each question asked each month, and does the report show the number of answers behind every figure?
• Does it ask through a signed-in app, signed out, or through developer access ³? From what location?
• Does it show a range for each figure, or only a single score?
• Does it report a “rank” inside AI answers? Positions shift so much between runs that they mean little ¹.

Be wary of anyone who guarantees a spot in AI answers. None of the companies behind these assistants publishes how it chooses which businesses to name, so nobody outside them can promise one.

5. Set up visit tracking

• In Google Analytics, check the AI Assistant channel each month ⁹. Look for the source chatgpt.com, which ChatGPT adds to its links ¹⁰. Check where visits from claude.ai show up, since Google’s examples don’t name Claude.
• In Google Search Console, watch clicks in the Web search report. They include AI Overviews and AI Mode, mixed in with ordinary results ⁷.
• In your Google Business Profile, open Performance each month and write down calls, website clicks and direction requests ¹¹. These matter most for Google AI Mode.

6. Ask every new customer how they found you

This will be your most reliable record. Add “How did you hear about us?” to your contact form, with “ChatGPT or another AI assistant” as a choice next to Google, a referral and the rest. Ask on every call and write the answer down. Leave out your own test visits and calls, and any from staff or your web company.

7. Log every change with its date

Write down the date each change goes live, such as a new service page, an updated Business Profile or a batch of new reviews. Then compare the months before and after on all three records. Assistants and their models change too, so an improvement may not be yours, and a drop may not be your fault.

When the share of answers naming you, the visits and the booked jobs all rise together, that is a much stronger sign than any one number alone. Even then, it shows what happened, not proof of why.

Questions owners ask

What does an AI visibility tool actually measure?

Most ask ChatGPT, Google’s AI and other assistants a list of questions on a schedule and count how often your business is named or linked. That shows how often you appear for those exact questions. It doesn’t show visits or customers, and the results depend on the questions, the number of runs and the kind of account the tool uses.

Is there a free way to check my AI visibility?

Yes. Write ten to forty questions your customers ask, naming your towns. Ask each one in ChatGPT, Claude and Google AI Mode in a new, private conversation on several different days. Record whether you were named each time. The share of answers that name you is your visibility for those questions.

How do I track traffic from ChatGPT in Google Analytics?

Open the AI Assistant channel in Google Analytics, which groups visits from tools like ChatGPT. ChatGPT also adds utm_source=chatgpt.com to the links it sends. Some visits still arrive with no source, for example when someone types your web address after reading an answer, so ask new customers how they found you too.

Can I see traffic from Google AI Overviews and AI Mode?

Not on its own. Google counts clicks from AI Overviews and AI Mode in your overall Google search traffic in Search Console, and Google Analytics counts them as organic search. For AI Mode, also watch your Business Profile calls and direction requests, since its answers often link to Google’s own business listing.

What is a good AI visibility score?

There is no standard score. A useful figure is the share of answers that name you, for a fixed set of questions, on each assistant, with the number of answers shown. Compare it with your own past results, not with another tool’s score, and ignore any rank inside AI answers.

How many times should I ask the same question?

More often than you might think. Answers change from run to run, so a single check proves little. Ask each question on several different days, in several wordings, so you have a few dozen answers per assistant each month. Even then, treat a change smaller than 10 to 15 points as possible luck.

See it on your own website

Send us your website address and we’ll make a free preview of your homepage, rewritten from this research. You pay only if you accept it: $2,000 a month, month to month, with no build fee.

Get your free homepage preview

Source: Merchant Context: the current website offer · checked 2026-10-14.

Keep reading

• AI search optimization for home service businesses.