How we research what AI assistants recommend

1. What we asked
2. What an answer can and can’t tell you
3. How we count
4. How to use the research
5. Questions owners ask
6. How we got these numbers
7. See it on your own website

What we asked

Between September 2 and October 14, 2026, we asked ChatGPT, Claude and Google AI Mode to recommend a local contractor 47,209 times: ChatGPT 19,577, Claude 16,358 and Google AI Mode 11,274.

The questions covered four trades (HVAC, plumbing, electrical and roofing), 147 specific jobs and 45 towns: 16 in Pinellas County, 16 in Pasco County and 13 in Hillsborough County. Most of the answers, 46,350, were about towns in Pinellas County. 41,561 answers were about one of the 147 jobs; the rest asked for the trade in general.

Each question was worded the way a homeowner might type it, naming a town or ZIP code and often a time, such as “tonight” or “next month”. Each one was asked in a fresh conversation with web search on, so no answer was shaped by an earlier one.

We kept only answers that searched the web and came back complete. Questions about marketing agencies and question sets written for one client were left out.

We collected the answers with software, not through the apps customers use. ChatGPT answered through OpenAI’s Codex command-line tool and Claude through Anthropic’s Claude Code, both with web search on and the same model families as the apps. Google AI Mode answers came from DataForSEO, a search-data service that returns Google’s AI Mode results for a chosen location. The apps have their own instructions and search tools, so a customer can get a different answer. What carries over is which details and which kinds of sites the assistants relied on.

The models changed during the period: ChatGPT answered as gpt-5.5 (11,613) and gpt-5.6-luna (7,979); Claude answered as claude-sonnet-5 (10,420) and claude-sonnet-5-5 (5,940); Google AI Mode answered as gemini-3.5-flash (11,290).

What an answer can and can’t tell you

These are observations of answers, not experiments. When an answer says it couldn’t find a price, that tells you what was missing for that question on that day. It doesn’t prove that adding a price would change who gets recommended.

We asked the same questions many times, so the counts show what we tested, not how often real customers search. A question asked a hundred times is one question asked a hundred times, not a hundred customers.

Assistants change how they search and answer. Every post gives the dates of its answers, and an older post describes the assistants as they were then.

How we count

What an answer mentioned

For each service post, a writer read that job’s answers and named the things they kept returning to, each with a few phrases that mean it. Code counts how many answers contain any of those phrases. The writer never types a number; every count on a post comes from the code.

What an answer said it couldn’t find

We look for sentences in which the assistant says it couldn’t find, confirm or verify something, then sort them by what was missing. Across the 41,561 job answers, 6,407 (15%) said this about at least one company. By subject:

• A phone number or other way to reach the company: 3,851 answers
• Reviews it could check: 1,563 answers
• When the company could do the work: 956 answers
• Whether the company serves the customer’s town or ZIP code: 755 answers
• A license or insurance it could check: 662 answers
• A price or how the job is priced: 501 answers

Almost all of these came from one assistant. Claude said it couldn’t confirm something in 6,144 of its 14,467 job answers; ChatGPT in 241 of 17,275 and Google AI Mode in 22 of 9,819. As we asked it, Claude usually tells the customer what it couldn’t check; the others seldom do. So these counts describe what Claude couldn’t find, not how often every assistant was missing something.

The sorting is a set of word rules, so we checked it by hand. We read 20 random sentences for each subject and counted how many were sorted correctly: a phone number 19 of 20, the service area 19 of 20, availability 19 of 20, reviews 18 of 20, a license or insurance 17 of 20 and a price 15 of 20. The price counts are the least exact: some sentences in that group say a company stated it had no extra fee.

Where an answer linked

We group each linked page by the kind of site it is on: a contractor’s own website, a directory or review site, a Google page, an official license lookup, a booking page or a guide. Sites outside the named groups are counted as contractors’ own websites; a manual check of the 120 most-linked of them found almost all were.

Google AI Mode is reported separately. In 7,242 of its 9,819 job answers, the links opened Google’s own listing for a business rather than a website.

The earlier study of why businesses were left out

Before this research, we coded assistant conversations from late July to August 23, 2026, across eight trades: HVAC, plumbing, electrical, roofing, landscaping, dental, residential cleaning and auto repair. For each business an assistant considered, we recorded the reasons it gave for including it or leaving it out.

That produced 1,251 recorded reasons, many of them close variations of the same idea. For each, we count the conversations where it came up and how many of those it was the reason a business was left out. For example, whether the business served the area asked about came up in 554 conversations and cost a business its place in 350 of them. One conversation can involve several reasons, so the counts overlap and are never added together.

Quotes

Quotes are the assistants’ own words, copied exactly, with the question and the date. We only publish a sentence that names no business, person or website and contains no phone number, license number, rating or link.

How to use the research

The research is most useful one job at a time:

1. Open the service post for a job you want more of.
2. Read what the assistants told customers to ask, and answer each question on your page in plain words.
3. Fix what the assistants said they couldn’t find, starting with a phone number and the towns you serve.
4. Note the date the page changed.
5. Afterward, ask the same questions again and watch calls and form requests. That is the only way to know whether the change helped your business.

Questions owners ask

Do you name the businesses the assistants recommended?

No. The posts report what the answers checked, asked and couldn’t find. They never rank or name the contractors the assistants recommended, and quotes that name a business are left out.

Is this a ranking of the best contractors?

No. It describes how the assistants answered, not which company is best. An assistant’s choice depends on what it could find and confirm on the day it searched.

Why ChatGPT, Claude and Google AI Mode?

They are widely used, and each one searches the web before answering a question like these. That makes their answers a record of what a contractor’s pages and listings showed on that day.

How current is the research?

The answers in this snapshot run between September 2 and October 14, 2026. We keep asking, and each post states the dates of the answers it reports.

Will following the research guarantee a recommendation?

No. Nobody can promise that. The research shows what the assistants were looking for. Whether supplying it changes recommendations for a business is something to measure after the change goes live.

See it on your own website

Send us your website address and we’ll make a free preview of your homepage, rewritten from this research. You pay only if you accept it: $2,000 a month, month to month, with no build fee.

Get your free homepage preview

Source: Merchant Context: the current website offer · checked 2026-10-14.