I asked ChatGPT to name the best roofer in 16 US cities, 95 times. It never once cited a roofer's website.

TL;DR

• I ran the same two questions ("who is the best roofer / HVAC company in {city}?") 95 times across 16 US cities, three times each, and logged every business named and every source cited.
• ChatGPT cited a business's own website zero times out of 261 sources. Everything came from directories, review sites, Reddit and magazines. Reddit alone was the single biggest source.
• Ask the same question twice and the top recommendation changes about half the time. The first-named business was different in 58% of repeats. So a screenshot of ChatGPT naming your business proves nothing.
• Roofing and HVAC behave completely differently. Of the HVAC companies ChatGPT named, 49% were in Google's top-3 map pack. For roofers it was 7%. Your map pack position may be doing nothing for you in AI.
• Reviews are a gate, not a race. Named businesses had roughly 2.5x the reviews of their local competition, but past that point, having more reviews than the next guy didn't get you named more often.
• What to do: get on the directory lists, be talked about on Reddit, get past roughly 150 reviews and stop counting.

Not once. Across 95 answers and 261 source citations, ChatGPT never linked to an individual roofing or HVAC company's own site. Every source it used was a directory, a review aggregator, Reddit, or a magazine. The businesses it recommended were real, named and linked. None of them got there because of their homepage.

If you're paying someone to optimise your website so AI will recommend you, that's the number to sit with.

Here's what I ran, what I found, and the prediction I got badly wrong.

What I did

Two questions, 16 US cities, three separate times each, through ChatGPT with web search on:

Who is the best roofer in {city}?

Who is the best HVAC company in {city}?

That's 96 queries. 95 returned. The cities ran from Phoenix and Chicago down to Bozeman and Sioux Falls, across nine states, storm-prone and not.

For each answer I recorded every business named, in order, plus every source it cited. Then for eight of those cities I pulled the actual Google map pack and the top 100 businesses by review count in each category, to check whether the businesses ChatGPT liked were the ones Google likes.

Before collecting anything I wrote down four predictions. Three were wrong, one badly. They're at the bottom.

Finding 1: zero own-site citations

261 citations. Here's where they went.

The most-cited domain in the whole study was reddit.com, with 91 citations. Then expertise.com with 45, bestprosintown.com with 25, consumeraffairs.com with 18. Twenty-six domains carried all 95 answers between them.

Businesses still got named, and their websites still showed up as clickable cards in the answer. But as a source, as the thing ChatGPT actually read to work out who was any good, business websites contributed nothing.

Which makes sense the second you say it out loud. Your website says you're excellent. So does every competitor's. That isn't evidence of anything. A Reddit thread where four people in Spokane argue about who actually turned up when they said they would is evidence.

The practical version: if you want AI to name you, the work is on the pages that list you, not the page you own.

Finding 2: roofing and HVAC are not the same game

I expected AI recommendations to broadly mirror Google's map pack, the block of three businesses with the map above it.

Across the 8 cities I checked in detail, 29% of the businesses ChatGPT named were sitting in that top-3 map pack.

But the average hides what actually happened.

Four businesses. Out of 59 roofers named across eight cities, four were in the map pack. In Phoenix, Denver, Boise and Charlotte, not a single one.

I didn't expect a seven-fold gap between two trades on the same platform in the same week, and I want to be careful about the explanation, because I've got one plausible mechanism and no way to test it here.

The typical HVAC company ChatGPT named had around 2,790 reviews. The typical roofer had 210. HVAC has big, old, heavily-reviewed companies that dominate both the map pack and the third-party "best of" lists, so both systems land on the same names. Roofing is more fragmented, with more storm-chasing and more churn, and the listicles ChatGPT reads are full of different companies than the ones Google's map pack surfaces, because the map pack weighs how close you are to the person searching and those lists don't.

If that's right, how much your map pack work carries into AI depends heavily on which trade you're in. If you're a roofer, mostly it doesn't.

Finding 3: ask twice, get a different answer

Of the businesses named the first time round, 65% were still named in both repeat runs. Any two identical runs shared about 63% of their names.

The number I'd put on a slide is this one: the business ChatGPT named first was different in 58% of cases. Same question, same day, same settings.

Asking three times surfaced about 1.4 times as many businesses as asking once. Across the study, 229 different businesses got named, and 78 of them, a third, showed up in exactly one of 95 answers.

City size barely mattered (between 59% and 69% overlap across four size bands) and neither did trade. That's worth noting, because it suggests the churn is just how the system picks, not a symptom of thin data in small towns.

So when a vendor sends you a screenshot of ChatGPT naming your business, they've shown you one roll of the dice. Run it again an hour later and it's close to a coin flip whether the top name is the same. That screenshot isn't proof of anything. The honest way to measure this is a trend across repeated queries over 30 days.

Finding 4: reviews are a gate, not a race

The businesses ChatGPT named had a typical review count of 405. Comparing them against the top 100 businesses by review count in the same city and category, the typical business in that group had 149.

So volume matters. The businesses getting named sat in roughly the top quarter of their local market on review count.

Then I checked the obvious follow-up: among the businesses that did get named, did the ones with more reviews get named more often?

No. Not even slightly. I compared each business's review count against how many times it was named, and the two moved independently. A roofer in Bend with 14 reviews got named all three times. An HVAC company in Boise with 5,913 reviews also got named all three times. If review count were driving how often you appear, that couldn't happen.

Reviews get you into the pool. They don't move you up inside it. Which means there's a point where spending more effort on review volume stops buying you anything here, and based on this data that point is somewhere around 150.

What I predicted, and what I got wrong

I wrote these down before collecting anything.

The map pack one is the one that stings. I've spent two years telling people that solid local SEO carries over into AI search. For HVAC, at 49%, that holds up. For roofing, at 7%, I was wrong, and a week ago I'd have told you the opposite with a straight face.

What I'd actually do with this

Not advice, just where these numbers point.

Get onto the lists that get read. Expertise, BestProsInTown, ConsumerAffairs, ThreeBestRated, your local "best of" sites. Boring, unglamorous, and it's 48% of the citations.

Be worth mentioning on Reddit. Not faking it, which is both against Google's stated guidance and transparently obvious to everyone including the people you're trying to fool. But if nobody in your city's subreddit has ever mentioned your company, you're missing from the single biggest source in this study.

Get past roughly 150 reviews, then stop optimising for volume. The gate is real. The race isn't.

Don't buy AI visibility measured in screenshots. Ask for a 30-day trend across repeated queries, or don't pay.

If you're a roofer, don't assume your map pack position is helping you here. In eight cities, mine says it isn't.

Limitations

Where this study is thin, so you can weigh it properly.

One wording only. "Best roofer in X" is one of many ways to ask, and I have no evidence this transfers to "most reliable" or "cheapest". That's the next test.

Sixteen cities isn't a representative sample of the US. Eight for the map pack and review comparison.

Three runs shows that the answers move around. It doesn't tell you how much they move on average over a longer period.

These are responses pulled through an API. Someone logged into ChatGPT with their own history and location may see something different.

The roofing versus HVAC explanation is my best guess at a mechanism, not something I demonstrated.

All of this shows things happening together, not one thing causing another. Nothing here proves that getting more reviews or joining a directory makes an AI name you.

It's also one point in time. These systems update.

The data

The underlying dataset is 494 rows: every business named in every run, with the sources cited for each answer. If you want to check a number or run your own cut of it, email me and I'll send it over.

Next test is whether changing the wording of the question changes who gets named.

Method: ChatGPT with web search enabled, queried via the DataForSEO API, US location, English, 3 August 2026. Map pack and business listing data from the same source, same date. 96 queries attempted, 95 returned.