All articlesSEO

We Got a Client to the Top of ChatGPT. Ten Days Later I Could Not Reproduce It.

A rebuilt website put a senior placement firm into ChatGPT's answers in two metros. We re-ran the same prompts ten days later. What came back explains why you cannot measure AI visibility by checking once.

We Got a Client to the Top of ChatGPT. Ten Days Later I Could Not Reproduce It.

In late September we finished a rebuild of the website for 1st CHOICE Advisory Services, a senior placement firm that has been helping Northwest families since 1993. Nothing exotic. A technically clean site, clear about who they are and what they do, with their Google Business Profile straightened out to match.

By our kickoff call on October 9, ChatGPT was recommending them.

I screen-shared it on that call because it was genuinely a nice moment. Ask it for the best senior placement agencies in Vancouver, and there they were. Ask it for Portland, separately, and there they were again, with a green "best overall starting point" badge next to their name. They had not been in those answers at all before we started.

Then, writing this article, I did something I had not done on the call. I ran the same two prompts again, on October 19, through a third-party tool with no account and no history attached to it. Twice each.

What came back is the actual lesson, and it is better than the win.

What the re-run showed

Vancouver held. 1st CHOICE came back in the recommended list in both runs, at position two, cited with a link to their site. The rebuild did what it was supposed to do.

Portland did not. In two runs, ten days after I watched them sit at the top of that answer, they were not named at all. Six other agencies were. The green "best overall starting point" badge I had pointed at on the call was there both times, attached to a competitor.

And then the part that stopped me.

In those two Vancouver runs, minutes apart, from the same tool, on the same prompt, ChatGPT described 1st CHOICE two different ways:

  • Run one: 4.4 stars, 9 reviews, linked to their old domain.
  • Run two: 5.0 stars, 271 reviews, linked to their current one.

Same company. Same question. Same machine. Ten minutes apart. A 30-fold difference in the review count a family would see.

271 is the real number. They have earned it the hard way over three decades. But one in two families asking that question got told 9.

This is why your own screen is the wrong instrument

About once a month a client sends me a screenshot. They typed their own community into ChatGPT, did not see themselves, and the message has a certain tone to it.

I understand the panic. The answer is always the same: that result was assembled for you, and it tells you almost nothing.

AI search is hyper-personalized. It is skewed on an individual basis. Your location, your history, your previous conversations, what the model infers about your intent, all of it shapes the answer before it reaches you. Location alone is enough to throw it: if your phone registers on the southeast side of Portland, the model answers as though you are standing there.

But the Vancouver runs show something past personalization, because there was no person in them. No account. No history. No preferences to skew anything. And the answer still changed, on a factual detail, inside ten minutes.

So when you search yourself and come up empty, you have learned one thing: that this model, at this moment, did not name you. That is not a ranking. It is not even a measurement.

Eric raised a possibility on our Monday call that I cannot rule out either: the model knows your history, knows you are connected to that company, and may be assuming you already know that answer, so it offers you something else. We do not know. It is one more reason not to use your own screen as the scoreboard.

He also made the right comparison. In the early days of keyword ranking, nobody really knew what made a page rank, and everyone had a confident theory anyway. We are in that phase again. I am suspicious of anyone who tells you otherwise.

What we do instead

We do not run our clients' names through our own machines and call it reporting.

The tooling we use runs each prompt 20 to 25 times, across several different language models, and builds a baseline from the spread. One answer is an anecdote. Twenty-five answers across multiple models, repeated monthly, is a number you can watch move.

That is the difference between "I did not see us in ChatGPT this morning" and "your mention rate across buying-intent prompts went from this to that." The first is noise. The second you can manage against.

It is also why we report AI mention rate on a monthly cycle rather than on request. A single check, on a single day, by a person the model already has opinions about, is the least reliable version of this that exists. If I had only taken the October 9 screenshot, I would be telling this client they own Portland. They do not.

AI does not hallucinate about you. It believes bad data about you.

Most people get this backwards, and it changes what you should actually do.

When an AI says something wrong about a community, people assume the model invented it. In our experience it almost never did. It found something, and it trusted it.

The 9-reviews answer is a clean example. ChatGPT did not make up a number. It read a Google business record that carries their old domain and a fraction of their reviews, and it reported what it read. The other run read the current record and reported 271. Two real sources, two different truths, one of them years out of date.

We have a senior living client with two properties where I have watched the same thing: AI stating things that are simply not true, and every time we traced it, the source was real. An out-of-date description on a directory. A line in some listing from years ago saying they accepted a payer they no longer accept.

Which is good news, because a hallucination is unfixable and a bad citation is just a chore.

The re-run turned up a second one of those chores, which is why I am glad I did it. One run cited a subdomain of their site that families are not meant to be reading. It is reachable, so the model read it, so it counts as them. Nobody would have found that by looking at the website.

Old content outranks your new content, and that is a job

Eric made a point on the Monday call that reframed this for me.

AI leans on older material, especially material that has accumulated comments or engagement, because that activity reads as credibility. Your carefully updated article from last month has none of that yet. The thin post from 2019 with a comment thread does.

So the model keeps citing the old thing. Not because it is better, but because it looks more established.

That turns AI visibility into maintenance work rather than publishing work. When we find an old page a client is being referenced for, the question is not "what should we write next." It is one of three:

  • Does this need updating with current information?
  • Does this need to be noindexed?
  • Does this need a 301 redirect somewhere better?

AI will not tell you which. It is referencing, not advising. Somebody has to go and look.

Where senior care AI answers actually come from

No surprises in the big names. Care.com, Caring.com, A Place for Mom. The model can produce those out of a hat.

Two things did surprise me.

The ugly directories punch far above their weight. In assisted living specifically there are a handful of advisor and listing sites that are genuinely powerful sources, and they are some of the least attractive websites on the internet. Design has nothing to do with whether a model trusts a source.

Local still matters, including the boring local. Both Vancouver runs cited the state association's member directory as a source. Not a marketing asset. A membership list. Your chamber of commerce, regional associations, the organizations you already belong to and never thought of as marketing. We are currently curious whether membership directories like BNI function as a linking authority, which is not something I had considered until this conversation.

Daniel added the one I would have underrated: social profiles feed this too, and a Facebook page that has been quiet since 2023 is a data point about you whether you like it or not.

Press releases are the genuinely contradictory one. We have seen AI pull from press releases. We have also run a press release and watched nothing happen. Both are true, so we do not yet have a rule for it, and I would rather say that than pretend otherwise.

What to do this week

Do not open ChatGPT and search for yourself. You already know how that goes.

Instead, pick the three things a family would actually ask before choosing you. Not your name. The question that comes before your name.

Then ask somebody outside your organization, in a different city, to run them. Then have them run them again an hour later. The gap between those two results, never mind the gap with yours, is the whole point of this article.

After that the work is unglamorous and it is the work: make sure your Google Business Profile and the directories all say the same true thing, including the ones you forgot you had, and go clean up the old pages that are still speaking for you.

Greatness Digital is a digital marketing agency in Oregon that works only with businesses that serve seniors: senior living communities, home care agencies, placement advisors, elder law firms, and senior move managers.

Want to know what families actually see when they ask about you? Let's measure it properly.

Frequently asked questions

Why doesn't ChatGPT show my business when I search for it?
Often because the result was personalized for you. AI search factors in your location, your history, and what it infers about your intent, so your own screen is not a reliable sample of what families see. Results also vary between sessions with no person involved at all: we ran the same prompt twice through a neutral tool ten minutes apart and got different facts about the same company both times.
How do you actually measure AI visibility?
By running the same buying-intent prompts many times across several models and tracking the baseline over time, rather than checking once. We run each prompt 20 to 25 times across multiple models and report mention rate monthly, because one answer on one day is an anecdote rather than a measurement.
Does AI hallucinate about senior living communities?
Less often than people assume. In our experience the wrong information usually traces back to a real source: an outdated directory listing, a duplicate business profile, an old description. In one run ChatGPT reported a client as having 9 reviews when they have 271, because it read a stale record carrying their previous domain. That is bad data being trusted, not invention, which means it can be fixed at the source.
What sources does AI pull from for senior care?
The major directories like Care.com, Caring.com and A Place for Mom, plus a surprising amount from small assisted living advisor and listing sites with no design value at all. State and regional association member directories came up in our own testing. Chambers of commerce and social profiles carry weight too. Press releases are inconsistent in our experience.
Why does AI keep citing my old article instead of my new one?
Older pages that have accumulated comments and engagement read as more established, even when the information is stale. The fix is maintenance rather than publishing: update the old page, noindex it, or redirect it somewhere current.

Want this kind of thinking on your brand?

We build the websites, campaigns, and AI systems senior-serving organizations use to grow. Let's talk about yours.

Let's Talk