AI Search Optimization, Part 4: Measuring What the Dashboard Can't See

The last three parts were all about doing things — front-loading answers, keeping schema honest, publishing an llms.txt. This part is about the uncomfortable question that follows all of it: how do you know any of it worked?

Here's the problem in one sentence. You can't improve what you can't see, and Google Search Console can't see AI search at all. GSC is the dashboard every SEO habit is built around, and for the entire category of "did an LLM cite my page," it is simply blind. So you need other instruments. We run two, and they're deliberately complementary — one you can automate, one you can't.

Why GSC is blind

Search Console reports Google blue-link impressions and clicks. That's it. When Google shows an AI Overview above the ten blue links, or when someone gets their answer from ChatGPT or Perplexity or Gemini and never touches Google at all, none of that appears in your Performance report. There's no "AI Overview impressions" tab. Third-party engines are, obviously, not Google's to report on either.

So the most-watched dashboard in the discipline has a growing hole in the middle of it, and that hole is AI search. Every signal below exists to fill part of that hole. Neither fills all of it — that's why there are two.

Signal one — referral click-throughs (automatable)

When someone clicks a link inside an AI answer, the resulting visit usually arrives with a referrer. Perplexity cites its sources with clickable links; ChatGPT's browsing answers do too; Copilot and Gemini attach citations. Click one, land on our page, and the request carries a referrer domain like perplexity.ai or chatgpt.com.

We already capture the referring domain of every visit — and only the domain. Never the full URL, never the path, never the query string. It gets folded into an aggregated daily counter, and that behaviour is disclosed in our privacy policy. This matters twice: it keeps us honest with visitors, and it means the AI-referral signal is a free byproduct of analytics we already run rather than some new tracking apparatus.

Given that we already have the domain, catching AI engines is just a lookup table:

export const AI_SEARCH_REFERRERS: Record<string, string> = {
  'chatgpt.com': 'ChatGPT',
  'perplexity.ai': 'Perplexity',
  'gemini.google.com': 'Gemini',
  'copilot.microsoft.com': 'Copilot',
  'claude.ai': 'Claude',
  // …you.com, poe.com
};

export function aiSearchEngine(domain: string): string | null {
  return AI_SEARCH_REFERRERS[domain] ?? null;
}

We already send ourselves a daily digest email with the day's traffic metrics, so the natural home for this was one more line in that digest. It now prints:

AI search referrals yesterday: N — Perplexity 4, ChatGPT 2, Gemini 1

...or none yet when the count is zero.

One implementation detail worth calling out, because it's the kind of thing that quietly gives you wrong numbers. The digest's general "top referrers" section only lists the top 15 domains by volume. At our current scale, an AI engine that sent three clicks can easily fall outside that top 15 — buried under social and search referrers — and if you compute the AI total by scanning the top-15 list, you'll undercount and sometimes report a flat zero on a day that wasn't. So we query the AI-referral total explicitly against the full referrer set, not by filtering the display list. The number is right even when no AI engine cracks the visible leaderboard.

Now the honest limitation. This signal only catches citations that got clicked. An AI answer that names you, quotes your number, summarises your method — and doesn't get clicked, or links you without the user following it — is completely invisible to this instrument. And a huge fraction of AI answers are exactly that: the whole value proposition of an AI answer is that the user doesn't have to click through. So referral click-throughs are a real signal, but they're the visible tip of a much larger iceberg. Which is why there's a second instrument.

Signal two — the citation panel (manual)

The only way to see the un-clicked citations is to go ask the engines directly. There's no API that reports "you were mentioned in 12,000 answers this month." You have to look.

So we keep a fixed list of roughly 30 target queries — our winnable long-tails, the exact phrases each tool page is built to own — and every month or so we sit down for about twenty minutes and paste each one into ChatGPT, Perplexity, and Gemini. For each query we log three things:

  1. Were we cited? Named, linked, or quoted — yes or no.
  2. Who was cited instead? Which competitors are showing up in the answer we want.
  3. Did the engine state a wrong fact? If an AI confidently gives a wrong number for something we have a page about, that's a page we can sharpen to own the answer — a content to-do, not just an observation.

That last one is the sleeper. Wrong-fact corrections are the highest-leverage output of the whole panel: the engine has told you exactly where the authoritative answer is missing, and Part 2's front-loaded-answer work is the fix. This is the ground truth no dashboard will ever hand you — twenty minutes of manual looking beats any amount of inferring from referrer logs.

Concretely, pages like our test grade calculator and classroom timer each have a cluster of queries we run through the panel — "how many can I get wrong and still pass," "fullscreen classroom timer no ads," that sort of thing. When an engine answers those with a competitor's page or a wrong threshold, we know precisely what to fix next.

Putting it together

Three things are worth tracking over time:

  • AI-referral sessions — the digest number, watched as a trend. Is it growing month over month? A rising line means the click-through-worthy citations are increasing.
  • Citation share vs competitors — from the panel. Of your 30 queries, on how many are you cited at all, and how does that move?
  • Wrong-fact corrections — logged as a content backlog. Each one is a concrete page-improvement task straight from Part 2's playbook.

And a note on expectations, because it's easy to demoralise yourself here. At pre-launch or low-traffic volume, the AI-referral line in the digest will read 0 / none yet for a good while. That's not a bug and it's not failure — that zero is your baseline. The entire point of instrumenting this early is so that when the first Perplexity click lands, you see it, and you have a dated starting point to measure the climb from. You can't watch a number move if you never started watching.

The on-page work being measured here — the front-loaded answers and drift-proof schema — is Part 2. Measurement without that work is just watching a flat line; the work without measurement is flying blind. You want both.

Next, in Part 5, we get to the part that's hardest for competitors to copy and hardest for an AI to answer around: the tool-invocation moat — why a page that does something beats a page that merely says something.