Back to all posts
guides 14 min read

SEO KPIs for the AI Era: Measuring Organic and LLM Visibility

Serap Gündoğdu ·
SEO KPIs for the AI Era: Measuring Organic and LLM Visibility

Layer 1: Technical SEO KPIs - Crawlability and AI Bot Access

Why SEO KPIs Are Changing: Organic Search and AI Visibility

Rankings, impressions, and organic sessions were built for a results page where most answers ended in a click. Google’s AI Overviews and AI Mode, ChatGPT search, and Perplexity now compose an answer from several sources and show it inside the interface. A reader can get what they came for without opening your page. Your report then shows flat or falling clicks, and nothing in it explains why.

Rank and click data still describe something real. Search Console shows how often Google displayed your pages and how often people chose them. It cannot tell you whether an AI answer quoted a page, whether an AI crawler could fetch it, or whether a visit came from a chatbot link. Check Google’s current Search documentation on whether AI feature activity gets its own filter in Search Console or is folded into the regular Web data, and don’t build a KPI on a guess.

The practical response is to measure in three layers.

The first layer is technical foundations: crawlability, indexability, rendering, and bot access. It is the prerequisite for the other two. A page blocked in robots.txt, returning a 5xx error, or showing its main content only after JavaScript runs has a problem in classic search and in AI retrieval alike. Crawlers that don’t execute JavaScript see an empty shell.

The second layer is organic search performance: clicks, impressions, click-through rate, average position, index coverage, and the share of clicks that come from non-brand queries.

The third layer is AI visibility: citations in generated answers, brand mentions, referral sessions from AI products, and AI bot requests in your server logs. Some of these you can count exactly. Others you can only sample, because the same question can produce different answers on different runs. For those, a trend over several weeks is a usable number and a single reading is not.

The order matters when you read the results. If citations drop and a robots.txt rule started blocking an AI bot the week before, the cause is in layer one, and no amount of content work in layer three will fix it. Diagnosing from the bottom up saves you from chasing symptoms.

That leaves the question you came with: which KPIs belong in the report, and how do you calculate each one? For every metric below you get the formula, the data source, and whether a tool measures it or you have to sample it by hand.

Layer 1 is the only layer a crawler can measure end to end. Seodisias crawls the site on your own machine, with JavaScript rendering, no URL limit and no sign-up, and the exports give you every ratio below. Search Console data, server logs and LLM citations are outside the crawl and appear in later layers.

Define the denominator before you compute anything. Unless noted, it is all internal HTML URLs the crawl reached.

  • Indexable URL ratio = indexable URLs with status 200 / all crawled HTML URLs. Source: status code, canonical and meta robots columns. A low value on a small site usually points to parameter URLs or stray noindex tags.
  • Status code health = URLs returning 4xx or 5xx / all crawled URLs, reported separately for each class. Add the share of URLs that sit behind a redirect chain of two hops or more, and the longest chain. Source: response codes and redirect data from the crawl.
  • Render gap = pages where the rendered HTML differs from the raw HTML in title, canonical, main content or internal links / pages compared. Source: raw and rendered snapshots of each page. This is the check for content that exists only after JavaScript runs. Which AI bots execute JavaScript differs by provider and changes over time, so read each bot’s current documentation. Content present in raw HTML is readable by every bot.
  • AI bot access matrix = for each user agent (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended), the share of your priority URLs that robots.txt allows. Source: the robots.txt file, evaluated per URL under RFC 9309. Keep training bots, search bots and user-triggered fetchers in separate rows, because blocking one is a different decision from blocking another. The OpenAI bots page documents its own agents; check each vendor’s documentation for the rest.
  • Meta robots conflicts = URLs whose meta robots, X-Robots-Tag header, canonical or sitemap entry contradict each other (for example noindex on a page listed in the sitemap) / crawled URLs. Also count pages with nosnippet or a low max-snippet value, since snippet limits can restrict how Google quotes a page. Confirm the exact effect in Google’s documentation before you report it.
  • Structured data coverage = template pages with valid schema.org markup / template pages that should have it. Count one sample per template, not every URL, or large catalogs will drown the result.
  • Sitemap compliance = sitemap URLs that return 200 and are indexable / all sitemap URLs. Also list indexable crawled URLs missing from the sitemap.
  • Core Web Vitals pass rate = URLs meeting LCP of 2.5 seconds or less, INP of 200 ms or less and CLS of 0.1 or less / URLs measured. A URL passes only when all three pass. Thresholds come from web.dev.
  • AI Ready results = checks passed / checks run, taken from the AI Ready audit. Report the failing checks as a list, since the ratio alone does not tell you what to fix.

llms.txt presence is worth a yes/no line in the report. No measured effect on visibility has been shown for it, so it does not belong in a score.

Compute every ratio once per crawl and keep the raw counts next to each percentage. When a number moves between crawls, the counts show whether the numerator or the denominator changed.

Once the technical layer is measured, the next question is how that access turns into search performance. These metrics predate AI search, and they still anchor every report. AI answers do not replace them. Most also need a segment or a formula before the number means anything.

Search Console Performance report (review weekly)

  • Clicks and impressions. Impressions show demand and visibility, and clicks show what that visibility turned into. Read them together. Impressions rising while clicks stay flat usually means you rank on deeper pages or a SERP feature takes the click.
  • CTR. Clicks divided by impressions. A sitewide CTR hides a lot, so compare it by query group and by page type.
  • Average position. Position averages mislead. A new page ranking 80 for hundreds of long-tail queries can drag the average down while your best pages improve. Filter by page or query group before drawing conclusions.
  • Non-brand click ratio. Non-brand clicks divided by total clicks. Filter the query report with a regex of your brand terms and its inverse. A low ratio means the site mostly gets visitors who already know it, so growth in discovery is not showing up yet.

Ranking distribution (monthly) Search Console does not report this directly. Export query and page data, then count the share of ranking URLs or tracked queries in positions 1-3, 4-10 and 11-20. Pages at 11-20 are the cheapest to push onto page one, so that bucket is worth watching on its own.

Indexing and crawling (weekly, and after every release)

  • Index coverage. Indexed URLs divided by submitted sitemap URLs, taken from the Page indexing report. The reasons listed for excluded URLs tell you what to fix first.
  • Crawl statistics. The Crawl stats report under Settings shows total crawl requests and average response time. A rising response time often arrives before a drop in crawl requests. Check it monthly, or sooner after a migration.

Analytics (monthly)

  • Organic sessions. Sessions from the organic search channel. Compare against clicks from Search Console. The two will not match exactly, but a growing gap points to tracking or consent problems.
  • Organic conversions. Key events attributed to organic sessions. Traffic that never converts is a vanity number, so report conversions per landing page group.

Keep the same definitions every period. Brand regex, date ranges and page groupings should not change from one report to the next, or the trend lines stop being comparable.

Layer 3 depends on the first two: a bot that cannot fetch a page cannot cite it. Its metrics are noisier than the ones above, and some are manual. Citation rate is the first AI visibility KPI, and you calculate it by hand. Pick a fixed set of queries, for example 30 to 50 that match your core topics, and run them on the same platforms (ChatGPT, Perplexity, Copilot) on the same schedule. Citation rate is the number of answers that cite your domain divided by the total number of answers you tested. No tool produces this number for you automatically. Someone has to read the answers and log them in a spreadsheet.

Brand mention ratio uses the same sample. Count the answers that name your brand, with or without a link, and divide by the total. Keep it separate from citation rate. A model that mentions you without linking tells you something different from one that links to a specific page.

Both numbers are noisy. The same query can return different answers an hour apart, and results shift by platform, location and account state. To reduce the noise, run each query two or three times per round and record the average. Even then, treat a single reading as one sample and not a fact.

Referral traffic is the most familiar signal. In your analytics tool, filter sessions by source domains such as chatgpt.com and perplexity.ai, and add Copilot and Gemini domains as you find them in your reports. Referrers are not always passed along. Some apps strip them, and some visits show up as direct traffic. Read the number as a floor. Real AI-driven visits are at least this high.

Server logs give you the most solid direct data in this layer. Count requests per day for each AI bot user agent, the share of those requests that returned a 200, and the number of distinct URLs fetched. A bot that requests only your homepage reaches a different part of your site than one that works through your product pages. Bot names and their purposes (training, search, user-triggered fetches) change, so check each provider’s current documentation before you build filters.

None of these four numbers means much as a single reading. If your citation rate is 12 percent this month, that figure depends on your query list, your platforms and the day you ran the test. If it was 8 percent last quarter and 12 percent now, with the same list and the same method, you have a direction. The same goes for bot requests and referral sessions. Freeze the method, record the date, and compare each round with the previous one.

Write down the sample size next to every figure. A change from 3 of 40 answers to 5 of 40 is within the range of ordinary variation. A rise that continues across three rounds is worth acting on.

Baseline and Crawl: Getting Started with Seodisias

To put the layers into practice, start with a full crawl. Enter the root URL in Seodisias and run it. There is no account, no URL limit, and JavaScript rendering is included. Crawl data stays on your machine, and the app runs on Windows, macOS and Linux. Keep rendering on, because the render gap check needs both the raw and the rendered HTML of each page.

When the crawl ends, export four things: status codes, indexability (with canonical and meta robots), render differences, and the robots.txt result for each AI bot you track. Then run the AI Ready audit on the same site and export it next to the crawl data.

Paste the exports into a spreadsheet with one row per URL, then add a baseline tab with these formulas:

KPIFormula
Indexable URL ratioindexable 200 URLs / all crawled URLs
Error rate(4xx + 5xx URLs) / all crawled URLs
Render gapURLs whose title, canonical, main content or internal links differ between raw and rendered HTML / rendered URLs
AI bot accesscritical URLs allowed for a bot / all critical URLs
Sitemap compliancesitemap URLs that return 200 and are indexable / sitemap URLs

If column B holds the status code and column C the indexability label, the first one is =COUNTIFS(B:B,200,C:C,"Indexable")/(COUNTA(A:A)-1).

The numbers also point to causes:

  • High render gap: content, links or tags only exist after JavaScript runs. Open the flagged pages and check what the script injects. A bot that skips JavaScript sees the raw version.
  • Bot blocked in robots.txt: that bot cannot fetch your critical URLs. Check the Disallow rules and the user-agent group that matches the bot.
  • Low indexable URL ratio: group the non-indexable URLs by reason (noindex, canonical to another URL, redirect, 4xx/5xx). The largest group is where to start. Redirects and errors are crawlability problems, while noindex and canonicals are directive problems.

The citation, log and Search Console metrics from layers 2 and 3 need other sources, so give them their own columns in the same sheet. Date the baseline tab, because every later crawl gets compared against it.

Reporting KPIs: Dashboard, Frequency, and Action Triggers

With the baseline in place, use this one-page template at every report. It has three layers, and each metric has a formula, a source, and a named owner (tool or manual).

LayerKPIFormulaSourceCadenceMeasured by
1 TechnicalIndexable URL ratioindexable 200 URLs / crawled URLsSeodisias crawlMonthlyTool
1 TechnicalRender gappages where raw and rendered HTML differ in title, canonical, main content or internal links / crawled pagesSeodisias crawlMonthlyTool
1 TechnicalAI bot accesscritical URLs open to your target bots / critical URLsrobots.txt checkMonthlyTool
2 OrganicNon-brand click rationon-brand clicks / total clicksSearch ConsoleMonthlyManual export
2 OrganicIndex coverageindexed URLs / submitted URLsSearch ConsoleMonthlyManual export
3 AICitation rateanswers citing you / answers testedFixed query listQuarterlyManual
3 AIAI referral sessionssessions from AI source domainsAnalyticsMonthlyManual, read as a lower bound

Seodisias covers Layer 1 only. Layers 2 and 3 stay manual or in other tools.

Baseline and trend. Run a full crawl and export every metric once. That snapshot is your baseline. Repeat the crawl monthly, then compare each quarter against the baseline, not against last month. For citation rate, use the same queries, the same platforms and the same interval every time. The same question can return different answers, so report the trend across quarters and ignore any single run.

Action triggers. Each trigger maps to one fix.

  • Organic clicks drop: check the Layer 1 indexable URL ratio first, then the status code share, then index coverage. If those hold, look at the render gap. Compare the date of the drop with Google Search Status before you change anything.
  • Non-brand click ratio falls: the drop is in query coverage, so check which query groups lost impressions.
  • Render gap rises: content depends on JavaScript. Move the missing elements into the initial HTML.
  • AI bot access falls: diff robots.txt against the previous crawl and restore the rules for the bots you want.
  • Citation rate falls: confirm the bots can still fetch the cited pages, then check that the pages still return 200 and show their content in raw HTML. Only then revisit the content.

Write the decision next to each KPI in the report. A number with no fix attached stays out of the dashboard.

Looking for something else?

Search every guide on the blog without going back to the list.