LEYUN · SITE SPEC & COMPONENT LIBRARY独立站技术规范手册

Fixing Blank Pages Served to AI Crawlers Caused by JavaScript Rendering

Section:CRAWL & INDEX · Reviewed:2026-11-20

Key takeawayWhen page body content is injected client-side by JavaScript and the crawler does not execute JS, both Googlebot and AI crawlers receive an empty HTML shell. The telltale sign: no body text can be found in the page source, and the page appears blank in AI-generated summaries.

When this applies

  • Product and content sites built with client-side frameworks such as React, Vue, Nuxt, or Next.js
  • Pages where body text, specifications, or prices are inserted after useEffect or client-side requests complete
  • Pages whose source code contains no body text while the browser renders them correctly
  • Pages from which AI search only extracts the title and description
How rendering methods affect what crawlers see
Rendering methodBody text seen by the crawlerRisk levelRemedy
Pure client-side rendering (CSR)Essentially emptyHighSwitch to server-side rendering or static generation
Client-side hydrationEmpty or partial, depending on injection timingMediumEnsure key content is already output on the server
Server-side rendering (SSR)CompleteLowMaintain and monitor caching behavior
Static generation (SSG)CompleteLowKeep build output consistent with CDN cache
Deferred loading (lazy injection)Empty above the fold, content appears laterHighMove core content out of the lazy-loading scope
Bot-specific renderingContent differs by user agentHighUse with caution; easily judged as cloaking

Step-by-step diagnosis

  1. Fetch the target page with curl, save the response to a file, and check whether it contains core text such as product names and specifications.
  2. Identify the rendering method by looking for framework markers in the source head (__NEXT_DATA__, window.__NUXT__, etc.).
  3. Prefer server-side rendering or static generation so core content lives in the HTML source instead of being injected client-side.
  4. If client-side rendering is unavoidable, at least output above-the-fold essentials (title, specifications, first paragraph) from the server.
  5. Check whether any "hide content from bots" logic or user-agent-based rendering exists, and remove it if so.
  6. In GSC, use Live Test to view what Googlebot actually renders and confirm the body text is readable.
  7. Spot-check responses for AI crawler user agents (e.g., GPTBot, PerplexityBot) the same way, confirming they no longer receive empty shells.

Common mistakes

  • Assuming Google will always execute JavaScript - Googlebot does render, but rendering is a delayed, resource-limited step; key content should still live in the source.
  • Placing important content exclusively behind collapsed panels and tab switches, where crawlers are likely to miss it.
  • Serving different content to crawler user agents - that is a cloaking risk that Google explicitly rejects.
  • Judging by what the browser renders - you must check the source code and the HTML crawlers actually receive.
  • Overlooking AI crawlers: most, including GPTBot and ClaudeBot, do not execute JavaScript and depend on the raw source even more than Googlebot.

FAQ

Does Googlebot execute JavaScript?
It renders, but as a deferred process with limited resources and a rendering queue delay. The reliable approach in technical SEO is still to keep core content present in the HTML source.
Why do AI crawlers more often get blank pages?
Most answer engines' crawlers neither execute JavaScript nor perform deferred rendering; they read the raw response only. So the same page can show a snippet in Google results yet be blank in AI answers - a common phenomenon.
After moving to SSR, what else should be watched?
Watch CDN caching behavior: if the cache stores a user-state or shell response, crawlers still receive the wrong version.
Version notes2026-11-20: first release, compiled from official Google documentation and public technical sources; updated as engine rules change. Engine behaviour details follow the latest official Google documentation.

Note: this page is a summary of publicly documented search-engine rules and is for self-check reference only. Engine behaviour is subject to the latest official Google documentation; verify against your own Search Console data.