Fixing Blank Pages Served to AI Crawlers Caused by JavaScript Rendering
Key takeawayWhen page body content is injected client-side by JavaScript and the crawler does not execute JS, both Googlebot and AI crawlers receive an empty HTML shell. The telltale sign: no body text can be found in the page source, and the page appears blank in AI-generated summaries.
When this applies
- Product and content sites built with client-side frameworks such as React, Vue, Nuxt, or Next.js
- Pages where body text, specifications, or prices are inserted after useEffect or client-side requests complete
- Pages whose source code contains no body text while the browser renders them correctly
- Pages from which AI search only extracts the title and description
| Rendering method | Body text seen by the crawler | Risk level | Remedy |
|---|---|---|---|
| Pure client-side rendering (CSR) | Essentially empty | High | Switch to server-side rendering or static generation |
| Client-side hydration | Empty or partial, depending on injection timing | Medium | Ensure key content is already output on the server |
| Server-side rendering (SSR) | Complete | Low | Maintain and monitor caching behavior |
| Static generation (SSG) | Complete | Low | Keep build output consistent with CDN cache |
| Deferred loading (lazy injection) | Empty above the fold, content appears later | High | Move core content out of the lazy-loading scope |
| Bot-specific rendering | Content differs by user agent | High | Use with caution; easily judged as cloaking |
Step-by-step diagnosis
- Fetch the target page with curl, save the response to a file, and check whether it contains core text such as product names and specifications.
- Identify the rendering method by looking for framework markers in the source head (__NEXT_DATA__, window.__NUXT__, etc.).
- Prefer server-side rendering or static generation so core content lives in the HTML source instead of being injected client-side.
- If client-side rendering is unavoidable, at least output above-the-fold essentials (title, specifications, first paragraph) from the server.
- Check whether any "hide content from bots" logic or user-agent-based rendering exists, and remove it if so.
- In GSC, use Live Test to view what Googlebot actually renders and confirm the body text is readable.
- Spot-check responses for AI crawler user agents (e.g., GPTBot, PerplexityBot) the same way, confirming they no longer receive empty shells.
Common mistakes
- Assuming Google will always execute JavaScript - Googlebot does render, but rendering is a delayed, resource-limited step; key content should still live in the source.
- Placing important content exclusively behind collapsed panels and tab switches, where crawlers are likely to miss it.
- Serving different content to crawler user agents - that is a cloaking risk that Google explicitly rejects.
- Judging by what the browser renders - you must check the source code and the HTML crawlers actually receive.
- Overlooking AI crawlers: most, including GPTBot and ClaudeBot, do not execute JavaScript and depend on the raw source even more than Googlebot.
FAQ
Does Googlebot execute JavaScript?
It renders, but as a deferred process with limited resources and a rendering queue delay. The reliable approach in technical SEO is still to keep core content present in the HTML source.
Why do AI crawlers more often get blank pages?
Most answer engines' crawlers neither execute JavaScript nor perform deferred rendering; they read the raw response only. So the same page can show a snippet in Google results yet be blank in AI answers - a common phenomenon.
After moving to SSR, what else should be watched?
Watch CDN caching behavior: if the cache stores a user-state or shell response, crawlers still receive the wrong version.
Version notes2026-11-20: first release, compiled from official Google documentation and public technical sources; updated as engine rules change. Engine behaviour details follow the latest official Google documentation.
Related reading
Why a Site Burns Through Crawl Budget: Causes and an Optimization ChecklistComplete Troubleshooting Guide for "Crawled - Currently Not Indexed" in Google Search Console"Discovered - Currently Not Indexed" in GSC: Causes and FixesServer Log Analysis for Independent Sites: Quickly Locating Why Crawlers Stay AwayDynamic URL Parameters Flooding Google's Index with Low-Quality PagesNote: this page is a summary of publicly documented search-engine rules and is for self-check reference only. Engine behaviour is subject to the latest official Google documentation; verify against your own Search Console data.