JavaScript rendering and AI crawlers: make sure your content is visible
Why JavaScript-only content can be invisible to some crawlers, how to test your pages, and the rendering approaches that keep them readable by search and AI.
The problem in one sentence
If your page’s important content isn’t in the HTML the server sends, you are relying on every crawler to run your JavaScript — and not all of them do it reliably.
How different crawlers handle JavaScript
- Googlebot renders JavaScript using a recent version of Chromium, but Google describes rendering as a separate step that can happen after the initial crawl.
- Bingbot also renders JavaScript, with its own capacity limits.
- Other crawlers, including various AI-product crawlers, generally do not publish detailed rendering capabilities. Many tools that fetch pages for AI answers prioritise speed and may work mostly from raw HTML.
Because rendering behaviour is not documented consistently across AI crawlers, the safe assumption is: content that matters should be in the initial HTML.
What tends to go missing
- Product descriptions and prices loaded from an API after page load
- Headings and body text in single-page applications without server rendering
- Links inside JavaScript-only navigation
- Tab and accordion content fetched on click
- Reviews and FAQs injected by third-party widgets
How to test your pages
- View source (not “inspect”). Search for a key sentence from the page. If it isn’t in the source, it isn’t in the initial HTML.
- Fetch with curl:
curl -s https://example.com/page/ | grep -i "your key sentence". - Disable JavaScript in your browser and reload. What’s left is roughly what a non-rendering crawler sees.
- Use Google’s URL Inspection tool to see Google’s rendered HTML and compare.
Rendering approaches compared
| Approach | Content in initial HTML? | Notes |
|---|---|---|
| Static site generation (SSG) | Yes | Fastest; ideal for marketing and content pages |
| Server-side rendering (SSR) | Yes | Good for frequently changing or personalised pages |
| Hybrid / islands | Yes, for static parts | Interactive components hydrate on top of HTML |
| Client-side rendering only | No | Risky for any page you want found and cited |
| Dynamic rendering | Partly | Google describes it as a workaround, not a long-term solution |
This site, for example, is statically generated: every heading, paragraph, FAQ and link is in the HTML before any JavaScript runs. Interactive parts — like the AI answer demo on our homepage — add behaviour on top.
What to fix first
- Templates for your most important pages (services, products, pricing, key guides).
- Navigation and internal links — crawlers discover pages through them.
- Structured data generated server-side, matching visible content.
- Third-party widgets that hold important content (reviews, FAQs) — consider server-rendered alternatives.
Rendering is one part of crawler access; the other is permission. See AI crawlers and robots.txt.