Your Website May Be Invisible to AI — The JavaScript Problem

For years, “JavaScript SEO” meant making sure Googlebot could see what your customers see. Most modern websites solved that problem well enough. They invested in client-side frameworks, headless commerce, and rich interactive experiences, confident that Google’s rendering stack would keep up. AI crawlers have not kept up. Most of the systems now mediating discovery — GPTBot, ClaudeBot, PerplexityBot, and their peers — do not behave like Googlebot. They fetch raw HTML, follow a limited set of links, and often do little or no JavaScript execution. To them, a modern single-page app (SPA) or heavily scripted page can look like a bare skeleton: a shell with almost no content inside.1 If your product, category, or help content only “exists” after your JavaScript runs, a meaningful share of AI crawlers may never see it at all. In other words: your website can be highly visible to Google and still be largely invisible to AI. For brands that treat AI discovery as an emerging channel, this is not a technical detail. It is whether the channel can see the surfaces where you are trying to compete.

The New Visibility Gap: Google vs. AI Crawlers

Traditional search created a single reference point for technical teams: if Googlebot could crawl, render, and index a page, you were safe. Google invested heavily in JavaScript rendering; SPAs and headless architectures were a manageable problem. AI crawlers have different constraints. Across multiple technical analyses, the same patterns emerge:

• Most AI crawlers fetch only static HTML. They request a URL once, grab the HTML, and do limited or no client-side rendering.1

• Client-side content often disappears. Text, product details, and structured information rendered via JavaScript — carousels, accordions, dynamic product tiles, filters — may not be visible in the HTML snapshot those crawlers receive.

• Critical journeys look thin or empty. Single-page applications, complex headless commerce builds, and JS-heavy templates can appear as empty containers with a logo, a nav bar, and very little body content. A 2025 analysis of JavaScript rendering capabilities across 23 major AI crawlers found that 69% cannot execute JavaScript, meaning dynamic content — product listings, user-generated data, and real-time updates — is entirely invisible to the majority of AI discovery systems.2 From the crawler’s perspective, it is as if portions of the modern web barely exist.

Why This Matters for AI Discovery

If your site is hard for AI crawlers to interpret, two things happen:

• Your content is less likely to be ingested into the model’s view of your brand. AI systems rely on what they can see and parse at scale. If your product pages, category explanations, and help content are buried behind JavaScript, they contribute less — or not at all — to how AI systems understand your brand and your category.

• You are at a disadvantage when AI systems curate answers. When a customer asks “Which product is best for me?” or “What are the top options for this use case?”, AI systems synthesize from sources they can easily read, extract, and cross-reference. If your site is effectively opaque to their crawlers, the system will lean more heavily on competitors, review sites, and third-party explainers that are easier to ingest. The business consequence is clear: you can run an excellent website for humans and for Google — and still be structurally under-represented in AI answers. In a world where AI increasingly shapes the consideration set, that is not a front-end problem. It is a growth problem.

A Layered Solution Set

1. Fix the Fundamentals (Everyone Should Do This)

These steps are low-effort and broadly applicable:

• Audit AI crawler access. Check your robots.txt and firewall rules for GPTBot, ClaudeBot, PerplexityBot, and similar user-agents. Many organizations have inadvertently blocked them via copy-pasted disallow rules or generic bot protections.

• Expose critical content in static HTML. Ensure that product names, key attributes, pricing blocks, category descriptions, and important FAQs appear in the raw HTML that loads on first request — not only after JavaScript execution. That may mean duplicating the core text of tabs or accordions in a static block, rendering key product tiles server- side instead of purely client-side, or providing an HTML fallback for dynamic components.

• Strengthen structured data and entity clarity. Add schema markup for products, organizations, FAQs, and other important entities, and make sure names, IDs, and attributes are consistent. When AI crawlers see partial content, clean structured data can still give them enough signal to understand what the page represents. These fundamentals do not guarantee AI visibility. But without them, you are asking AI systems to evaluate you in a channel where they literally cannot read the surfaces you are optimizing.

2. Use Architectural Fixes Where They Actually Help

For many modern stacks, fundamentals are necessary but not sufficient. This is where architectural approaches come in:

• Edge / CDN-level pre-rendering. Use Cloudflare Workers, Vercel Edge, Akamai, or similar to intercept requests from known AI user- agents and serve them a pre-rendered HTML snapshot of the page. This approach keeps your existing SPA experience for humans, gives AI crawlers a stable content-rich HTML version, and avoids deep changes to core application code.

• Selective server-side rendering (SSR) or static generation. You don’t need to rebuild the world. Start with a small set of high-value routes — product listing pages, product detail pages, key comparison or solution pages — and render them server-side or via hybrid static generation. That way, the pages that matter most for AI discovery are always HTML-complete at first load.

• Static equivalents of interactive components. For critical modules (filters, configurators, multi-step flows), consider exposing a simplified HTML version that AI crawlers can read, even if the full experience remains interactive for users. These approaches tackle the core issue — what AI crawlers can see — without fragmenting your web presence or creating a parallel site to maintain.

3. Companion / “Shadow” Sites as a Narrow Tool, Not a Doctrine

There are cases where architectural change is impractical:

• The commerce platform is locked down by a vendor.

• The CMS is brittle and cannot be easily adapted.

• Organizational politics or timelines make core refactoring unrealistic in the near term. In those scenarios, a companion content site — a lightweight, static, AI- optimized layer that mirrors key information — can be a pragmatic bridge. It can provide AI crawlers with clean, structured representations of your products, categories, and brand story, and can live under a subdomain or separate domain with a narrow, clear purpose: being machine-legible. But it comes with trade-offs. It is an additional surface to maintain and keep in sync. It can create internal confusion if not clearly positioned as a technical layer, not a new brand destination. And it does not solve underlying architectural debt; it works around it. For that reason, Kairosphere does not treat companion sites as a default solution. They are a tool for constrained environments, not an operating model.

What Fixing the Problem Actually Produces

The business case for addressing AI crawler visibility is no longer theoretical. Early deployments of AI-optimized content delivery are producing measurable results in production environments. The approach — intercepting AI crawler traffic and serving a clean, JavaScript-free version of key pages while leaving the human experience entirely unchanged — is now being applied by enterprise brands at scale. Akamai Technologies, one of the world’s leading cybersecurity and cloud computing companies, applied this approach to a subset of pages tied to its Inference Cloud product — a high-consideration purchase where extensive pre-purchase research is expected. Working with Scrunch, an AI brand monitoring and optimization platform, Akamai deployed AI-optimized page delivery across those pages and measured results against untreated pages over six weeks.3 The results were specific:

• 85% more total AI citations

• 38% more unique prompts generating citations

• 364% increase in brand presence for non-branded prompts — the discovery queries where customers evaluate options before committing to any brand

• 133% increase in brand presence versus competitors on ChatGPT alone These are not traffic numbers. They are eligibility numbers — evidence that AI systems were previously unable to read the site’s content adequately, and that making it legible changed what the systems recommended.

Two observations worth drawing from this.

First, the magnitude of the gap. A 364% increase in non-branded prompt presence implies substantial prior invisibility — not a marginal rendering issue but a structural one. For brands running modern JavaScript-heavy architectures, the assumption that AI crawlers are seeing what Google sees is likely wrong in ways that compound over time as AI discovery grows.

Second, the approach preserved the human experience entirely. AI crawlers receive what they need; human visitors receive the full interactive experience. The two audiences have fundamentally different requirements, and the right solution reflects that rather than forcing a compromise.

The investment case follows directly. If AI-mediated discovery is increasingly where high-intent customers encounter brands, and if the primary technical barrier to that discovery is a solvable rendering problem, then solving it is not a cost of doing business. It is an investment in channel access.

Connecting the Technical Problem to the Channel Strategy

From a distance, the JavaScript problem can look like a niche technical issue. It is not. It is a channel issue.

If AI systems cannot reliably see your site:

You are starting from behind in the competition for eligibility — the probability that AI will include you in a shortlist when a high-value prompt occurs.

Your investments in ecosystem influence, reviews, publishers, communities, and AI-aligned content are competing with one hand tied behind your back, because the primary source of truth about your products is partially invisible.

The goal of fixing JavaScript visibility is not to chase another technical checklist. It is to make sure that when you fight the battles that actually matter — the high-intent, AI-mediated decisions your best customers make — the channel can at least see your case.

AI discovery should be managed as a channel. Making your site AI-legible is table stakes for playing that game.

References

1. Vercel, “The Rise of the AI Crawler,” December 2024. Analysis of AI crawler behavior across Vercel’s network, including 569 million GPTBot requests.

2. SearchViu, “AI Crawlers & JavaScript Rendering,” November 2025. Analysis of JavaScript rendering capabilities across 23 major AI crawlers. Finding: 69% of AI crawlers cannot execute JavaScript.

3. Chris Andrews, CEO and Co-Founder, Scrunch. LinkedIn post, April 6, 2026. Documents Akamai Technologies’ deployment of Scrunch’s Agent Experience Platform (AXP) across Inference Cloud product pages and six-week performance comparison against untreated pages.