Verify your website’s technical readiness for ChatGPT Search, Claude, Perplexity, and AI answer engines. Inspect crawler directives, structured data, and llms.txt.
Verify crawler access for ChatGPT Search, Claude, and Perplexity, inspect JSON-LD schemas, detect /llms.txt, and get actionable recommendations.
The Technical AI Readiness Score is an objective, deterministic 0–100 rating based exclusively on verifiable technical signals. We do not invent rankings or estimate unverified traffic. If a check cannot be performed, it is marked as unavailable.
Many site owners inadvertently block all AI bots, cutting off real-time search engine citations. Our scanner explicitly differentiates between:
OAI-SearchBot, Claude-SearchBot, PerplexityBot
These bots crawl live web pages to synthesize factual answers and display clickable attribution links to users. Allowing them drives referral traffic.
GPTBot, ClaudeBot
These crawlers scrape content to build or fine-tune future LLM base weights. You can choose to disallow these while keeping search bots allowed.
* Note: Allowing AI search crawlers is a necessary technical prerequisite for indexing, but does not guarantee that your website will be cited for any given user query.
Analyzes robots.txt user-agent directives for OAI-SearchBot, Claude-SearchBot, and PerplexityBot. Allowing these bots enables real-time citation retrieval and source linking in answer engines.
Validates structured data schemas (Article, FAQPage, Organization, Product, HowTo). LLMs leverage structured JSON-LD entities to extract factual knowledge without hallucination.
Evaluates whether AI agents can discover new canonical URLs via XML sitemaps and whether an LLM summary file (/llms.txt) exists to provide markdown-first domain context.
Examines single authoritative H1 tags, logical H2 hierarchy, HTML language declarations, and text-to-HTML density to ensure text is extractable without heavy client-side JS dependency.
Measures HTTP status (200 OK), TLS/SSL encryption, and response latency. Secure, high-speed servers ensure AI crawlers do not abort due to handshake timeouts.
As users transition from standard search engine results pages (SERPs) to synthesized AI answers, traditional SEO tactics are being augmented by Generative Engine Optimization (GEO).
Most site administrators conflate training bots with search bots. If your robots.txt contains a blanket Disallow: / for all user-agents, you are actively blocking citation bots like OAI-SearchBot and Claude-SearchBot from retrieving your site when answering live user queries.
The optimal strategy is selective access: disallow foundation model scrapers (e.g., GPTBot, ClaudeBot) if you wish to protect training copyright, while explicitly allowing search citation bots.
Large language models generate text probabilistically, which can occasionally lead to hallucinated dates, prices, or author credentials. Schema.org JSON-LD provides an explicit, deterministic layer of knowledge.
Pages enriched with Article, FAQPage, Product, and Organization schemas give AI search engines unambiguous factual anchor points that increase source confidence.
Similar to how robots.txt directs traditional web crawlers, /llms.txt offers an LLM-friendly index of a domain. It lists key documentation files, tutorials, and summaries in clean, unformatted markdown.
Hosting an /llms.txt file dramatically reduces parsing latency and context window overhead for AI agents interacting with your site.
AI crawlers frequently use lightweight, headless HTTP clients that do not execute complex client-side JavaScript applications. If your core content is hidden behind client-side rendering or lack proper <h1> and <h2> tags, extraction quality suffers.
Ensure your HTML delivers readable, server-rendered text with clear thematic section breaks and descriptive headings.
Read our in-depth guides on optimizing content and technical infrastructure for generative search engines.
Common questions about AI crawlers, robots.txt directives, llms.txt, JSON-LD structured data, and optimizing websites for generative search engines.
Check technical SEO, heading hierarchy, image alt tags, HTTPS redirects, and sitemaps with full diagnostic reports.
Decode JSON Web Tokens, inspect claims, verify HMAC signatures, calculate expiration timers, and audit token security.
Browse our full suite of free developer utilities, SEO checkers, text tools, and image converters.
