AI Visibility

AI Visibility

Ranking in Google is half the job now. The other half is being readable by the assistants people ask instead of searching — and a surprising number of WordPress sites are invisible to them without knowing it.

Contexta → AI Visibility → Access Check answers the question directly.

Why a robots.txt editor isn't enough

Classic SEO plugins let you edit robots.txt. That tells you what you wrote — not what a crawler actually receives.

The common failure has nothing to do with robots.txt: a CDN or firewall blocks the request before WordPress is ever reached. Cloudflare's AI crawler controls do exactly this, and no amount of editing robots.txt reveals it.

So Contexta verifies outcomes instead of settings.

What the check does

Click Run AI Access Check. It takes about 15 seconds and runs four tests.

1. robots.txt rules

Parses your live robots.txt and reports, per crawler, whether it is allowed. Nine crawlers are checked:

If any are blocked, Allow all AI crawlers fixes it in one click.

2. Live bot fetch test

The important one. Contexta requests your homepage from your own server using the real user-agent strings of GPTBot, OAI-SearchBot and PerplexityBot, and reports the HTTP status that actually comes back.

• Reachable (200) — the crawler gets your page
• Blocked / Challenged by CDN — something between the internet and WordPress is refusing it, and the panel names the CDN it detected

If Cloudflare is blocking, the fix lives outside WordPress: Cloudflare dashboard → AI Crawl Control (or Security → Bots) and allow AI crawlers. Contexta tells you this explicitly because no plugin can change it for you.

3. Index visibility

Signals that would remove your whole site from both search results and AI answers:

• Search engine visibility — the WordPress setting that adds a site-wide noindex
• Homepage meta robots — a noindex from your SEO plugin or theme
• X-Robots-Tag header

Each has a one-click fix where WordPress permits one. When the noindex comes from your SEO plugin rather than core, Contexta says so and points you at the right settings screen — it will not silently fight another plugin.

4. Content without JavaScript

AI crawlers generally do not execute JavaScript. If your content only appears after JS runs — common with JS-rendered themes and some page builders — assistants see an empty page. Contexta fetches one of your articles and checks whether the main content exists in the raw HTML.

llms.txt

llms.txt is an emerging convention: a plain-text index at your domain root that tells AI assistants what your site is about and which pages matter.

Generate llms.txt from your top content builds one from your proven Search Console top pages, with AI-written descriptions for each.

Details worth knowing:

• It is served virtually at /llms.txt — no file is written to disk, so it works on locked-down hosts and cannot be lost during a deploy.
• If a real llms.txt file already exists, or another plugin serves one, Contexta steps aside and tells you rather than creating a conflict.
• Without an API key it still generates, using your existing page titles instead of AI-written descriptions.
• Regenerate llms.txt refreshes it after your content or Search Console data changes.

How often to run this

The access check is worth running monthly, and any time you change host, add a CDN, or change firewall rules. A Cloudflare setting flipped by a well-meaning admin can make your site invisible to every AI assistant overnight, and nothing in WordPress will warn you.