---
title: "AI Crawler Visibility Checker — GPTBot, ClaudeBot & More, Alp Korpe"
description: "Tests a URL against 13 real crawlers and search bots, including GPTBot, ClaudeBot, and PerplexityBot, by reading robots.txt and llms.txt and sending live…"
url: https://yigitalpkorpe.com/projects/isvisible/
site: yigitalpkorpe.com
---

[Projects](https://yigitalpkorpe.com/projects/) / Web Tools

*Web Tools · Selected work*

# IsVisible.ai

Free AI crawler visibility checker — tests whether GPTBot, ClaudeBot, and 11 others can reach your site.

Stack

-   Crawler Simulation
-   robots.txt & llms.txt Parsing
-   REST API

Links

-   [Try IsVisible.ai ](https://isvisible.ai)

Impact

-   Scores 100/100, Grade A, when run on this site

![IsVisible.ai — Free AI crawler visibility checker — tests whether GPTBot, ClaudeBot, and 11 others can reach your site.](https://yigitalpkorpe.com/_astro/isvisible.CLxIaPRy_ZGk35t.webp)

[IsVisible.ai](https://isvisible.ai) checks whether AI crawlers can actually reach a website. Paste in a URL and it tests against 13 crawlers and search bots: search engines (Googlebot, Bingbot), AI training crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider, Applebot-Extended, meta-externalagent), and AI answer engines (ChatGPT-User, OAI-SearchBot, Claude-User, PerplexityBot). It reads the site’s [robots.txt](https://developers.google.com/search/docs/crawling-indexing/robots/intro) and llms.txt, sends live requests using each crawler’s real user agent, and merges the declared rules with what actually happens to produce a 0-100 visibility score and an agent-by-agent access map.

## Why sites block AI crawlers without knowing it

The Hardal team has been building an AI visibility reporting product, one that shows companies which AI crawlers are actually visiting their site. Working on that, the same gap kept turning up: sites expected to show up when someone asked ChatGPT or Claude about them, with no idea they were blocking the exact crawlers that make that possible. Old robots.txt rules written for a pre-LLM web, a missing llms.txt, a WAF setting nobody had touched in years. None of it shows up unless someone actually simulates the crawlers and checks. IsVisible.ai is the free, standalone check for that specific gap.

## Seeing it work

Run against this site, yigitalpkorpe.com scores 100 out of 100, Grade A: fully accessible to Google Search, Bing/Copilot, Google Gemini and AI Overviews training, GPTBot training, and the rest of the 13 crawlers it checks.

![IsVisible.ai report for yigitalpkorpe.com: 100/100, Grade A](https://yigitalpkorpe.com/_astro/isvisible-score.DL5ycjDu_ZyX2vE.webp)

## How the crawler simulation works

Getting the crawler simulation right took more tweaking than expected. Different bots honor different rules: some respect a wildcard `Disallow`, some only match their exact user-agent string, some ignore robots.txt for training purposes but respect it for live retrieval. Redirects add another layer: a crawler blocked on the final URL but allowed on a redirecting one produces a different result than checking either URL in isolation. How a site responds to a user agent it’s never seen before turned into its own category of edge case, since some servers default to blocking unfamiliar bots rather than allowing them.

## API access

The tool is free with no signup for the standard check, and it also exposes an unauthenticated REST API for checking a site programmatically, useful for teams that want the same crawler-access check as part of a CI pipeline or a recurring monitor, rather than a one-off manual lookup.

## Related projects

-   [Page Density Checker](https://yigitalpkorpe.com/projects/seo-tool/) analyzes the pages a crawler can reach for density and duplication.
-   [AI-Powered Content Engine](https://yigitalpkorpe.com/projects/blog-automation/) automates the production side of the same SEO workflow.

[Previous AI-Powered Content Engine](https://yigitalpkorpe.com/projects/blog-automation/) [Next Daily Reporting Automation](https://yigitalpkorpe.com/projects/daily-reporting/)

---

Canonical HTML: https://yigitalpkorpe.com/projects/isvisible/

Site map: https://yigitalpkorpe.com/sitemap.xml · Summary for LLMs: https://yigitalpkorpe.com/llms.txt · Agent instructions: https://yigitalpkorpe.com/agents.md · All endpoints: https://yigitalpkorpe.com/agents/