$0.50 per result

AI Access Audit

Check whether AI crawlers such as GPTBot, ClaudeBot and PerplexityBot can access your site. Audits robots.txt rules, llms.txt and AI access policy.

Run it on Apify All tools

Who it is for

Anyone who wants their content quoted by AI assistants rather than skipped by them.

What you would use it for

• Find out whether GPTBot, ClaudeBot and PerplexityBot are allowed to read the site at all.
• Catch a robots.txt rule that blocks AI crawlers by accident — a common side effect of copy-pasted boilerplate.
• Check for an llms.txt file and whether the page carries the metadata an assistant needs to cite it.
• Audit a portfolio of client sites in one run and rank them by score.

What you get back

One JSON record per page checked. These are the actual fields returned by the most recent successful run — read from the live catalogue, not a documented ideal.

Sample output

Trimmed from a real run.

{
  "actor": "artifact-machine-ai-access-audit",
  "actorVersion": "0.1.0",
  "auditedAt": "2026-07-29T05:03:21.813Z",
  "url": "https://neighbornest.kids/",
  "origin": "https://neighbornest.kids",
  "score": 20,
  "grade": "F",
  "issueCounts": {
    "critical": 0,
    "high": 2,
    "medium": 2,
    "low": 0,
    "total": 4
  },
  "checks": {
    "robotsTxt": {
      "exists": false,
      "score": 0,
      "blockedSearchBots": 0,
      "blockedUserAgentBots": 0,
      "trainingBotsWithPolicy": 0,
      "wildcardDisallow": false
    },
    "llmsTxt": {
      "exists": false,

Pricing

$0.50 per result. Pay-per-event through Apify — no subscription, and a run that returns nothing costs nothing.

Calling it from code

Any HTTP client works. Replace <TOKEN> with your Apify API token.

curl -X POST "https://api.apify.com/v2/acts/artifact-machine~artifact-machine-ai-access-audit/run-sync-get-dataset-items?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://example.com"}]}'

Questions

Does blocking AI crawlers affect normal search rankings?

No. GPTBot, ClaudeBot and PerplexityBot are separate user agents from Googlebot and Bingbot. Blocking the AI crawlers does not change classic search indexing — but it does mean AI assistants cannot read or cite the page.

What is llms.txt?

A proposed plain-text file at the root of a site that tells large language models which pages matter and how to read them. It is a convention rather than a standard, and it is not required — this audit reports whether one exists rather than treating its absence as a failure.

Other tools

Schema Audit - JSON-LD Structured Data Validator · Security Headers Checker · Sitemap Health Audit - XML Sitemap Validator · Tech Stack Analyzer - Detect Website Technologies

Details read from the live Apify catalogue on 2026-07-29.