---
title: AI Crawl Control
pcx_content_type: overview
tags:
  - AI
sidebar:
  order: 1
head:
  - tag: title
    content: Overview
description: Monitor and control how AI services access your website content.
products:
  - ai-crawl-control
---

import {
	Description,
	Feature,
	Plan,
	LinkButton,
	RelatedProduct,
	Card,
	Stream,
} from "~/components";

<Plan type="all" />

<Description>

Monitor and control how AI services access your website content.

<Stream
	id="c2f3d8aada64a53e6cc118e5af834601"
	title="Introduction to AI Crawl Control"
	thumbnail="1m37s"
/>

</Description>

AI companies use web content to train their models and power AI applications. AI Crawl Control (formerly AI Audit) gives you visibility into which AI services are accessing your content, and provides tools to manage access according to your preferences.

With AI Crawl Control, you can:

- **See which AI services access your content** - Monitor the dashboard to see crawler activity and request patterns
- **Control access with granular policies** - Set allow or block rules for individual crawlers
- **Monitor robots.txt compliance** - Track which crawlers follow your directives and create enforcement rules
- **Explore monetization options** - Set up pay per crawl pricing for content access [(private beta)](/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/)
- **Deploy with zero configuration** - Works automatically on all Cloudflare plans

<LinkButton href="/ai-crawl-control/get-started/">Get started </LinkButton>

---

## Features

<Feature
	header="Manage AI crawlers"
	href="/ai-crawl-control/features/manage-ai-crawlers/"
	cta="Manage AI crawlers"
>
	Control how AI crawlers interact with your domain.
</Feature>

<Feature
	header="Analyze AI traffic"
	href="/ai-crawl-control/features/analyze-ai-traffic/"
	cta="Analyze AI traffic"
>
	Gain insight into how AI crawlers are interacting with your pages.
</Feature>

<Feature
	header="Track robots.txt"
	href="/ai-crawl-control/features/track-robots-txt/"
	cta="Track robots.txt"
>
	Track the health of `robots.txt` files and identify which crawlers are
	violating your directives.
</Feature>

<Feature
	header="Pay Per Crawl"
	href="/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/"
	cta="Pay per crawl"
>
	Allow AI crawlers to access content by paying per crawl.
</Feature>

---

## Use cases

<Card title="Publishers and content creators">
	Publishers and content creators can monitor which AI crawlers are accessing
	their articles and educational content. Set policies to allow beneficial
	crawlers while blocking others.
</Card>

<Card title="E-commerce and business sites">
	E-commerce and business sites can identify AI crawler activity on product
	pages and business information. Control access to sensitive data like pricing
	and inventory.
</Card>

<Card title="Documentation sites">
	Documentation sites can track how AI crawlers are accessing their technical
	documentation. Gain insight into how AI crawlers are engaging with your site.
</Card>

---

## Related Products

<RelatedProduct header="Bots" href="/bots/" product="bots">
	Identify and mitigate automated traffic to protect your domain from bad bots.
</RelatedProduct>

<RelatedProduct header="Web Application Firewall" href="/waf/" product="waf">
	Get automatic protection from vulnerabilities and the flexibility to create
	custom rules.
</RelatedProduct>

<RelatedProduct header="Analytics" href="/analytics/" product="analytics">
	View and analyze traffic on your domain.
</RelatedProduct>