AddSearchBot
AI2Bot
Ai2Bot-Dolma
aiHitBot
Amazonbot
Andibot
anthropic-ai
Applebot
Applebot-Extended
Awario
bedrockbot
bigsur.ai
Brightbot 1.0
Bytespider
CCBot
TerraCotta
ChatGPT Agent
ChatGPT-User
Claude-SearchBot
Claude-User
Claude-Web
ClaudeBot
CloudVertexBot
cohere-ai
cohere-training-data-crawler
Cotoyogi
Crawlspace
Datenbank Crawler
Devin
Diffbot
DuckAssistBot
Echobot Bot
EchoboxBot
FacebookBot
facebookexternalhit
Factset_spyderbot
FirecrawlAgent
FriendlyCrawler
Gemini-Deep-Research
Google-CloudVertexBot
Google-Extended
Google-Firebase
GoogleAgent-Mariner
GoogleOther
GoogleOther-Image
GoogleOther-Video
GPTBot
iaskspider/2.0
ICC-Crawler
ImagesiftBot
img2dataset
ISSCyberRiskCrawler
Kangaroo Bot
LinerBot
meta-externalagent
Meta-ExternalAgent
meta-externalfetcher
Meta-ExternalFetcher
meta-webindexer
MistralAI-User
MistralAI-User/1.0
MyCentralAIScraperBot
netEstate Imprint Crawler
NovaAct
OAI-SearchBot
omgili
omgilibot
OpenAI
Operator
PanguBot
Panscient
panscient.com
Perplexity-User
PerplexityBot
PetalBot
PhindBot
Poseidon Research Crawler
QualifiedBot
QuillBot
quillbot.com
SBIntuitionsBot
Scrapy
SemrushBot-OCOB
SemrushBot-SWA
ShapBot
Sidetrade indexer bot
Thinkbot
TikTokSpider
Timpibot
VelenPublicWebCrawler
WARDBot
Webzio-Extended
wpbot
YaK
YandexAdditional
YandexAdditionalBot
YouBot