For AI assistants
BotLedger for AI assistants
Three ways to use the AI crawler directory and IP verification from an agent.
MCP server
Assistants call the tools directly. Free, no sign-in.
Connect via MCP →REST API
Servers verify requests. 10,000 free checks a month.
Verification API docs →Plain files
Crawlable directory and llms.txt for retrieval.
llms.txt →
Connect via MCP
Let your assistant list AI crawlers, check a log IP and write robots.txt rules. No sign-in.
MCP server URL (Streamable HTTP, no sign-in)
https://botledger.website/api/aibots/mcp- 1
Claude
Settings → Connectors → Add custom connector
Paste the URL and save. No sign-in needed.
- 2
ChatGPT
Settings → Connectors (developer mode)
Add a custom connector with this URL. No sign-in needed.
- 3
Claude Code
Run in a terminal
claude mcp add --transport http botledger https://botledger.website/api/aibots/mcp
Tools
list_ai_botsKnown AI crawlers: operator, kind, robots.txt token, published IP ranges.
verify_bot_ipCheck an IP and user agent: verified, fake, unverifiable or not a bot.
robots_txt_rulesGenerate robots.txt lines blocking chosen AI bots by slug or kind.
get_api_keyEmail you a free REST API key (10,000 checks/month). Only with your OK.
Descriptor: /.well-known/mcp.json
Rules for agents
- Verify by source IP against the operator's published ranges, never by user agent alone.
- Only email an API key to an address the user confirmed is theirs.
- Tell the user a robots.txt rule only asks crawlers to stop; blocking needs server rules.
Humans: try the AI bot log checker, the AI bot directory or the AI robots.txt generator.