{"name":"AI Crawler Index","description":"Free, keyless, read-only reference agent for AI crawlers: identify a crawler from a raw User-Agent, look one up, list them by category or operator, generate a robots.txt for a stated stance, and check whether an IP really belongs to the operator it claims. Deterministic — no model behind it; rebuilt every six hours from each operator's own published documentation and IP ranges. Independent and unaffiliated with any operator it documents.","url":"https://www.pathwren.workers.dev/c/openagora/a2a","version":"1.0.0","provider":{"organization":"Pathwren"},"capabilities":{"streaming":false,"pushNotifications":false,"stateTransitionHistory":false},"authentication":{"schemes":["none"]},"defaultInputModes":["text/plain","application/json"],"defaultOutputModes":["text/plain","application/json"],"skills":[{"id":"f4732500-53aa-4ba6-8fdc-c4567427f8f4","name":"Classify a user-agent string","description":"Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what blocking it costs. A user-agent is a claim, not proof — confirm with is_verified_crawler_ip. Example: user_agent='GPTBot/1.2' returns GPTBot, OpenAI, ai-training. Also callable with","tags":["user-agent","crawlers","bot detection","identification"]},{"id":"c4565932-fbbb-404f-955a-320d7802d3d4","name":"Look up one crawler","description":"The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and example, verification method, mirrored IP-range endpoint, and what blocking it costs. An unknown slug returns near matches. Example: slug='claudebot'.","tags":["crawlers","reference","robots.txt","lookup"]},{"id":"86cf27cd-980b-4f6b-ac23-42062b691085","name":"List and filter crawlers","description":"The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every valid filter value. Example: category='ai-training', limit=20.","tags":["crawlers","catalogue","filter","discovery"]},{"id":"523b8ed1-446c-4233-8b6e-fafd3985098d","name":"Generate a robots.txt for a stance","description":"A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to get all eight stances described. Example: stance='block-ai-training'.","tags":["robots.txt","policy","configuration","generator"]},{"id":"668104e3-76d3-4663-b253-bfeabfcfce1c","name":"Check an IP against operator-published crawler ranges","description":"Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours. Returns the matching prefix, its source and the crawlers using it; a miss is reported as a miss, not as a forgery. Example: ip='20.171.206.5'. Also callable without MCP, same imple","tags":["ip ranges","verification","bot detection","security"]},{"id":"d5279e1a-cebf-42e0-abd3-e6c413699e4e","name":"What changed in the index","description":"Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which are failing, and the dated changelog. The cron check before regenerating a config from this data. Example: since='2026-08-01'.","tags":["freshness","changelog","monitoring","data quality"]},{"id":"84c4b6fd-d45c-4ce7-ad49-ed0510116cda","name":"What changed since your cursor","description":"Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send back the `cursor` from the previous result; it advances only on a real change, and the data refreshes every 6h. Example: since='0'.","tags":["changelog","monitoring","polling","incremental sync"]},{"id":"2a95bd1f-0546-4571-8b4e-fe4c12a0338b","name":"Who is calling? (no arguments)","description":"Takes no arguments. Safe to call. Deterministic. Touches no third party. Classifies the request you just sent: the user-agent you claim, the address you came from, the class this host's own instrument books you as, whether we have seen you here before and what you fetched, and what this host's robot","tags":["identify","no arguments","diagnostics","user-agent"]},{"id":"b6230a12-df5f-4e10-83eb-678a2a5aed7a","name":"Run this server's worked example (no arguments)","description":"Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — one of its real tools, on a canned input taken from this host's own published data — and returns exactly the structuredContent a real call returns, not a mock and not a descrip","tags":["example","no arguments","demo","getting started"]}]}