AgentsMapBot
Operated by Primordial Aspect · LinkedIn
AgentsMapBot is the crawler of Agents Map, a research project that maps and counts AI agents on the internet: MCP servers, A2A agents, and agents with on-chain identities. It publishes dated counts with their sources and method.
What it does
- It reads public registries of agents, within each source's terms: the Official MCP Registry, at most hourly, and the ERC-8004 identity registry on Ethereum, Base and BNB Chain.
- It checks the MCP endpoints those registries list or declare. It reads only the metadata an MCP server publishes for its clients (server/discover, or initialize and notifications/initialized, then tools/list). It never calls a tool and never tries to log in. Each endpoint is checked at most about once a week.
- It looks for A2A agent cards at /.well-known/agent-card.json and /.well-known/agent.json, only on hosts those registries name, and at card URLs that ERC-8004 registrations declare. It reads cards and never sends a task. Each location is checked at most about once a week.
- It fetches the registration files that ERC-8004 registrations point to, each at most about once a month.
- It is gentle: one request at a time per site, at most one per second, a cap per site per run, and no retries. It connects only over https and only to public addresses.
How to recognise it
Every request carries the User-AgentAgentsMapBot/<version> (+https://primordialaspect.com/agentsmapbot)
and MCP requests name the client "AgentsMapBot".
What it keeps
The metadata it reads, with the time it was read. It never publishes whether an endpoint accepts requests without logging in.
Opting out
Add this to the robots.txt at the root of your host:
User-agent: AgentsMapBot Disallow: /
A narrower Disallow path excludes only matching URLs. AgentsMapBot reads robots.txt once per run, before any other request to a host. Rules for "User-agent: *" are not applied: the URLs it visits were published for agents and their clients, and it reads them as a client would. Name AgentsMapBot to opt out.