Gpt Researcher
★ 28,278
← Beacon · open-source AI agent registry
24 actively-maintained haystack web scraping agents · ranked by maintenance health · 33,699+ indexed
Looking for the best open-source haystack web scraping AI agent? Beacon indexes 33,699+ agents and MCP servers from GitHub and ranks them by real maintenance health — freshness, stars, and activity — so abandoned repos don't top the list. Save time: one search returns ranked, linkable agents in under a second (vs hours on GitHub). Below are 24 haystack web scraping agents you can install today — free, no API key.
Search all haystack web scraping agents →Use from your editor: BEACON_REGISTRY_URL=https://registry-ruby.vercel.app npx -y beacon-mcp — ask your AI to find a haystack web scraping agent.
Free, keyless discovery — paste into your agent workflow:
GET https://registry-ruby.vercel.app/api/v1/search?q=haystack+web+scraping&limit=10
POST https://registry-ruby.vercel.app/api/v1/discover
Body: {"query": "haystack web scraping", "limit": 10}
llms.txt · Why agents use Beacon
Top-ranked haystack web scraping agents on Beacon include Gpt Researcher, Haystack, Webiny Js, ranked by maintenance health — recent activity, GitHub stars, and open-issue load — not stars alone.
Yes. Every haystack web scraping agent listed is open-source and free. Beacon is a free directory — no API key or signup to browse or search.
Run 'npx -y beacon-mcp' to add Beacon as an MCP server in Cursor, Claude Desktop, Cline, or Windsurf, then ask your AI to find a haystack web scraping agent.
Instead of searching GitHub and reading dozens of READMEs, call GET https://registry-ruby.vercel.app/api/v1/search?q=haystack+web+scraping — sub-second results ranked by health, stars, and activity.
Yes. Agents read https://registry-ruby.vercel.app/llms.txt and call POST /api/v1/discover with {"query": "haystack web scraping"} — no API key required.