ai-crawler repositories
Trending repositories tagged ai-crawler, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
GitHub · TypeScript · 205,629 stars · 10,754 forks · +2,082 stars in 24 hours · Trending score 5.92
-
2
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
GitHub · Python · 32,236 stars · 3,302 forks · 0 stars in 24 hours · Trending score 0.00
-
3
puneetindersingh/open-seo-crawler
Free SEO crawler, website audit and AI crawler checker. Self-hosted and open source: crawls a site for technical SEO issues and checks that Googlebot, Bingbot and AI crawlers (GPTBot, PerplexityBot, OAI-SearchBot) get the same readable page. CMS-aware, XLSX export, free Screaming Frog alternative.
GitHub · Python · 65 stars · 20 forks · 0 stars in 24 hours · Trending score 0.00
-
4
makralabs/makra
Makra is a memory layer between the web and your AI agents. It serves real-time structured data from the open web at the cost of a vector search.
GitHub · Python · 30 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00