ai-crawlers repositories

Trending repositories tagged ai-crawlers, ranked by trending score. Updated daily.

  1. 1
    aivisibilitycensus/geo-skills

    9 open Claude skills for GEO / AEO: check if AI search can read a page, collect and analyze how often AI engines name a brand (with confidence ranges), plus schema parity, AI crawler log checks and migration guards. From the AI Visibility Census.

    GitHub · Python · 5 stars · 4 forks · +2 stars in 24 hours · Trending score 0.60

  2. 2
    George-Tsoukakis/ScrapeLock

    ScrapeLock – Content & AI Bot Protection for WordPress: blocks scrapers, AI training bots and fake Googlebots, rate limits, token-protected XML feeds.

    GitHub · PHP · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.10

  3. 3
    codevibe-technology/ai-bot-access

    Can AI crawlers, search engines and link-preview bots (GPTBot, ClaudeBot, PerplexityBot, Googlebot, LinkedInBot and more) really read your site? robots.txt plus a real request per bot, share-preview check, bot IP verification and access-log reports.

    GitHub · TypeScript · 6 stars · 5 forks · 0 stars in 24 hours · Trending score 0.07

  4. 4
    spcaeo/skald

    Self-hosted Google Analytics alternative that reads your server access log: real people, AI assistants, crawlers, bots and the scanners probing your site, which JavaScript analytics never see. Leads and revenue by source. Every figure counted twice. No cookies, no third-party script. Python, SQLite, Docker.

    GitHub · Python · 22 stars · 2 forks · 0 stars in 24 hours · Trending score 0.05

  5. 5
    rubenmarcus/aeo.js

    Answer Engine Optimization for the modern web. Make your site discoverable by ChatGPT, Claude, Perplexity & AI search engines. Generates llms.txt, robots.txt, sitemap, JSON-LD & more.

    GitHub · TypeScript · 149 stars · 17 forks · 0 stars in 24 hours · Trending score 0.00

  6. 6
    tianpengg/geo-monitor

    Self-hosted AI crawler monitoring from Nginx / OpenResty access logs. FastAPI + Vue, source verification, page coverage and collector health.

    GitHub · HTML · 99 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00