pzaino/thecrowler

A Content Discovery and Development Platform. Empowering Cybersecurity, AI, Marketing, and Finance professionals and researchers to discover, analyze, and interact with the web in all its dimensions.

GitHub repository with 55 stars and 11 forks.

Language: Go

Topics: crawler, crawling, golang, indexer, indexing, search-engine, automation, content-detection, content-discovery, cyber-security

Open provider repository

Latest metric snapshot

2026-06-05: 55 stars and 11 forks.

Similar repositories

  1. 1. gosom/scrapemate

    Golang Crawling and scraping framework

    GitHub repository with 200 stars and 26 forks.

    Trending score: 0.35; stars gained: +1; forks gained: +0.

    Language: Go

    Topics: go-framework, golang, web-crawler, web-scraping, crawler, go

  2. 2. PhiFever/AfdianToMarkdown

    爱发电爬虫(afdian.com)

    GitHub repository with 62 stars and 11 forks.

    Trending score: 0.04; stars gained: +0; forks gained: +0.

    Language: Go

    Topics: afdian, crawler, mcp, mcp-server, spider

  3. 3. openclaw/crawlkit

    Shared Go infrastructure for local-first crawler archives.

    GitHub repository with 38 stars and 5 forks.

    Trending score: 0.03; stars gained: +0; forks gained: +0.

    Language: Go

    Topics: crawler, go, infra

Trending in Go

  1. 1. esengine/DeepSeek-Reasonix

    DeepSeek-native AI coding agent for your terminal. Engineered around prefix-cache stability — leave it running.

    GitHub repository with 18,425 stars and 1,092 forks.

    Trending score: 5.71; stars gained: +1,388; forks gained: +87.

    Language: Go

    Topics: agent, agent-framework, ai-agent, ai-coding, cli, coding-agent

  2. 2. alibaba/open-code-review

    Battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in fine-tuned ruleset (NPE, thread-safety, XSS, SQL injection), OpenAI & Anthropic compatible.

    GitHub repository with 2,572 stars and 126 forks.

    Trending score: 4.93; stars gained: +1,116; forks gained: +39.

    Language: Go

    Topics: agent, code-review, code-review-assistant, harness, repository-level-context

  3. 3. ollama/ollama

    Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

    GitHub repository with 173,245 stars and 16,455 forks.

    Trending score: 3.96; stars gained: +222; forks gained: +40.

    Language: Go

    Topics: llama, llm, llms, go, golang, ollama

  4. 4. MatinSenPai/SenPaiScanner

    A light-weight scanner for Cloudflare IPs, written in Golang

    GitHub repository with 1,224 stars and 75 forks.

    Trending score: 3.75; stars gained: +126; forks gained: +5.

    Language: Go

  5. 5. maximhq/bifrost

    Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

    GitHub repository with 5,504 stars and 698 forks.

    Trending score: 3.70; stars gained: +61; forks gained: +12.

    Language: Go

    Topics: ai-gateway, gateway, gateway-services, generative-ai, guardrails, llm

  6. 6. nianzhibai/91

    nine one

    GitHub repository with 947 stars and 284 forks.

    Trending score: 3.40; stars gained: +40; forks gained: +9.

    Language: Go

Trending topic: crawler

  1. 1. firecrawl/firecrawl

    The API to search, scrape, and interact with the web at scale. 🔥

    GitHub repository with 128,993 stars and 7,677 forks.

    Trending score: 4.80; stars gained: +954; forks gained: +49.

    Language: TypeScript

    Topics: ai, ai-agents, ai-crawler, ai-scraping, ai-search, crawler

  2. 2. ScrapeGraphAI/Scrapegraph-ai

    Python scraper based on AI

    GitHub repository with 26,744 stars and 2,486 forks.

    Trending score: 1.91; stars gained: +105; forks gained: +7.

    Language: Python

    Topics: scraping, scraping-python, llm, web-crawler, web-scraping, ai-scraping

  3. 3. us/crw

    Fast, lightweight Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

    GitHub repository with 139 stars and 13 forks.

    Trending score: 0.78; stars gained: +3; forks gained: +2.

    Language: Rust

    Topics: ai, ai-agents, crawler, data-extraction, docker, firecrawl

  4. 4. leadita/tech-stack-datasets

    Open datasets of companies & websites grouped by technologies they use (CSV & JSON). Discover who uses Shopify, Stripe, Woocommerce, HubSpot, and more.

    GitHub repository with 59 stars and 10 forks.

    Trending score: 0.49; stars gained: +2; forks gained: +0.

    Topics: companies, competitive-analysis, crawler, datasets, hubspot, lead-generation

  5. 5. gosom/scrapemate

    Golang Crawling and scraping framework

    GitHub repository with 200 stars and 26 forks.

    Trending score: 0.35; stars gained: +1; forks gained: +0.

    Language: Go

    Topics: go-framework, golang, web-crawler, web-scraping, crawler, go

  6. 6. 0xDarkMatter/flarecrawl

    Flarecrawl CLI — Cloudflare Browser Rendering alternative to Firecrawl, cost-efficient at scale

    GitHub repository with 6 stars and 0 forks.

    Trending score: 0.33; stars gained: +1; forks gained: +0.

    Language: Python

    Topics: browser-rendering, cli, cloudflare, crawler, favicon, firecrawl