projectdiscovery/katana

A next-generation crawling and spidering framework.

GitHub repository with 16,952 stars and 1,135 forks.

Language: Go

Topics: crawler, web-spider, gocrawler, spider-framework, cli, headless, hacktoberfest

Open provider repository

Latest metric snapshot

2026-06-05: 16,952 stars and 1,135 forks.

Similar repositories

  1. 1. gosom/scrapemate

    Golang Crawling and scraping framework

    GitHub repository with 200 stars and 26 forks.

    Trending score: 0.35; stars gained: +1; forks gained: +0.

    Language: Go

    Topics: go-framework, golang, web-crawler, web-scraping, crawler, go

  2. 2. pzaino/thecrowler

    A Content Discovery and Development Platform. Empowering Cybersecurity, AI, Marketing, and Finance professionals and researchers to discover, analyze, and interact with the web in all its dimensions.

    GitHub repository with 55 stars and 11 forks.

    Trending score: 0.05; stars gained: +0; forks gained: +0.

    Language: Go

    Topics: crawler, crawling, golang, indexer, indexing, search-engine

  3. 3. PhiFever/AfdianToMarkdown

    爱发电爬虫(afdian.com)

    GitHub repository with 62 stars and 11 forks.

    Trending score: 0.04; stars gained: +0; forks gained: +0.

    Language: Go

    Topics: afdian, crawler, mcp, mcp-server, spider

  4. 4. openclaw/crawlkit

    Shared Go infrastructure for local-first crawler archives.

    GitHub repository with 38 stars and 5 forks.

    Trending score: 0.03; stars gained: +0; forks gained: +0.

    Language: Go

    Topics: crawler, go, infra

Trending in Go

  1. 1. esengine/DeepSeek-Reasonix

    DeepSeek-native AI coding agent for your terminal. Engineered around prefix-cache stability — leave it running.

    GitHub repository with 18,252 stars and 1,087 forks.

    Trending score: 5.71; stars gained: +1,388; forks gained: +87.

    Language: Go

    Topics: agent, agent-framework, ai-agent, ai-coding, cli, coding-agent

  2. 2. alibaba/open-code-review

    Battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in fine-tuned ruleset (NPE, thread-safety, XSS, SQL injection), OpenAI & Anthropic compatible.

    GitHub repository with 1,979 stars and 108 forks.

    Trending score: 4.49; stars gained: +545; forks gained: +21.

    Language: Go

    Topics: agent, code-review, code-review-assistant, harness, repository-level-context

  3. 3. ollama/ollama

    Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

    GitHub repository with 173,212 stars and 16,447 forks.

    Trending score: 3.96; stars gained: +222; forks gained: +40.

    Language: Go

    Topics: deepseek, gemma, gemma3, glm, go, golang

  4. 4. kubernetes/kubernetes

    Production-Grade Container Scheduling and Management

    GitHub repository with 122,695 stars and 43,255 forks.

    Trending score: 3.87; stars gained: +65; forks gained: +21.

    Language: Go

    Topics: kubernetes, go, cncf, containers

  5. 5. MatinSenPai/SenPaiScanner

    A light-weight scanner for Cloudflare IPs, written in Golang

    GitHub repository with 1,179 stars and 71 forks.

    Trending score: 3.75; stars gained: +126; forks gained: +5.

    Language: Go

  6. 6. avelino/awesome-go

    A curated list of awesome Go frameworks, libraries and software

    GitHub repository with 174,609 stars and 13,287 forks.

    Trending score: 3.72; stars gained: +196; forks gained: +8.

    Language: Go

    Topics: awesome, awesome-list, go, golang, golang-library, hacktoberfest

Trending topic: crawler

  1. 1. firecrawl/firecrawl

    The API to search, scrape, and interact with the web at scale. 🔥

    GitHub repository with 128,853 stars and 7,670 forks.

    Trending score: 4.80; stars gained: +954; forks gained: +49.

    Language: TypeScript

    Topics: ai, ai-agents, ai-crawler, ai-scraping, ai-search, crawler

  2. 2. CharlesPikachu/videodl

    Videodl: A lightweight video downloader written in pure python. (轻量级视频下载器，优先高清无水印，支持抖音，快手，小红书，B站，TikTok，YouTube，FIFA+，优酷，腾讯，爱奇艺，1905电影网，乐视，芒果，咪咕，PPTV，搜狐，Facebook，Twitter，新浪微博，今日头条，网易公开课，全民K歌，CCTV央视频，酷狗音乐MV，新片场，知乎，百度贴吧，TED等海量流媒体平台)

    GitHub repository with 2,207 stars and 397 forks.

    Trending score: 2.34; stars gained: +24; forks gained: +2.

    Language: Python

    Topics: video-downloader, cntv, douyin, crawler, migu, bilibili

  3. 3. ScrapeGraphAI/Scrapegraph-ai

    Python scraper based on AI

    GitHub repository with 26,738 stars and 2,486 forks.

    Trending score: 1.91; stars gained: +105; forks gained: +7.

    Language: Python

    Topics: ai-crawler, ai-scraping, ai-search, crawler, data-extraction, firecrawl-alternative

  4. 4. apify/crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    GitHub repository with 23,677 stars and 1,417 forks.

    Trending score: 1.27; stars gained: +20; forks gained: +0.

    Language: TypeScript

    Topics: apify, automation, crawler, crawling, headless, headless-chrome

  5. 5. 0x676e67/wreq

    An ergonomic Rust HTTP Client with TLS fingerprint

    GitHub repository with 842 stars and 109 forks.

    Trending score: 1.21; stars gained: +3; forks gained: +0.

    Language: Rust

    Topics: akamai, client, crawler, fingerprint, http, http-client

  6. 6. 0xMassi/webclaw

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

    GitHub repository with 1,287 stars and 148 forks.

    Trending score: 1.20; stars gained: +17; forks gained: +0.

    Language: Rust

    Topics: ai-agents, cli, llm, markdown, mcp, rust