ArchiveTeam/wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
GitHub repository with 139 stars and 19 forks.
Language: C
Topics: archiveteam, archiving, crawl, crawler, crawlers, crawling, downloader, ftp, lua, scraper