GitHub · DIGITAL Command Language
scrapy/protego
A pure-Python robots.txt parser with support for modern conventions.
- Stars
- 98
- Forks
- 43
- Stars gained in 24 hours
- +4
- Forks gained in 24 hours
- +8
Topics: hacktoberfest, python, robots-parser, robots-txt
Latest data point: Nov 21, 2026 · 98 stars · 43 forks