data-collection repositories
Trending repositories tagged data-collection, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl-mcp-server
Official Firecrawl MCP server, the context API for AI agents: search, scrape, and interact with the web, plus parse, crawl, map, Agent and Monitor, from any MCP client.
GitHub · JavaScript · 7,860 stars · 939 forks · +13 stars in 24 hours · Trending score 2.43
-
2
airbytehq/airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
GitHub · Python · 22,411 stars · 5,550 forks · +5 stars in 24 hours · Trending score 2.32
-
3
jitsucom/jitsu
Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
GitHub · TypeScript · 5,137 stars · 405 forks · +1 stars in 24 hours · Trending score 1.25
-
4
cporter202/API-mega-list
This GitHub repo is a powerhouse collection of APIs you can start using immediately to build everything from simple automations to full-scale applications. One of the most valuable API lists on GitHub—period. 💪
GitHub · JavaScript · 8,403 stars · 1,640 forks · −1 stars in 24 hours · Trending score 1.18
-
5
NaiboWang/EasySpider
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
GitHub · JavaScript · 44,801 stars · 5,393 forks · +2 stars in 24 hours · Trending score 1.11
-
6
worldbench/awesome-embodied-data-pyramid
🔥 Data Pyramid for Embodied Manipulation: A Survey
GitHub · Python · 212 stars · 9 forks · 0 stars in 24 hours · Trending score 1.10
-
7
galaxy-io/filament
Pluggable data replication with checkpointing, batching, and integrity events
GitHub · Go · 336 stars · 47 forks · 0 stars in 24 hours · Trending score 1.06
-
8
justoneapi-labs/taobao-api
淘宝API、淘宝接口、淘宝爬虫示例。
GitHub · Python · 26 stars · 8 forks · +4 stars in 24 hours · Trending score 0.79
-
9
2captcha/2captcha-mcp
2Captcha MCP server for AI agents with web search, web scraping, browser automation, captcha solving, and scalable web data collection.
GitHub · JavaScript · 41 stars · 3 forks · 0 stars in 24 hours · Trending score 0.78
-
10
estuary/flow
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
GitHub · Rust · 992 stars · 101 forks · +1 stars in 24 hours · Trending score 0.73
-
11
brightdata/brightdata-mcp
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
GitHub · JavaScript · 2,724 stars · 337 forks · 0 stars in 24 hours · Trending score 0.69
-
12
cloudquery/cloudquery
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
GitHub · Go · 6,560 stars · 566 forks · +1 stars in 24 hours · Trending score 0.61
-
13
getodk/collect
ODK Collect is an Android app for filling out forms. It's been used to collect billions of data points in challenging environments around the world. Contribute and make the world a better place! ✨📋✨
GitHub · Kotlin · 807 stars · 1,446 forks · 0 stars in 24 hours · Trending score 0.52
-
14
Rekin226/aquascope
Explore river records and run reproducible water analyses in your browser. Open hydrology data, methods, and exports.
GitHub · Python · 31 stars · 49 forks · 0 stars in 24 hours · Trending score 0.32
-
15
chapmanjacobd/library
99+ CLI tools to build, browse, and blend your media library
GitHub · Python · 488 stars · 16 forks · 0 stars in 24 hours · Trending score 0.30
-
16
arielshakaramiro/hf-model-watcher
Daily catalog of new Hugging Face models in vision, speech and Indonesian LLMs, with noise filtering, CSV catalog and daily reports, auto-committed by GitHub Actions.
GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.23
-
17
plan-player-analytics/Plan
Player Analytics plugin for Minecraft Server platforms - View player activity of your server with ease. :calendar:
GitHub · Java · 1,041 stars · 211 forks · 0 stars in 24 hours · Trending score 0.23
-
18
olostep/olostep-mcp-server
MCP server for Olostep — the web scraping, crawling, and search infrastructure used by top AI companies. Gives any MCP-compatible AI agent the ability to scrape, crawl, batch-extract, and search the web in real time.
GitHub · TypeScript · 25 stars · 12 forks · 0 stars in 24 hours · Trending score 0.14
-
19
getodk/central-frontend
Vue.js based frontend for ODK Central
GitHub · TypeScript · 46 stars · 83 forks · 0 stars in 24 hours · Trending score 0.10
-
20
proxidize/x-scraper
A Twitter/X scraper built with Playwright for browser automation and OpenAI GPT-4 for AI-powered tweet analysis. Features timeline scraping, historical search, keyword search, checkpoint/resume, proxy support, and comprehensive data extraction.
GitHub · Python · 38 stars · 9 forks · 0 stars in 24 hours · Trending score 0.10
-
21
DouglasNeuroInformatics/OpenDataCapture
An electronic data capture platform for administering remote and in-person clinical instruments
GitHub · TypeScript · 119 stars · 25 forks · 0 stars in 24 hours · Trending score 0.09
-
22
LeoRigasaki/Anime-dataset
A collection of anime data from MyAnimeList, Anilist and Jikan
GitHub · Python · 22 stars · 0 forks · 0 stars in 24 hours · Trending score 0.08
-
23
WuKongSecurity/SpiderBOX
SpiderBox - 虫盒 - 爬虫逆向资源导航站
GitHub · CSS · 159 stars · 29 forks · 0 stars in 24 hours · Trending score 0.07
-
24
opensnowcat/opensnowcat-enrich
OpenSnowcat Enricher (Apache 2.0 License)
GitHub · Scala · 40 stars · 9 forks · 0 stars in 24 hours · Trending score 0.05