qflen/nsa-from-scratch

From-scratch reimplementation of DeepSeek's Native Sparse Attention (arXiv:2502.11089) in Triton + CUDA Hopper WGMMA. 7.4x faster than FlashAttention-3 at 64k context. Five-model training fleet, perplexity sweep, LongBench v2, MoBA comparison.

GitHub repository with 6 stars and 0 forks.

Language: Python

Topics: attention-mechanism, cuda, deepseek, flash-attention, gpu-kernels, hopper, llm, long-context, native-sparse-attention, nsa

Open provider repository

24h trend summary

Trending score 0.00, activity score 0.00, stars gained +0, forks gained +0.

Latest metric snapshot

2026-06-05: 6 stars and 0 forks.

Similar repositories

  1. 1. lucidrains/x-transformers

    A concise but complete full-attention transformer with a set of promising experimental features from various papers

    GitHub repository with 5,887 stars and 510 forks.

    Trending score: 0.68; stars gained: +2; forks gained: +0.

    Language: Python

    Topics: artificial-intelligence, deep-learning, attention-mechanism, transformers

  2. 2. hasibul0912/ChunkWise

    📝 Streamline text processing in Arabic and English with ChunkWise, a library offering 31 chunking strategies for NLP and RAG systems.

    GitHub repository with 5 stars and 0 forks.

    Trending score: 0.05; stars gained: +0; forks gained: +0.

    Language: Python

    Topics: aggregation, attention-mechanism, chunk, chunkwise-processing, csv, data-analysis

  3. 3. qflen/nsa-from-scratch

    From-scratch reimplementation of DeepSeek's Native Sparse Attention (arXiv:2502.11089) in Triton + CUDA Hopper WGMMA. 7.4x faster than FlashAttention-3 at 64k context. Five-model training fleet, perplexity sweep, LongBench v2, MoBA comparison.

    GitHub repository with 6 stars and 0 forks.

    Trending score: 0.00; stars gained: +0; forks gained: +0.

    Language: Python

    Topics: attention-mechanism, cuda, deepseek, flash-attention, gpu-kernels, hopper

Trending in Python

  1. 1. NousResearch/hermes-agent

    The agent that grows with you

    GitHub repository with 182,513 stars and 31,295 forks.

    Trending score: 5.95; stars gained: +1,867; forks gained: +361.

    Language: Python

    Topics: ai, ai-agent, ai-agents, anthropic, chatgpt, claude

  2. 2. chopratejas/headroom

    Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.

    GitHub repository with 14,053 stars and 885 forks.

    Trending score: 5.69; stars gained: +2,829; forks gained: +175.

    Language: Python

    Topics: agent, ai, anthropic, compression, context-engineering, context-window

  3. 3. Imbad0202/academic-research-skills

    Academic Research Skills for Claude Code: research → write → review → revise → finalize

    GitHub repository with 27,616 stars and 2,272 forks.

    Trending score: 5.52; stars gained: +1,079; forks gained: +89.

    Language: Python

    Topics: academic-pipeline, academic-writing, ai-research, claude, claude-code, literature-review

  4. 4. rohitg00/ai-engineering-from-scratch

    Learn it. Build it. Ship it for others.

    GitHub repository with 28,711 stars and 4,695 forks.

    Trending score: 5.32; stars gained: +1,261; forks gained: +238.

    Language: Python

    Topics: agents, ai, ai-agents, ai-engineering, computer-vision, course

  5. 5. vinta/awesome-python

    An opinionated list of Python frameworks, libraries, tools, and resources

    GitHub repository with 301,435 stars and 28,046 forks.

    Trending score: 4.60; stars gained: +518; forks gained: +24.

    Language: Python

    Topics: awesome, collections, python, python-frameworks, python-libraries, python-tools

  6. 6. Alishahryar1/free-claude-code

    Use claude-code for free in the terminal, VSCode extension or discord like OpenClaw (voice supported)

    GitHub repository with 32,540 stars and 4,942 forks.

    Trending score: 4.56; stars gained: +467; forks gained: +82.

    Language: Python

Trending topic: attention-mechanism

  1. 1. lucidrains/x-transformers

    A concise but complete full-attention transformer with a set of promising experimental features from various papers

    GitHub repository with 5,887 stars and 510 forks.

    Trending score: 0.68; stars gained: +2; forks gained: +0.

    Language: Python

    Topics: artificial-intelligence, deep-learning, attention-mechanism, transformers

  2. 2. nndl/nndl

    邱锡鹏《神经网络与深度学习》(蒲公英书)理论书 v2 与通识版

    GitHub repository with 18,809 stars and 3,667 forks.

    Trending score: 0.29; stars gained: +0; forks gained: +1.

    Topics: attention-mechanism, chinese, deep-learning, machine-learning, neural-networks, textbook

  3. 3. hasibul0912/ChunkWise

    📝 Streamline text processing in Arabic and English with ChunkWise, a library offering 31 chunking strategies for NLP and RAG systems.

    GitHub repository with 5 stars and 0 forks.

    Trending score: 0.05; stars gained: +0; forks gained: +0.

    Language: Python

    Topics: aggregation, attention-mechanism, chunk, chunkwise-processing, csv, data-analysis

  4. 4. qflen/nsa-from-scratch

    From-scratch reimplementation of DeepSeek's Native Sparse Attention (arXiv:2502.11089) in Triton + CUDA Hopper WGMMA. 7.4x faster than FlashAttention-3 at 64k context. Five-model training fleet, perplexity sweep, LongBench v2, MoBA comparison.

    GitHub repository with 6 stars and 0 forks.

    Trending score: 0.00; stars gained: +0; forks gained: +0.

    Language: Python

    Topics: attention-mechanism, cuda, deepseek, flash-attention, gpu-kernels, hopper