joris-sta/llama-cpp-pascal
Pre-tensor-core (Pascal GTX 1070) inference optimizations for llama.cpp: native Turbo3 compressed-KV flash attention, PTQ1 ternary DP4A matmuls, chunked gated-delta-net, TurboQuant/VBR KV on 80K-262K contexts
GitHub repository with 18 stars and 0 forks.
Language: Cuda
Topics: cuda, flash-attention, gtx1070, kv-cache, llama-cpp, pascal, quantization, ternary