huawei-csl/MACaQ
SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.
GitHub repository with 293 stars and 21 forks.
Language: C++
Topics: ascend, audio, audio-generation, diffusers, gguf, gguf-quantization, image-generation, llama-cpp, llm, local-ai