NeuraCryption/SPECTRA
Compress any LLM in seconds, load it for inference and in one click you have an AI that could have taken hours of vLLM configuration and what not to setup and load. Of course there are opensource compressed models but this is for people who have build their own AI, finetuned it, etc.
GitHub repository with 20 stars and 2 forks.
Language: Python