jranaraki/vllm-fit
A CLI tool designed to simply recommend (conservative), and/or profile (to maximize resource utilization) vLLM engine arguments for any HuggingFace model on the user's current hardware.
GitHub repository with 7 stars and 0 forks.
Language: Python
Topics: hf, huggingface, llm, llm-inference, localai, vllm