XHToken/Spark-plugin
Spark-plugin for Spark2.5 model, enables seamless loading and serving Spar2.5 large language model within vLLM framework. Implements custom model plugin to support weights loading, tokenizer and inference runtime without modifying original vLLM source code.
GitHub repository with 5 stars and 1 forks.
Language: Python
Topics: deployments, llm, python3, vllm, vllm-plugins