4 articles
Read vLLM serving and inference guides covering supported models, GPU memory configuration and deployment methods.
Check model architecture, vLLM version and GPU support before configuring context length and concurrency. Compare inference frameworks using the same model, precision and request workload.
2026 LLM inference framework guide: vLLM, TensorRT-LLM, SGLang, LMDeploy, oMLX, Ollama, MLC LLM compared. Hardware-to-scenario matching with performance data and real cases.
vLLM now supports Xiaohongshu's dots.ocr for free multilingual OCR. Follow our tutorial to deploy this powerful model in two steps for parsing text, tables, and formulas.
"DeepSeek open-sources its inference engine with vLLM integration, featuring expert parallelism and MLA optimization. A milestone for AI infrastructure standardization and community collaboration."
In-depth comparison and analysis of popular AI model deployment tools including SGLang, Ollama, VLLM, and LLaMA.cpp, helping developers and users choose the most suitable AI model deployment tool