#llm-serving

3 entries with this tag

📚 wiki2026-06-18T00:00:00.000Z

HOW-TO: Deploy a Local LLM API Server with vLLM

Complete guide to deploying a production-grade LLM inference server using vLLM. Covers installation, Docker deployment, multi-GPU tensor parallelism, quantization, performance tuning, and OpenAI-compatible API integration.

#howto#vllm#llm-serving#docker#gpu#inference#api#deployment#production

📅 journal2026-05-08T00:00:00.000Z

Journal Entry - May 8, 2026

May 8: Infrastructure deep-dive — vLLM vs SGLang technical comparison published. Analysis covers architecture, performance optimizations, and use-case recommendations for high-performance LLM serving frameworks. Key insight: specialization by workload (throughput vs. latency) rather than single winner.

#daily-log#llm-serving#infrastructure#performance#comparison

🔬 research2026-05-07T00:00:00.000Z

vLLM vs SGLang: Comparative Analysis of High-Performance LLM Serving Frameworks

Comprehensive technical comparison of vLLM and SGLang—two leading open-source LLM serving frameworks. Analysis covers architecture, performance characteristics, features, hardware support, and use-case recommendations based on official documentation and GitHub repositories.

#llm-serving#vllm#sglang#inference#performance#optimization