索引
标签
来自 vLLM 官方博客文章自带的 tags,共 73 个。
#performance 67 篇
2026-10-14Adaptive Verification Without a Confidence Head15 min2026-10-14Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM14 min2026-10-12Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM16 min2026-10-09vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL7211 min2026-10-07DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 011 min2026-09-24Watermarking in vLLM12 min2026-09-22Announcing vllm-metal: Concurrent Serving on Apple Silicon12 min2026-09-21PD Serving of Qwen3.8-2.4T15 min2026-09-18Scaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLM4 min2026-09-15How we trained the fastest DSpark for Kimi-K3 using GB300 NVL728 min2026-09-15vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B30013 min2026-09-14vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm8 min2026-09-13Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput7 min2026-09-10Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X16 min2026-09-08GLM 5.3 Optimizations, Part 1: Hybrid HiSparse Offloading in vLLM10 min2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-09-01MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH325 min2026-08-22Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM17 min2026-08-21IsoExec: Unified Execution to Eliminate Trainer-Inference Mismatch in SkyRL10 min2026-08-20VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training9 min2026-08-17Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni21 min2026-08-14Adaptive Verification in vLLM: DSpark confidence-scheduled verification7 min2026-08-12Day 0 Support for Qwen3.8-2.4T-A95B on vLLM3 min2026-08-07Efficient Decode Context Parallelism with vLLM for Long Context Workloads11 min2026-08-06vLLM Reaches 25K Total TPS/GPU on Qwen3.59 min2026-07-29Optimizing vLLM on Arm CPUs9 min2026-07-27Kimi K3 Is Here: Efficient Day-0 Support on vLLM23 min2026-07-23From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM16 min2026-07-22A Preview of Production-Scale Kimi K3 Support on vLLM12 min2026-07-16Keeping vLLM Production Quality: A Look Inside CI, Benchmarking, and the Release Process19 min2026-07-15TML Inkling on vLLM: Day-0 Support with Optimized Performance8 min2026-07-14vLLM x TileRT: Specialized Decode for Latency-Critical Serving7 min2026-07-13EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark12 min2026-07-06vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan15 min2026-07-01Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni19 min2026-06-23Engineering TTS Inference in vLLM-Omni21 min2026-06-02Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents18 min2026-05-28From Text to Multimodal Routing: Hardening Vision Signals in vLLM Semantic Router11 min2026-05-26EAGLE 3.1: Advancing Speculative Decoding Through Collaboration Between the EAGLE Team, vLLM, and TorchSpec4 min2026-05-18vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache12 min2026-05-11vLLM Tops the Artificial Analysis Leaderboard7 min2026-04-22The State of FP8 KV-Cache and Attention Quantization in vLLM20 min2026-03-24Model Runner V2: A Modular and Faster Core for vLLM7 min2026-03-13P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM12 min2026-03-04vLLM Triton Attention Backend Deep Dive10 min2026-02-27Beyond Porting: How vLLM Orchestrates High-Performance Inference on AMD ROCm17 min2026-02-26Efficiently serve dozens of fine-tuned models with vLLM on Amazon SageMaker AI and Amazon Bedrock10 min2026-02-13DeepSeek-V3.2 on GB300: Performance Breakthrough12 min2026-02-03Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell (Part I)9 min2026-02-01GPT-OSS Performance Optimizations on NVIDIA Blackwell: Pushing the Pareto Frontier7 min2026-01-08Inside vLLM’s New KV Offloading Connector: Smarter Memory Transfer for Maximizing Inference Throughput14 min2025-12-19vLLM-Omni Diffusion Cache Acceleration4 min2025-12-17vLLM Large Scale Serving: DeepSeek @ 2.2k tok/s/H200 with Wide-EP8 min2025-11-13Shared Memory IPC Caching: Accelerating Data Transfer in LLM Inference Systems6 min2025-11-10No More Train-Inference Mismatch: Bitwise Consistent On-Policy Reinforcement Learning with vLLM and TorchTitan6 min2025-10-26Zero-Reload Model Switching with vLLM Sleep Mode17 min2025-10-09SemiAnalysis InferenceMAX: vLLM and NVIDIA Accelerate Blackwell Inference7 min2025-08-20Introduction to torch.compile and How It Works with vLLM14 min2025-08-05vLLM Now Supports gpt-oss4 min2025-06-30MiniMax-M1 Hybrid Architecture Meets vLLM: Long Context, Fast Inference5 min2025-01-27vLLM V1: A Major Upgrade to vLLM's Core Architecture11 min2025-01-14Structured Decoding in vLLM: a gentle introduction12 min2024-10-23Serving LLMs on AMD MI300X: Best Practices16 min2024-10-17How Speculative Decoding Boosts vLLM Performance by up to 2.8x9 min2024-09-05vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction12 min2023-11-14Notes on vLLM v.s. DeepSpeed-FastGen3 min2023-06-20vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention8 min#ecosystem 33 篇
2026-09-22Announcing vllm-metal: Concurrent Serving on Apple Silicon12 min2026-09-14vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm8 min2026-09-07Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin15 min2026-08-20VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training9 min2026-07-23Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving9 min2026-07-21Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router19 min2026-07-14vLLM x TileRT: Specialized Decode for Latency-Critical Serving7 min2026-07-10vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs8 min2026-06-29Micro-Agent: Beat Frontier Models with Collaboration inside Model API10 min2026-06-16Beyond One Model: Fusion in vLLM Semantic Router11 min2026-06-10DiffusionGemma: The First Diffusion LLM (dLLM) Natively Supported in vLLM12 min2026-06-09Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs6 min2026-06-05vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing21 min2026-06-03Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI4 min2026-06-02Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents18 min2026-05-28Speculators v0.5.0: DFlash Support and Online Training5 min2026-05-28From Text to Multimodal Routing: Hardening Vision Signals in vLLM Semantic Router11 min2026-05-14Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models7 min2026-03-10vLLM Semantic Router v0.2 Athena: ClawOS, Model Refresh, and the System Brain21 min2026-01-23Building Mixture-of-Models on AMD GPUs with vLLM-SR8 min2026-01-05vLLM Semantic Router v0.1 Iris: The First Major Release8 min2026-01-02Introducing vLLM Playground: A Modern Web Interface for Managing and Interacting with vLLM Servers5 min2025-12-19vLLM-Omni Diffusion Cache Acceleration4 min2025-12-16AMD × vLLM Semantic Router: Building the System Intelligence Together9 min2025-12-14Token-Level Truth: Real-Time Hallucination Detection for Production LLMs12 min2025-12-13Diving into speculative decoding training support for vLLM with Speculators v0.3.010 min2025-12-09Advancing Low‑Bit Quantization for LLMs: AutoRound x LLM Compressor5 min2025-11-30Announcing vLLM-Omni: Easy, Fast, and Cheap Omni-Modality Model Serving4 min2025-11-19Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale12 min2025-10-27From Monolithic to Modular: Scaling Semantic Routing with Extensible LoRA8 min2025-09-11vLLM Semantic Router: Next Phase in LLM inference4 min2025-02-21Introducing AIBrix: A Scalable, Cost-Effective Control Plane for vLLM4 min2025-01-21High Performance and Easy Deployment of vLLM in K8S with vLLM production-stack4 min#hardware 26 篇
2026-10-09vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL7211 min2026-09-22Announcing vllm-metal: Concurrent Serving on Apple Silicon12 min2026-09-18Scaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLM4 min2026-09-15vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B30013 min2026-09-14vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm8 min2026-09-10Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X16 min2026-09-07Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin15 min2026-08-12Day 0 Support for Qwen3.8-2.4T-A95B on vLLM3 min2026-07-29Optimizing vLLM on Arm CPUs9 min2026-07-13EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark12 min2026-07-10vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs8 min2026-06-02Accelerating vLLM-Omni Inference with AutoRound Quantization9 min2026-06-01vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation17 min2026-02-27Beyond Porting: How vLLM Orchestrates High-Performance Inference on AMD ROCm17 min2026-02-13DeepSeek-V3.2 on GB300: Performance Breakthrough12 min2026-02-03Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell (Part I)9 min2026-02-01GPT-OSS Performance Optimizations on NVIDIA Blackwell: Pushing the Pareto Frontier7 min2026-01-23Building Mixture-of-Models on AMD GPUs with vLLM-SR8 min2025-12-16AMD × vLLM Semantic Router: Building the System Intelligence Together9 min2025-12-09Advancing Low‑Bit Quantization for LLMs: AutoRound x LLM Compressor5 min2025-11-11Fast and Affordable LLMs serving on Intel Arc Pro B-Series GPUs with vLLM8 min2025-10-16vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU11 min2025-10-09SemiAnalysis InferenceMAX: vLLM and NVIDIA Accelerate Blackwell Inference7 min2025-05-12Introducing vLLM Hardware Plugin, Best Practice from Ascend NPU5 min2025-02-24PTPC-FP8: Boosting vLLM Performance on AMD ROCm9 min2024-10-23Serving LLMs on AMD MI300X: Best Practices16 min#model-support 21 篇
2026-08-12Day 0 Support for Qwen3.8-2.4T-A95B on vLLM3 min2026-08-10Announcing Day-0 Support for NVIDIA Nemotron 3.5 Lightning on vLLM10 min2026-07-15TML Inkling on vLLM: Day-0 Support with Optimized Performance8 min2026-06-04Announcing Day-0 Support for NVIDIA Nemotron 3 Ultra on vLLM6 min2026-04-28Run Highly Efficient Multimodal Agentic AI with NVIDIA Nemotron 3 Nano Omni Using vLLM7 min2026-04-24DeepSeek V4 in vLLM: Efficient Long-context Attention17 min2026-04-02Announcing Gemma 4 on vLLM: Byte for byte, the most capable open models3 min2026-03-11Run Highly Efficient and Accurate Multi-Agent AI with NVIDIA Nemotron 3 Super Using vLLM5 min2025-12-15Run Highly Efficient and Accurate AI Agents with NVIDIA Nemotron 3 Nano on vLLM5 min2025-10-31Run Multimodal Reasoning Agents with NVIDIA Nemotron on vLLM4 min2025-10-28Chasing 100% Accuracy: A Deep Dive into Debugging Kimi K2's Tool-Calling on vLLM10 min2025-10-23Now Serving NVIDIA Nemotron with vLLM5 min2025-09-29DeepSeek-V3.2-Exp in vLLM: Fine-Grained Sparse Attention in Action7 min2025-09-11vLLM Now Supports Qwen3-Next: Hybrid Architecture with Extreme Efficiency3 min2025-08-19GLM-4.5 Meets vLLM: Built for Intelligent Agents4 min2025-08-05vLLM Now Supports gpt-oss4 min2025-06-30MiniMax-M1 Hybrid Architecture Meets vLLM: Long Context, Fast Inference5 min2025-04-11Transformers modeling backend integration in vLLM6 min2025-04-05Llama 4 in vLLM4 min2025-01-27Introducing vLLM Inference Provider in Llama Stack8 min2024-07-23Announcing Llama 3.1 Support in vLLM5 min#multimodal 19 篇
2026-09-18Scaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLM4 min2026-09-01MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH325 min2026-08-20VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training9 min2026-07-27Kimi K3 Is Here: Efficient Day-0 Support on vLLM23 min2026-07-22A Preview of Production-Scale Kimi K3 Support on vLLM12 min2026-07-15TML Inkling on vLLM: Day-0 Support with Optimized Performance8 min2026-07-01Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni19 min2026-06-23Engineering TTS Inference in vLLM-Omni21 min2026-06-02Accelerating vLLM-Omni Inference with AutoRound Quantization9 min2026-05-14Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models7 min2026-01-31Streaming Requests & Realtime API in vLLM16 min2025-12-19vLLM-Omni Diffusion Cache Acceleration4 min2025-12-15Encoder Disaggregation for Scalable Multimodal Model Serving9 min2025-11-30Announcing vLLM-Omni: Easy, Fast, and Cheap Omni-Modality Model Serving4 min2025-11-13Shared Memory IPC Caching: Accelerating Data Transfer in LLM Inference Systems6 min2025-10-31Run Multimodal Reasoning Agents with NVIDIA Nemotron on vLLM4 min2025-09-05Serving Geospatial, Vision, and Beyond: Enabling Multimodal Output Processing in vLLM10 min2025-08-19GLM-4.5 Meets vLLM: Built for Intelligent Agents4 min2025-04-05Llama 4 in vLLM4 min#speculative-decoding 18 篇
2026-10-14Adaptive Verification Without a Confidence Head15 min2026-09-24Watermarking in vLLM12 min2026-09-15How we trained the fastest DSpark for Kimi-K3 using GB300 NVL728 min2026-09-13Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput7 min2026-09-10Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X16 min2026-08-23Exploring Speculative Decoding in vLLM on AMD GPUs339 min2026-08-14Adaptive Verification in vLLM: DSpark confidence-scheduled verification7 min2026-07-23From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM16 min2026-07-15TML Inkling on vLLM: Day-0 Support with Optimized Performance8 min2026-05-28Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor3 min2026-05-28Speculators v0.5.0: DFlash Support and Online Training5 min2026-05-26EAGLE 3.1: Advancing Speculative Decoding Through Collaboration Between the EAGLE Team, vLLM, and TorchSpec4 min2026-05-11vLLM Tops the Artificial Analysis Leaderboard7 min2026-03-30Extracting hidden states from vLLM7 min2026-03-13P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM12 min2025-12-13Diving into speculative decoding training support for vLLM with Speculators v0.3.010 min2025-09-05Inside vLLM: Anatomy of a High-Throughput LLM Inference System43 min2024-10-17How Speculative Decoding Boosts vLLM Performance by up to 2.8x9 min#large-scale-serving 15 篇
2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-09-01MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH325 min2026-07-23From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM16 min2026-05-14Elastic Expert Parallelism in vLLM10 min2026-05-06Serving Agentic Workloads at Scale with vLLM x Mooncake10 min2026-02-03Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell (Part I)9 min2025-12-17vLLM Large Scale Serving: DeepSeek @ 2.2k tok/s/H200 with Wide-EP8 min2025-12-15Encoder Disaggregation for Scalable Multimodal Model Serving9 min2025-12-13vLLM Router: A High-Performance and Prefill/Decode Aware Load Balancer for Large-scale Serving4 min2025-11-22Streamlined multi-node serving with Ray symmetric-run4 min2025-09-05Inside vLLM: Anatomy of a High-Throughput LLM Inference System43 min2025-04-23Accelerating RLHF with vLLM, Best Practice from OpenRLHF5 min2025-02-21Introducing AIBrix: A Scalable, Cost-Effective Control Plane for vLLM4 min2025-02-17Distributed Inference with vLLM4 min2025-01-21High Performance and Easy Deployment of vLLM in K8S with vLLM production-stack4 min#disaggregation 13 篇
2026-10-14Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM14 min2026-10-12Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM16 min2026-09-29Taking vLLM Apart: A Practical Guide to Disaggregated Serving19 min2026-09-21PD Serving of Qwen3.8-2.4T15 min2026-09-10Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X16 min2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-08-06vLLM Reaches 25K Total TPS/GPU on Qwen3.59 min2026-07-23From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM16 min2026-07-14vLLM x TileRT: Specialized Decode for Latency-Critical Serving7 min2026-05-18vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache12 min2026-05-06Serving Agentic Workloads at Scale with vLLM x Mooncake10 min2026-04-21Disaggregated Serving for Hybrid SSM Models in vLLM13 min2026-04-07Next-Level Inference: Why Your Single-Node vLLM Setup Needs Prefill-Decode Disaggregation21 min#quantization 10 篇
2026-09-15vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B30013 min2026-08-12Day 0 Support for Qwen3.8-2.4T-A95B on vLLM3 min2026-06-02Accelerating vLLM-Omni Inference with AutoRound Quantization9 min2026-05-28Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor3 min2026-05-11A First Comprehensive Study of TurboQuant: Accuracy and Performance12 min2026-04-22The State of FP8 KV-Cache and Attention Quantization in vLLM20 min2026-02-13DeepSeek-V3.2 on GB300: Performance Breakthrough12 min2025-12-09Advancing Low‑Bit Quantization for LLMs: AutoRound x LLM Compressor5 min2025-02-24PTPC-FP8: Boosting vLLM Performance on AMD ROCm9 min2024-07-23Announcing Llama 3.1 Support in vLLM5 min#moe 8 篇
2026-09-15vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B30013 min2026-09-10Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X16 min2026-08-12Day 0 Support for Qwen3.8-2.4T-A95B on vLLM3 min2026-07-23From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM16 min2026-07-23Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving9 min2026-07-06vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan15 min2026-06-12MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning19 min2026-05-14Elastic Expert Parallelism in vLLM10 min#kv_cache 7 篇
2026-10-14Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM14 min2026-10-12Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM16 min2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-05-18vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache12 min2026-05-11A First Comprehensive Study of TurboQuant: Accuracy and Performance12 min2026-05-06Serving Agentic Workloads at Scale with vLLM x Mooncake10 min2026-04-22The State of FP8 KV-Cache and Attention Quantization in vLLM20 min#reinforcement-learning 7 篇
2026-10-14Finish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-Driven32 min2026-09-14vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm8 min2026-08-22Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM17 min2026-08-21IsoExec: Unified Execution to Eliminate Trainer-Inference Mismatch in SkyRL10 min2026-07-10vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs8 min2026-06-09Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs6 min2026-05-28Native RL APIs in vLLM12 min#community 6 篇
2026-06-03Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI4 min2026-04-14vLLM Korea Meetup 2026 Wrap-Up6 min2025-12-27Announcing vllm.ai Website and Some Community Updates2 min2025-09-16The First vLLM Meetup in Korea4 min2025-01-10vLLM 2024 Retrospective and 2025 Vision10 min2024-07-25vLLM’s Open Governance and Performance Roadmap4 min#developer 5 篇
2025-12-03Tracing Hanging and Complicated GPU Kernels Down To The Source Code15 min2025-11-20Building Clean, Maintainable vLLM Modifications Using the Plugin System12 min2025-10-28Chasing 100% Accuracy: A Deep Dive into Debugging Kimi K2's Tool-Calling on vLLM10 min2025-08-11CUDA Core Dump: An Effective Tool to Debug Memory Access Issues and Beyond16 min2025-01-10Installing and Developing vLLM with Ease5 min#vllm-omni 4 篇
2026-09-01MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH325 min2026-08-17Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni21 min2026-07-01Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni19 min2026-06-02Accelerating vLLM-Omni Inference with AutoRound Quantization9 min#attention 3 篇
2026-08-07Efficient Decode Context Parallelism with vLLM for Long Context Workloads11 min2026-07-06vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan15 min2026-03-04vLLM Triton Attention Backend Deep Dive10 min#inference 3 篇
2026-07-23Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving9 min2026-06-23Engineering TTS Inference in vLLM-Omni21 min2026-06-10DiffusionGemma: The First Diffusion LLM (dLLM) Natively Supported in vLLM12 min#models 3 篇
2026-09-13Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput7 min2026-07-27Kimi K3 Is Here: Efficient Day-0 Support on vLLM23 min2026-07-22A Preview of Production-Scale Kimi K3 Support on vLLM12 min#post-training 3 篇
2026-09-14vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm8 min2026-07-10vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs8 min2026-06-09Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs6 min#speculators 3 篇
2026-09-15How we trained the fastest DSpark for Kimi-K3 using GB300 NVL728 min2026-07-28Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding7 min2026-05-28Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor3 min#agentic 2 篇
2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-05-06Serving Agentic Workloads at Scale with vLLM x Mooncake10 min#agentic-routing 2 篇
2026-06-29Micro-Agent: Beat Frontier Models with Collaboration inside Model API10 min2026-06-02Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents18 min#deepseek 2 篇
2026-10-14Finish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-Driven32 min2026-10-07DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 011 min#dflash 2 篇
2026-07-28Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding7 min2026-05-28Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor3 min#distributed 2 篇
2026-09-15How we trained the fastest DSpark for Kimi-K3 using GB300 NVL728 min2026-08-17Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni21 min#kernels 2 篇
2026-10-07DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 011 min2026-09-13Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput7 min#nixl 2 篇
2026-10-14Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM14 min2026-10-12Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM16 min#parallelism 2 篇
2026-09-08vLLM x AgentX: Optimizing for Real-World Agentic Serving24 min2026-08-07Efficient Decode Context Parallelism with vLLM for Long Context Workloads11 min#prefix caching 2 篇
2026-07-27Kimi K3 Is Here: Efficient Day-0 Support on vLLM23 min2026-07-22A Preview of Production-Scale Kimi K3 Support on vLLM12 min#rlhf 2 篇
2026-08-20VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training9 min2026-05-14Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models7 min#async-rl 1 篇
2026-05-28Native RL APIs in vLLM12 min#benchmarking 1 篇
2026-05-11vLLM Tops the Artificial Analysis Leaderboard7 min#computex 1 篇
2026-06-01vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation17 min#cosmos3 1 篇
2026-08-17Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni21 min#day-0-support 1 篇
2026-06-12MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning19 min#deployment 1 篇
2026-06-01vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation17 min#dgx-spark 1 篇
2026-06-01vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation17 min#dspark 1 篇
2026-07-28Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding7 min#elastic-ep 1 篇
2026-05-14Elastic Expert Parallelism in vLLM10 min#evaluation 1 篇
2026-07-16Keeping vLLM Production Quality: A Look Inside CI, Benchmarking, and the Release Process19 min#expert-parallelism 1 篇
2026-05-14Elastic Expert Parallelism in vLLM10 min#fastvideo 1 篇
2026-09-01MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH325 min#fault-tolerance 1 篇
2026-05-14Elastic Expert Parallelism in vLLM10 min#frontend 1 篇
2026-01-02Introducing vLLM Playground: A Modern Web Interface for Managing and Interacting with vLLM Servers5 min#hpc-ops 1 篇
2026-07-06vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan15 min#kernel-fusion 1 篇
2026-05-11vLLM Tops the Artificial Analysis Leaderboard7 min#learning 1 篇
2026-06-03Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI4 min#llm-compressor 1 篇
2026-05-28Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor3 min#long-context 1 篇
2026-06-12MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning19 min#mixture-of-models 1 篇
2026-07-21Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router19 min#nemotron 1 篇
2026-06-01vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation17 min#peagle 1 篇
2026-07-28Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding7 min#production-serving 1 篇
2026-05-18vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache12 min#qwen3.8 1 篇
2026-09-21PD Serving of Qwen3.8-2.4T15 min#release 1 篇
2026-07-16Keeping vLLM Production Quality: A Look Inside CI, Benchmarking, and the Release Process19 min#sampling 1 篇
2026-09-24Watermarking in vLLM12 min#semantic-router 1 篇
2026-07-21Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router19 min#speculative_decoding 1 篇
2026-07-28Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding7 min#watermarking 1 篇
2026-09-24Watermarking in vLLM12 min