系统学习 vLLM 推理引擎
自动采集官方博客全文,做结构化归纳、主题分类与学习路径组织。
▸ 最新文章9 篇 全部 149 篇 →
在线拟合的逻辑回归从草稿 logits 估计接受概率,让无 confidence head 的草稿模型免训接入 AV。
Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLMpush 模式让 prefill 直接把 KV 写入 decode 预注册显存，TTFT 最高降 3 倍
Finish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-DrivenWe make vLLM rollout and the training forward give bitwise identical logprobs on DeepSeek-V4-Flash and Nemotron-3.5-Lightning-30B-A3B (NVFP4), keep 91% of rollout throughput on DeepSeek-V4-Flash, and open-source the agent skill.
Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM双向 KV 传输让 prefill 复用 decode 保留的历史 KV，多轮场景 TTFT 最高降 3.3 倍
vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72vLLM now runs on NVIDIA Vera Rubin NVL72 with day-0 model support, Rubin-tuned FlashInfer kernels, and locality-aware MoE, delivering 7.8x the per-GPU throughput of GB200 NVL72 on AgentX.
DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.
Taking vLLM Apart: A Practical Guide to Disaggregated ServingvLLM 分离式服务实战:拆分 prefill/decode 与 CPU 前端,用 KV cache 传输换取 goodput 与稳定尾延迟。
Watermarking in vLLMHow vLLM implements distribution-preserving Gumbel-max text watermarking with efficient GPU kernels, statistical detection, speculative decoding, and repeated-context safeguards.
Announcing vllm-metal: Concurrent Serving on Apple Siliconvllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.