要点速览
- Laguna XS.2 是 Poolside 首个开放权重模型:33B-A3B MoE,面向 agentic coding 与长周期软件任务。
- Red Hat AI 训练了 DFlash speculator:5 层、0.6B drafter 单次前向预测 8 个 token,由目标模型验证。
- drafter 在 500k 样本上训练 6 个 epoch(序列长 8192、采样 3072 个 block 位置),token 快 2-3×。
- 验证机制保证与单独使用大模型完全一致的生成质量(可证明),关键在于训练 DFlash 准确预测目标模型会接受的 token。
- 同步发布 LLM Compressor 构建的 FP8、NVFP4、INT4/INT8 量化 checkpoint,适配不同硬件与延迟需求。
解决什么问题
Laguna XS.2 作为面向 agentic coding 与长周期软件任务的开源 MoE 模型,需要在发布时就具备生产可用的推理性能与部署便利性。具体要解决三件事:模型能否在 vLLM 中开箱即用、推理能否通过推测解码进一步加速、以及能否提供多种量化格式以适配不同硬件与内存预算。
核心思路
Red Hat AI 与 Poolside 三线协作:首先把 Laguna XS.2 在发布当日作为一等公民直接集成进 vLLM,标准 API 即可部署。其次用 Speculators 库训练 DFlash speculator——一个 5 层、0.6B 的小 drafter,读取目标模型 hidden state,单次前向预测一个 token 块,再由 Laguna XS.2 单次验证:接受的 token 快速产出,验证步骤保证质量与单独使用大模型一致。drafter 在 UltraChat 200k SFT 与 Magpie-Align 的 500k 样本上训练,prompt 采样后由开启 thinking 的 Laguna XS.2 重新生成响应以对齐目标分布。最后用 LLM Compressor 发布 FP8、NVFP4、INT4/INT8 量化 checkpoint(compressed-tensors 格式)。
关键要点
- 一等 vLLM 集成 — Laguna XS.2 与 Poolside 协作,在发布时就直接集成进 vLLM 成为一等公民(first-class)模型,可通过标准 vLLM API 立即部署,无需等待社区后续适配。这是模型发布即生产可用的一部分。
- DFlash speculator 的结构与工作方式 — DFlash 被称为当前最先进的推测解码算法,代表超越 Eagle-3 范式的下一代方向。该 drafter 仅 5 层、0.6B 参数,以目标 Laguna XS.2 模型的 hidden state 为输入,单次前向预测一个 token 块;Laguna XS.2 随后用单次前向验证这些 token。验证步骤保证与单独使用大模型完全相同的生成质量;若 token 被接受,其产出速度远快于逐个自回归生成。
- 训练配方:数据与超参 — 训练数据为 Ultrachat 200k SFT 与 Magpie-Align 来源的 500k 样本;从各数据集采样 prompt 后,由开启 thinking 的 Laguna XS.2 重新生成响应,使训练分布对齐目标模型。共训练 6 个 epoch,使用 cosine 调度器、最大学习率 6e-4,序列长度 8192,每条序列随机采样 3072 个 block 位置。
- 效果:2-3× 提速且质量可证明无损 — 最终得到能以单次前向预测 8 个 token 的 5 层 drafter;经 Laguna XS.2 验证后,token 生成速度提升 2-3×,且依据推测解码验证理论可证明无生成质量损失。博客在两个不同数据集上给出了基准结果图,并提供了可直接试用的 vLLM recipe。
- LLM Compressor 量化 checkpoint — Poolside 团队用 LLM Compressor 库发布了 Laguna XS.2 的量化版本:FP8、NVFP4、INT4/INT8 三类变体,统一采用 compressed-tensors 格式,在保持模型质量的同时支持 vLLM 高效部署。开发者可依据硬件、延迟与内存需求选择最合适的变体。
- 可复制的开源生态协作模式 — 本文展示了一条端到端的开源协作路径:模型方(Poolside)提供开放权重,Red Hat AI 贡献推测解码训练(Speculators)与量化(LLM Compressor),vLLM 提供推理侧一等集成。speculator checkpoint、量化 checkpoint 与 recipe 均公开可得,可作为自研模型优化的参考模板。
关键数据
| 指标 | 数值 | 条件 |
|---|---|---|
| 目标模型规模 | 33B-A3B MoE | Laguna XS.2,Poolside Laguna 家族首个开放权重模型,面向 agentic coding |
| drafter 规模 | 5 层、0.6B 参数 | Laguna XS.2 的 DFlash draft 模型,输入来自目标模型的 hidden state |
| 单次前向预测 token 数 | 8 个 token | 训练完成后的 DFlash drafter 能力 |
| token 生成加速 | 2-3× | Laguna XS.2 + DFlash,经目标模型验证,可证明无质量损失 |
| 训练样本量 | 500k 样本 | Ultrachat 200k SFT + Magpie-Align,prompt 采样后由开启 thinking 的 Laguna XS.2 重新生成响应 |
| 训练超参 | 6 epochs;cosine 调度器;最大学习率 6e-4;序列长度 8192;每序列随机采样 3072 个 block 位置 | Laguna XS.2 DFlash speculator 的训练配置 |
“Laguna XS.2 is Poolside's first open-weight model in the Laguna family: a 33B-A3B MoE model built for agentic coding and long-horizon software tasks.”
“The model uses a small 5-layer, 0.6B draft model and hidden state inputs from the target Laguna XS.2 model to predict a block of tokens with a single forward pass.”
“The result is a 5-layer drafter that can predict 8 tokens out with a single forward pass.”
前置知识
MoE(Mixture of Experts)模型与 A3B(激活 3B 参数)的含义 · 推测解码基本流程:drafter 起草、目标模型验证 · agentic coding / 长周期软件任务场景的特点 · 常见量化格式 FP8、NVFP4、INT4/INT8 的大致定位差异 · Hugging Face Hub 与 vLLM serve 的基本使用
术语表
Laguna XS.2: Poolside Laguna 家族首个开放权重模型,33B-A3B MoE,面向 agentic coding 与长周期软件任务。DFlash: 当前最先进的推测解码算法:小 drafter 单次前向预测一个 token 块,由目标模型单次验证,超越 Eagle-3 的并行起草范式。Speculator: 经训练的 draft 模型及其 checkpoint,与目标模型搭配执行推测解码(如 Laguna-XS.2-speculator.dflash)。Speculators: vLLM 生态的推测解码模型训练库,本文中 DFlash speculator 即用它训练。LLM Compressor: vLLM 生态的模型量化库,用于产出 FP8、NVFP4、INT4/INT8 等量化 checkpoint。MoE (Mixture of Experts): 混合专家架构:总参数量大但每次前向只激活部分专家(如 33B 总参中激活 3B),兼顾容量与推理成本。compressed-tensors: vLLM 生态的量化模型存储格式,便于在 vLLM 中高效部署量化 checkpoint。NVFP4: NVIDIA 的 4 位浮点量化格式,面向低精度高吞吐推理的 checkpoint 变体之一。Thinking: 模型在给出回答前先产出推理/思考内容的模式;训练时开启以重新生成响应、对齐目标分布。
Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor
As organizations increasingly adopt AI-powered development tools, the need for high-performance agentic models that deliver both accuracy and operational efficiency has become critical. Laguna XS.2 is Poolside's first open-weight model in the Laguna family: a 33B-A3B MoE model built for agentic coding and long-horizon software tasks. As part of the Laguna XS.2 release, Red Hat AI and Poolside collaborated on serving and inference optimization, including first-class vLLM integration, a DFlash speculator checkpoint, and quantized checkpoints built with LLM Compressor. This release represents a significant milestone in production-ready AI deployment, with Laguna XS.2's quantized and speculator checkpoints optimized for speed and efficiency in real-world agentic applications.
Seamless Inference via vLLM Integration
In collaboration with Poolside, Laguna XS.2 was integrated directly into vLLM at launch as a first-class citizen, enabling immediate deployment through standard vLLM APIs.
Optimizing Performance with DFlash Speculative Decoding
To accelerate inference further, the Red Hat team trained a DFlash speculator for Laguna XS.2 using the Speculators library.
The DFlash algorithm is the current state of the art in speculative decoding. The model uses a small 5-layer, 0.6B draft model and hidden state inputs from the target Laguna XS.2 model to predict a block of tokens with a single forward pass. These tokens are then verified by the Laguna XS.2 model with a single pass. This verification step guarantees the same generation quality as using the large model alone; if the tokens are accepted, then they can be produced much more quickly per token than simply producing tokens one at a time using Laguna XS.2 autoregressively. The key is training DFlash to accurately predict tokens that Laguna XS.2 is likely to accept.
This model was trained on 500k samples from Ultrachat 200k SFT and Magpie-Align. Prompts were sampled from each dataset and responses were regenerated from Laguna XS.2, with thinking enabled. The model was then trained for 6 epochs using a cosine scheduler with a maximum learning rate of 6e-4, with a sequence length of 8192, and 3072 block positions were randomly sampled for each sequence.
The result is a 5-layer drafter that can predict 8 tokens out with a single forward pass. When verified with Laguna XS.2, it delivers tokens 2-3x faster with provably no loss in generation quality.
Fig 1: Benchmark results for Laguna XS.2 + DFlash on two different datasets.
The DFlash algorithm represents the next generation of speculative decoding, moving beyond the Eagle-3 paradigm to provide faster, parallel drafting that significantly reduces inter-token-latency. To test out the speculator yourself, check out the vLLM recipe.
Quantized Checkpoints with LLM Compressor
The Poolside team also released quantized Laguna XS.2 checkpoints using the LLM Compressor library. These checkpoints include FP8, NVFP4, INT4/INT8 variants in the compressed-tensors format to enable efficient deployment while maintaining model quality in vLLM.
LLM Compressor provides a flexible framework for applying various quantization techniques to LLMs. With these checkpoints, developers can choose the Laguna XS.2 variant that best fits their hardware, latency, and memory requirements.
Next Steps
- Explore the Laguna XS.2 models on the Hugging Face Hub
- Optimize your own models with LLM Compressor and Speculators