DSpark is a speculative decoding method published by Novita. We trained and released production-oriented DSpark speculators for Kimi-K2.6 and Kimi-K2.7-Code, integrated them with vLLM, and evaluated how their parallel block-drafting approach scales as the speculative window grows.
The checkpoints are designed to make Kimi serving faster without changing the target model. A lightweight draft model proposes a block of candidate tokens, and the full Kimi model verifies those candidates before they are accepted.
In our batch-size-1 vLLM nightly evaluations, DSpark with num_speculative_tokens=7 achieved an average throughput speedup of 2.55x for Kimi-K2.6 and 2.36x for Kimi-K2.7-Code. As the speculative window increased from n=3 to n=7, DSpark converted the additional draft tokens into throughput more consistently than the available Eagle3-MLA baselines.
The released checkpoints are:
- novita/kimi-k2.6-dspark for Kimi-K2.6
- novita/kimi-k2.7-code-dspark for Kimi-K2.7-Code
DSpark benchmark results for Kimi-K2.6 and Kimi-K2.7-Code
At batch size 1, increasing num_speculative_tokens from 3 to 7 raised DSpark’s average speedup from 2.17x to 2.55x for Kimi-K2.6 and from 2.12x to 2.36x for Kimi-K2.7-Code. The listed Eagle3-MLA baselines remained nearly flat on average.
| Target model | Draft model | n=3 avg speedup | n=7 avg speedup | Change from n=3 |
|---|---|---|---|---|
| Kimi-K2.6 | DSpark | 2.17x | 2.55x | +17.8% |
| Kimi-K2.6 | Eagle3-MLA | 2.01x | 2.01x | +0.2% |
| Kimi-K2.7-Code | DSpark | 2.12x | 2.36x | +11.7% |
| Kimi-K2.7-Code | Eagle3-MLA | 2.04x | 2.05x | +0.3% |
These values are simple, unweighted averages of benchmark-level speedups across GSM8K, MATH500, AIME, HumanEval, LiveCodeBench, and SPEED-Bench Coding; they are not weighted by prompt count. A larger draft window does not automatically improve throughput: the additional proposals must be accepted often enough to offset drafting and verification overhead.
LiveCodeBench makes the difference concrete. For Kimi-K2.6, DSpark moved from 1.86x at n=3 to 1.87x at n=7, while Eagle3-MLA fell from 1.67x to 1.48x. For Kimi-K2.7-Code, DSpark increased from 1.75x to 1.78x, while Eagle3-MLA fell from 1.69x to 1.52x. On SPEED-Bench Coding, DSpark increased from 2.19x to 2.41x for Kimi-K2.6 and from 2.15x to 2.31x for Kimi-K2.7-Code.
Evaluation setup: batch size 1, online vLLM nightly speculative decoding, greedy decoding, TP=8, max_model_len=20000, CUDA graphs enabled, and fuse_allreduce_rms=false. Higher tokens/sec and speedup are better. Batch size 1 isolates single-request decode behavior; results may differ under continuous batching or higher-concurrency production workloads. Full results are available on the Hugging Face pages for Kimi-K2.6 DSpark and Kimi-K2.7-Code DSpark.
Why DSpark improves Kimi speculative decoding
Our checkpoints apply DSpark to Kimi-K2.6 and Kimi-K2.7-Code as draft models for vLLM serving.
Relative to the DFlash parallel drafting backbone, DSpark adds two lightweight heads: a Markov logit-bias head for low-rank intra-block token dependency, and a per-position confidence head for acceptance prediction. DSpark proposes the full draft block in parallel, then uses these heads to improve block-level draft quality and estimate which positions the verifier is likely to accept.
We trained the draft models with a vendored fork of vLLM’s Speculators framework. During training, the draft consumes hidden states streamed from a live Kimi verifier running in vLLM, aligning the speculator with the verifier behavior it will encounter at serving time.
Our published tables use the available open-source Eagle3-MLA checkpoints as draft-model baselines under the same verifier and serving configuration. The central result is how the methods scale with a larger speculative window: from n=3 to n=7, DSpark’s average speedup increased by 17.8% for Kimi-K2.6 and 11.7% for Kimi-K2.7-Code, while the listed Eagle3-MLA baselines remained nearly flat on average and slowed down on both LiveCodeBench evaluations. This suggests that DSpark preserves useful draft quality more effectively as the window grows. The comparison is specific to the checkpoints, batch-size-1 workload, and serving configuration tested here.
How to serve Kimi-K2.7-Code with DSpark in vLLM
DSpark support currently requires a vLLM nightly build. For Kimi-K2.7-Code, use the following speculative decoding configuration:
uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
vllm serve moonshotai/Kimi-K2.7-Code \
--tensor-parallel-size 8 \
--max-model-len 20000 \
--trust-remote-code \
--compilation-config='{"pass_config": {"fuse_allreduce_rms": false}}' \
--speculative-config '{
"model": "novita/kimi-k2.7-code-dspark",
"num_speculative_tokens": 7,
"method": "dspark"
}'For Kimi-K2.6, use the same configuration and point speculative_config.model to novita/kimi-k2.6-dspark.
Known nightly-build caveats: draft-side FA3 AOT scheduling may require passing fast_build=True when building draft attention metadata, and CUDA graph capture may hit a FlashInfer all-reduce workspace-size error. The tested configuration disables fuse_allreduce_rms; see the linked Hugging Face pages for the current workarounds.
DSpark changes the decoding path without changing the target Kimi model. We recommend starting with num_speculative_tokens=7, then validating throughput, accepted length, inter-token latency, and end-to-end latency against representative production traffic. The best configuration will depend on workload shape, batching, and serving hardware.
