DSpark
DeepSeek's new speculative decoder guesses a whole block of tokens in one pass, then runs a tiny second corrector over the block to fix the wrong ones. In production it moves 51% more traffic through the same GPUs.
Speculative decoding speeds up a large model by having a small, cheap drafter guess the next several tokens, then letting the big model check them all in one pass and keep the good ones. The output is identical; you just wait less. The whole trick lives or dies on the drafter, and the field has been stuck choosing between two flawed ones: guess the tokens one at a time (accurate, but slow), or guess a whole block in parallel (fast, but the guesses decay toward the end of the block because each token was drafted blind to the ones before it).
Guessing a whole block at once gets sloppy toward the end, so DSpark bolts a corrector onto the back to walk through it in order. That is the opposite bet from rivals that sold pure parallel guessing as the whole answer.
DSpark, which DeepSeek has now put into production on its V4 models, refuses the choice. It drafts the block in parallel for speed, then runs a small second head back over that block to re-inject the dependency the parallel pass threw away — a corrector that asks, token by token, 'given what came before, is this still the right guess?' A third head reports how confident the draft is, which lets the server spend more verification effort on requests likely to pay off and less on the rest, scaling to how busy the GPUs are at that moment.
The payoff is the part that matters: this replaced DeepSeek's previous drafter in live serving, not in a benchmark. Under real traffic it generates tokens 57 to 85% faster for each user. The same GPUs now clear work faster, so it pushes 51% more total traffic through the fleet at DeepSeek's normal speed target. For an inference provider, that 51% is a straight cut in the machines needed to serve the same demand. The company also open-sourced DeepSpec, an MIT-licensed toolkit for training your own drafter on open models like Qwen and Gemma, where its guesses run longer before the big model rejects one than the best prior open methods.
One number to distrust: some coverage repeats a 661% throughput figure. The paper's own footnote defuses it. That ratio only appears at a punishing speed target where the old drafter had all but collapsed, so it measures a near-zero baseline, not a 6.6x steady-state win. The steady win is the 51%, and it is already spreading: vLLM, the serving stack much of the open-model world runs on, has folded DSpark in natively.
Pull the MIT-licensed DeepSpec toolkit from DeepSeek's Hugging Face org and train a drafter against your own Qwen or Gemma target; if you already serve on vLLM, the DSpark path is wired in and you can benchmark it against your current setup directly.
Read the paper at arxiv.org →The lenses
The facts
Concepts
How this connects
Tap a node to open it