Executive Summary
Google has released Multi-Token Prediction (MTP) drafters for its Gemma 4 family of open models. This new tool utilizes speculative decoding, pairing a lightweight drafter model with a larger target model to predict multiple tokens simultaneously. The initiative aims to solve the latency bottleneck in LLM inference, delivering up to a 3x speedup for developers without any degradation in output quality.
Key Takeaways
* Core Technology: Implements speculative decoding, where a small "drafter" model proposes several future tokens at once, which are then verified in a single pass by the main Gemma 4 model.
* Performance Boost: Delivers up to a 3x increase in tokens-per-second, significantly reducing inference latency.
* Quality Preservation: Output quality and reasoning logic remain identical to the base Gemma 4 model, as it retains final verification authority over the drafted tokens.
* Target Use Cases: Aims to improve responsiveness for real-time chat, coding assistants, agentic workflows, and on-device mobile applications.
* Efficiency: The drafters share the target model's KV cache, avoiding redundant computations and improving overall efficiency.
* Availability: The MTP drafters are available immediately on platforms like Hugging Face and Kaggle under an Apache 2.0 license and are supported by popular inference libraries including vLLM, MLX, and Hugging Face Transformers.
Strategic Importance
This release makes Google's open models more practical and competitive for low-latency, real-world applications, particularly on consumer-grade and edge hardware, thereby encouraging broader developer adoption.