Speculative Decoding & Rejection Sampling Simulator
Explore how small draft models speculate $\gamma$-token lookaheads verified in parallel by large target models without altering the output distribution. Observe exact rejection sampling probabilities $\alpha_i = \min(1, p/q)$, trace residual distribution sampling $p'(x)$ on first rejection, and analyze high-bandwidth memory (HBM) savings versus draft cost ratios.
Exact Target Distribution Preservation
Verify Leviathan et al. Theorem 1. When a draft token $x_j$ is rejected ($u_j > \alpha_j$), the simulator samples a replacement token from the normalized residual distribution p'(x) = max(0, p(x) - q(x)) / ∑ max(0, p(v) - q(v)). This ensures the marginal probability of emitting token $x$ is strictly identical to sampling directly from target model $p(x)$.
Lookahead Length ($\gamma$) vs Cost Ratio ($c$)
Modulate the $\gamma$ lookahead slider and observe the speedup curve (1 - α^(γ+1)) / ((1 - α)(γ·c + 1)). When acceptance rate $\alpha \approx 0.85$, increasing $\gamma$ from 1 to 5 drives speedup from 1.7× to 3.1×. However, beyond $\gamma > 6$, dimishing marginal returns and draft latency overhead flatten the throughput curve.
Domain Mismatch & The Slowdown Regime
Select Clinical Biochemistry Mismatch. When the draft model fails to anticipate specialized target domain vocabulary ($\alpha \approx 0.25$), speculative decoding costs more target compute than it saves. The empirical speedup drops below 1.0× (e.g. 0.72×), triggering the real-time slowdown alert.