Speculative decoding accelerates autoregressive generation by letting a faster draft model propose multiple tokens that a target model verifies in parallel without changing the target distribution.

This folder contains an end-to-end runnable speculative decoding fine‑tuning pipeline in which Llama‑3.2‑1B (Hugging Face) is trained on the Daring-Anteater dataset. This example focuses on training ...

Geng et al., Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning (NeurIPS 2023) — https://arxiv.org/abs/2305.13971. Applies the per-step valid ...