Field Notes

Speculative-Decoding

Tuning someone else's speedup

A published drafter promised 2.7–3.4× on a 27B model. I got 2.4× — but only after retuning the one number its instructions told me to set, and only because I checked the log instead of the throughput.