Local AI: How To Get 90+ Tok/S On Qwen

Qwen3.8 27B just hit the kind of local speed number that makes paid AI feel a lot less inevitable.

Ninety one tokens per second is not magic. It is cache, runtime, compression, and one very important draft path.

Qwen3.8 27B is a twenty seven billion parameter vision and coding model that can be squeezed into serious local hardware, then pushed much faster with the right serving stack. The headline number is not one setting anyone can copy blindly, because the twenty one gigabyte fit and the ninety one token speed come from different parts of the same ecosystem. That is what makes it useful: the real lesson is how quantization, FP8 cache, prefix caching, CUDA graphs, vLLM, and multi token prediction change what local AI can feel like. For developers, this is the difference between a model that merely loads and a model that might actually replace paid API usage for everyday work.

0:00 The 91 Token Run That Sounds Fake
0:51 What Qwen3.8 27B Actually Is
1:45 The Honest Version Of The Headline
2:47 What Actually Fits, And What Quantization Costs
3:49 The vLLM Knobs That Did The Work
5:05 Multi Token Prediction, And Its Catch
6:15 The GPU Ladder, 8GB To H200
7:19 Why Macs Load It But Do Not Fly
7:55 The Coding Benchmarks
8:53 Three Levels To Run It Yourself
10:04 The Context Trap
10:46 What This Means For Paid AI
11:35 First It Fits, Then It Gets Fast

Watch next:
Is this model actually any good, or just fast –
The run that goes faster than 91 tokens per second –
A repo that doubles your tokens per second –

Sources and workings:

@CodingHorizon on X:

Narration is synthesised.

#LocalAI #Qwen #vLLM

source