Q-Filters, in depth
How the scorer performs
Q-Filters is a token-eviction scorer: it decides which parts of a long conversation are safe to drop from memory. These tables go past the headline compression number and into whether its guesses are actually good — attention-correlation accuracy and generation perplexity. Also on the docs page, with full methodology.
How well does Q-Filters predict what a model will actually attend to?
Spearman correlation with the model's real attention (higher is better), vs. the K-norm/L2Norm baseline. Full write-up →
| Scorer | Llama-3.2-1B | Llama-3.2-3B | Qwen2.5-7B |
|---|---|---|---|
| Q-Filters, calibrated (query-SVD) | +0.783 (100% sign-correct) | +0.863 (100%) | +0.850 (100%) |
| K-norm / L2Norm | +0.460 (94.5%) | +0.410 (94.6%) | +0.402 (86.6%) |
Does that translate to better generations?
Generation perplexity with eviction active (lower is better), calibrated vs. fallback. Full write-up →
| Model / budget | Calibrated | Fallback | Gap to fp16 closed |
|---|---|---|---|
| Llama-3.2-1B (fp16 baseline 4.050) | |||
| Budget 256 (~4×) | 8.476 | 13.358 | 52% |
| Budget 128 (~8×) | 16.307 | 23.645 | 37% |
| Budget 64 (~16×) | 25.933 | 31.274 | 20% |
| Llama-3.2-3B (fp16 baseline 3.305) | |||
| Budget 256 (~4×) | 5.076 | 7.264 | 55% |
| Budget 128 (~8×) | 10.046 | 14.909 | 42% |
Want the main compression and speed numbers instead? Back to the summary table →