Run bigger models on the Mac you already have
Tell us your Mac and what you want to run. We'll show you which setting to use and how long a conversation will fit in your memory.
New here? What this actually does
As a chatbot reads your conversation it keeps notes on every word so far, so it doesn't have to re-read everything to write the next word. Those notes are called the KV cache, and they live in your Mac's memory. The longer the conversation, the bigger they get — and on a Mac, memory is the wall you hit first.
VeloxQuant-MLX stores those notes in a more compact form. Smaller notes mean longer conversations, or a bigger model, in the same Mac. This page works out which approach suits your machine.
Every number here is worked out live from the same code that ships in the tool itself, or read from real benchmark runs we've committed to the repository. Nothing is made up, and where a number is an estimate rather than a measurement, we say so.
Advanced — exact model shape
Only needed if your model isn't listed above. Picking a model sets these for you.
Where these numbers come from. The recommendation and the
cache sizes are calculated, not measured — they use the same rules as the
veloxquant recommend command
(veloxquant_mlx/tools/mac_recommender.py), and they tell you
when a method is unlikely to free real memory rather than quietly
overstating it. Conversation lengths and the page comparison are estimates
that assume memory grows evenly with length. The charts in step 3 are real
measurements committed under figures/. To measure your own Mac
instead of estimating, run python -m veloxquant_mlx benchmark.