What Matters When You Build a Machine for Local LLMs
Five hardware factors, ranked. Capacity decides whether it runs at all. Bandwidth decides how fast it feels.
People about to spend real money on a machine for local models usually ask which GPU is fastest. The better question is which limit they will hit first, because that changes what they should buy.
Five things decide whether a model runs on your machine and how it feels to use. They are not equally important. Here they are in order, with the one that stops you cold at the top. Click any of them.
The order is the whole point. Capacity is first because it is binary: the model fits or it does not, and when it does not, nothing else you bought matters. Bandwidth is second because once the model fits, reading the weights over and over is what sets the pace of every word. Everything after that is tuning.
Enjoyed this? get in touch.