This article is Part 8 of 15 in the Generative AI in Depth series. A 70B parameter model in FP16 weighs 140GB. That won’t fit on any consumer GPU. Quantization is how you shrink it to 35GB (Q4...

This article is Part 7 of 15 in the Generative AI in Depth series. The DeepSeek-R1-Distill-Qwen-7B model — a 7B model beating GPT-4 on maths olympiad problems — made distillation impossible to...

This article is Part 6 of 15 in the Generative AI in Depth series. Pre-training gives you a model that knows a lot. Fine-tuning gives you a model that does what you want. This article explains...

This article is Part 5 of 15 in the Generative AI in Depth series. You can run Gemma 4 12B inference on a single A100 with 23 GB to spare. Training the same model requires at least four A100s....

This article is Part 4 of 15 in the Generative AI in Depth series. Before you can run an LLM, one question matters above all others: does it fit? Not approximately — exactly, in bytes, on the ...