Скачать с ютуб What is LLM quantization? в хорошем качестве

What is LLM quantization?

In this video we define the basics of quantization and look at how its benefits and how it affects large language models. Links mentioned: Perplexity vs model size https://github.com/ggerganov/llama.cp... Qualitative comparison https://rentry.org/quants #largelanguagemodels #quantization #artificialintelligence #chatgpt #llama 0:00 Intro 0:33 Basic concept 0:51 Benefits 1:30 Quantization 101 2:48 Impact on model size and perplexity 3:48 Impact on inference speed 4:25 Qualitative analysis