GGUF

Definition

GPT-Generated Unified Format. A binary file format optimized for fast loading and running quantized models on consumer devices.

                Running llama-3-8b.Q4_K_M.gguf inside LM Studio.            

Local AI