PagedAttention
Definition
An memory management algorithm for vLLM reducing VRAM waste in Key-Value caches by partitioning them like OS pages.
An memory management algorithm for vLLM reducing VRAM waste in Key-Value caches by partitioning them like OS pages.