awesome_memory_management.md

memory management

• Tue, 12 Sep 2023 Efficient Memory Management for Large Language Model Serving with PagedAttention
  • PagedAttention
• Mon, 24 Mar 2025 Jenga: Effective Memory Management for Serving LLM with Heterogeneity
  • First, recent models often have heterogeneous embeddings with different sizes:
    • VLMs
    • Mamba
  • Second, to handle long contexts more efficiently, some new LLM architectures use only a subset of the prefix tokens to generate the next token.
    • sliding window