A Survey on Hybrid Caching Techniques to Reduce Latency in Large Language Model Systems (International Journal of Latest Technology in Engineering Management & Applied Science, 2026)
Large Language Models (LLM) have vast applications in diverse fields such as text summarization and generation, generative and conversational Artificial Intelligence (AI) and Natural Language Processing tasks. However, generation of content for each real-time task causes high computational cost and latency in LLMs.To address this drawback, the most effective solution proposed was - caching. Caching mechanisms were introduced to reuse a response instead of computing it for each redundant task. This survey explores various caching strategies from traditional key-value based techniques to the advanced hybrid strategies.The paper highlights the effectiveness of caching techniques in improving the overall performance of LLM systems. Through this survey hybrid caching mechanism is found to be most useful with an estimate of 15-25% reduction in latency and 10-20% improvement compared to traditional caching mechanisms.