Semantic Cache
Definition
Caching API completions by checking if new prompts are semantically similar to past prompts, saving costs.
Example Case
Retrieving a cached response for "how to write a loop" when user asks "writing loops".
Caching API completions by checking if new prompts are semantically similar to past prompts, saving costs.