Library / SDK
Inference & Serving
GPTCache
1Semantic 2cache 3for 4LLM 5API 6calls
the six Ws · specification
W1
Who
Maintained by Zilliz (zilliztech).
W2
What
A semantic cache that reuses similar LLM responses.
W3
Where
A Python library wrapping your LLM client.
W4
When
When cutting cost and latency on repeated queries.
W5
Why
Serves cached answers for semantically similar prompts.
W6
With
An embedding model and a vector store.
W7
Watch
Company-maintained; open-source (MIT), library only. Runs in your app; the cache lives in stores you control.