InferenceInference performance, quantization and cost optimization

上下文缓存

Reusing computed results for identical prefix content to cut repeated input cost.

Context caching heavily discounts cached tokens (e.g. DeepSeek's cache-hit price is about 1/10 of a miss), a big win for multi-turn chats, fixed system prompts and long-document Q&A.

Related terms