Why it matters
The NDSS-backed research identifies six inference-time cache attacks across vLLM, SGLang, GPTCache, and related stacks. Weak prefix and image cache keys plus semantic near-match errors can make distinct inputs share cached state, enabling poisoned responses, information leakage, and moderation bypass; the authors provide experimental artifacts and vendor disclosures.
My takeaway: Use canonical serialization and collision-resistant keys that include tenant, model, tokenizer, policy, and configuration identity. Isolate caches across trust boundaries, avoid semantic caching for authorization or safety decisions, verify hits before reuse, patch affected frameworks, and add adversarial collision and opposite-intent cases to regression tests.