Get the Brief
AmazonResearchMateriality 2

AWS shows bi-directional KV transfer in vLLM cutting TTFT

AWS describes bi-directional KV cache transfer in vLLM that lets prefill and decode nodes reuse computed context across turns. It reports time to first token improvements of up to 1.6x for Qwen3-32B and 3.3x for GLM-5-FP8 on EC2 P5en instances with EFA.

First seen 19 Oct, 16:03 UTC on AWS Machine Learning blog2 sourcesLast update 3h ago

What the sources say

Linked, never rewritten. Official means the lab itself.

Official2

AWS Machine Learning blog· 19 Oct, 16:03Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLMAWS Machine Learning blog· 19 Oct, 16:05Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

Press0

No independent coverage yet.

How it unfolded

Every source in the order it appeared. Times in UTC.

19 Oct 16:03AWS Machine Learning blog Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM19 Oct 16:05AWS Machine Learning blog Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

Keep up with Amazon