Workflows & Automation

Latent context compression lets agents avoid requiring tools to speak in text.

Coca-Cola compresses LLM agent tool observations into latent tokens, cutting tokens 70-89% and runtime 24-56% while keeping answer quality high.

Key Takeaways

  • Coca-Cola compresses LLM agent tool observations into compact latent tokens instead of lengthy text, reducing processing while preserving decision-relevant information.
  • It uses Action-Preserving Training to align the agent's next-action distributions under compressed and full observations, keeping the target agent's behavior consistent.
  • On four web-search QA benchmarks across Qwen and GLM model families, it lowers token consumption by 70.01% to 89.24% and shortens end-to-end runtime by 24.45% to 55.97%, while improving answer EM and F1 scores.
  • The method shifts agent context design from readable text to decision-ready signals, making tool observation pipelines lighter without sacrificing tested QA answer quality.
  • Future work will explore extending latent tool observations to more agent workflows and determining how much of the tool layer can move beyond plain text.

Coca-Cola Compresses LLM Agent Tool Observations, Cutting Tokens 70.01% to 89.24%

Meta description: Coca-Cola turns LLM agent tool observations into compact latent tokens, reducing token use 70.01% to 89.24% and runtime 24.45% to 55.97% in web-search QA tests

Tags: Coca-Cola, LLM agents, context compression, Qwen, GLM

Featured image: Abstract friendly illustration of text observations being folded into compact latent token blocks, no real people, no logos, no text.

A New Route for Agent Tool Observations

Coca-Cola LLM agent compression is aimed at builders of LLM agents that use web-search tools. Coca-Cola, or Context Compression via Continuous Latent Tokens, was introduced as a method that lets agents consume tool observations as compact latent representations instead of lengthy text. The gain is less token processing and shorter end-to-end runtime while keeping the agent’s decision behavior aligned with the full observation.

How It Preserves the Next Action

The method does not try to reconstruct the original tool output. Instead, it learns to preserve the information that matters for the agent’s next decision. Coca-Cola uses Action-Preserving Training to align next-action distributions under full and compressed observations while keeping the target agent frozen. In practical terms, the compressed observation is trained to be good enough for the agent to choose the same kind of next step.

What the Benchmarks Show

Across four web-search QA benchmarks, Natural Questions, TriviaQA, 2WikiMultiHopQA, and HotpotQA, and across two model families, Qwen and GLM, Coca-Cola achieves 70.01% to 89.24% lower total agent token consumption and 24.45% to 55.97% shorter end-to-end runtime. It also achieves higher answer EM and F1 than the uncompressed agent in every evaluated setting. EM and F1 are common answer-quality metrics.

Why It Matters for LLM Agents

In many agent workflows, tool observations are long, but only parts of them influence the next action. Coca-Cola shifts the design point from readable text to decision-ready signal. The key idea is that tool observations need to support the agent’s decisions; they do not have to remain readable text. For builders, this means the context pipeline can become lighter without giving up answer quality in the tested QA settings.

Open Questions and Next Steps

Feedback and discussion are welcome, especially on extending latent tool observations to other agent workflows. The broader question is how much of an agent’s tool layer needs to stay as text, and how much can be transformed into compact latent tokens that preserve the next action.

Comments (0)

  1. No comments yet. Be the first to share what worked for you.

Leave a comment

Comments are reviewed before they appear. Your email address is not published.