Scaling language models to long contexts is often bottlenecked by the size of the key-value (KV) cache. In deployed settings, long contexts are typically managed through compaction in token space via summarization. However, summarization can be highly lossy, substantially harming downstream performance. Recent work on Cartridges has shown that it is possible to train highly compact KV caches in latent space that closely match full-context performance, but at the cost of slow and expensive end-to-end optimization. This work describes an approach for fast context compaction in latent space through Attention Matching, which constructs compact keys and values to reproduce attention outputs and preserve attention mass at a per-KV-head level. We show that this formulation naturally decomposes into simple subproblems, some of which admit efficient closed-form solutions. Within this framework, we develop a family of methods that significantly push the Pareto frontier of compaction time versus quality, achieving up to 50x compaction in seconds on some datasets with little quality loss.

The Claude C Compiler doesn’t mark the end of software or compiler engineering. If anything, it opens the door wider. The easier implementation gets, the more room there is for genuine innovation.

President Donald Trump accused former President Barack Obama of giving away classified information when he discussed aliens during a recent podcast appearance.
“He gave classified information, he’s not supposed to be doing that,” Trump told reporters Thursday aboard Air Force One.
Pressed on if that meant aliens were real, Trump said he did not know “if they’re real or not.”
“I can tell you he gave classified information, he’s not supposed to be doing that,” the president said. Trump went on to suggest he could get the former president “out of trouble” by declassifying the related information.
Obama was asked about extraterrestrial life earlier this month during an interview with liberal commentator Brian Tyler Cohen, and responded, “they’re real.”