Researchers compress documents into compact model memory

Researchers Yifan Wang and Dejing Dou described a method for compiling a document into a compact representation that a language model can use when answering questions. Their approach, called MII, creates a 56-token state and translates it into embeddings for a reader model whose weights remain unchanged.

A related version, Residual MII, combines that global memory with selected passages from the original document. Tests covered Llama, Qwen and Mistral models. On 7,405 HotpotQA questions, the researchers reported higher exact-match accuracy than a full-context baseline while using roughly seven percent of its attention computation.

That figure measures attention operations during the evaluated task, rather than the complete cost of compiling, storing and serving documents. The method is a learned aid for answering questions, so its results should not be read as lossless document compression.