Live·Open questions in longevity research
All news
Scientific Computing

AI researcher Kosmos rewrote its memory algorithm; a human rejected changes tailored to the test

14 November 2026· 261114011

AI researcher Kosmos rewrote its memory algorithm; a human rejected changes tailored to the test

On 13 November 2026, Edison Scientific, the company behind Kosmos, published results from its work on Kosmos’s memory. According to the company, in three days, Kosmos assembled a team of four AI assistants, fully evaluated 21 versions of the algorithm, and ran hundreds of small experiments.

Kosmos reads scientific papers, writes and runs code, and suggests hypotheses and follow-up experiments to researchers. A project can last months, while an AI model can use only a limited amount of conversation history at a time. To keep working, Kosmos replaces older messages with a summary. Each round of summarization can lose a date, a calculation result, or the reason for an earlier decision.

Edison assembled test questions from instances in which Kosmos had searched for information it had forgotten after compression. A human selected questions that could be answered from the earlier conversation. The researchers replayed conversation histories with one, two, or three rounds of compression, applied the algorithm under evaluation, and asked the original question. To assess conversational memory, they added 50 randomly selected questions from LongMemEval, a benchmark for remembering earlier conversations.

In one mode, the agent answered directly from the compressed history; in the other, it could search saved files for information. Searching the archive forced the agent to interrupt its current work. The memory algorithm used by the open-source AI agent Prime rewrote the earlier history into a single summary. It preserved information from scientific work well, but lost conversational details when answering directly. The researchers used it as the starting point for improvements.

A human worked with Kosmos to choose the tasks and metrics: the aim was to improve answer accuracy while maintaining compression and response speed. Kosmos then received access to the code and the tools to run experiments. According to Edison’s head, the company’s research team consists of six or seven people.

In intermediate versions of the algorithm, a researcher found changes tailored specifically to LongMemEval. The authors describe them as follows:

The code contained special rules that routed LongMemEval-style conversations through a separate processing branch.

The researcher ruled out this approach. Kosmos retained the useful idea of dividing the history into parts and developed a general-purpose version of the algorithm. Later, a human suggested a final adjustment for cases in which a single tool call loads many large images.

The final algorithm processes different parts of the earlier conversation in parallel, extracting facts, numbers, dates, and corrections. Retained notes remain unchanged through subsequent rounds of compression, so facts undergo fewer rounds of summarization. A brief account of the current work, covering goals, decisions, and unresolved questions, is updated separately. When space for notes runs out, the oldest groups are removed. The most recent messages are kept verbatim, and the full history remains in the archive.

According to Edison’s chart, with the same AI model, the final algorithm scored about 70% when answering the selected LongMemEval questions directly, compared with 4% for Prime. With file search enabled, the scores were closer: about 86% for Kosmos and 80% for Prime. The information the agent needed was more often already present in the compressed history it used to continue its work.

Originally published on Telegram by Ukhvat NewsView on Telegram
Sources
#kosmos#ai-agents#memory-compression#longmemeval#benchmark-overfitting