Memory Without a Cloud Paywall

An AI model has a limited memory, and that much is trivial. What matters is the consequence: in a running system, new content continually crowds out the old. The agent does not forget everything at once, but fresh material pushes older material out of view. If you want a decision made three months ago to still hold today, you have to anchor it, that is, store it durably and make it findable on purpose. That anchoring is what we call agent memory.

The common assumption is this: for an agent to anchor well and recall reliably, it needs the large cloud model. That is only half true, because agent memory consists of two parts, and only one of them is the large model.

First search, then formulate

When an agent answers a knowledge question, it happens in two parts. First the system searches: out of thousands of notes it pulls the few that fit the question. Then it formulates: from those hits it writes an answer that names its sources and states plainly where the notes do not cover the question. The large model sits in the second part. The first is decided by the embedding model, and that is exactly where many German-language corpora fail.

Does the model match meaning or words?

An embedding assigns a vector to every text fragment, and the search returns the fragments whose vectors lie closest to the question. The promise is that meaning is compared, not wording: someone asking for „Datenverschlüsselung im Ruhezustand" should also find the note that speaks of „at-rest encryption".

That promise only holds as far as the model has learned the meaning. Where it has not, similarity falls back to the surface, and the model compares identical words instead of identical meaning. This is precisely what happens in German, because most embedding models are trained on English. Across language and domain boundaries they then compare word forms rather than content, so the search runs but returns the wrong thing.

The German recall weakness is therefore not a side issue but the proof: an embedding is only as semantic as its training. The right lever is not to translate the corpus into English, but to use a multilingual model that actually knows the meaning; in our case that is bge-m3. With it, German recall in our test rose from barely usable to reliable, and that without the cloud.

Does the answer have to come from the cloud?

We measured the second part, the cited answer. We pitted a local model of the 30-billion class against a cloud frontier model, both on the same German corpus, having forced the local model into a fixed, machine-checkable answer format with a list of sources. The local model reached roughly 89 percent of the rated quality, it backed its statements with real documents rather than invented sources, and it was faster than the cloud call and free per query.

So that we do not talk it up: the judge was itself an AI, the sample was small, and on the hardest questions the cloud was ahead. The conclusion is therefore not that the local variant beats the cloud, but that it should be the default and the cloud the escalation for the exception. Built this way, you pay the cloud only for the exception and no longer for every routine question.

What this changes

Once both parts run locally, four things change at once. The content and the questions asked of it stay in house, and it is the questions in particular that often reveal more than the documents. No per-query bill arises. The local language becomes first class instead of the by-product of an English stack. And the hardware demand stays modest: the embedding model takes one to two gigabytes and the 4-bit-quantized 30-billion model around 18 to 20, so a single graphics card with 24 GB of VRAM carries both.

What Creaminds contributes

We build agent memory as a service, hosted as well as operated sovereignly on the customer's premises. The stack described here — multilingual retrieval, local synthesis with an enforced format, and the cloud only as escalation — is one we run ourselves and tailor to the customer's corpus. If you are facing the task of letting an agent reliably reach your internal knowledge, in your own language, in house, and without a per-query cost, talk to us.