LLMs Do Not Remember Facts, They Encode Patterns

When people talk about large language models (LLMs), they often say that the model “stores knowledge” in its neural network weights. This sounds intuitive and convenient, but it is also deeply misleading. Treating an LLM as if it were a database full of facts leads to confusion about both its capabilities and its limitations.

A much more accurate picture emerges if we stop thinking about LLMs as knowledge containers and instead see them as pattern engines that have learned how ideas, statements, equations, and explanations tend to transform into each other.

To understand why this distinction matters, we need to look at what actually happens when a language model produces an answer.

• Why an LLM Is Not a Knowledge Database
• Even Scientific Formulas Are Patterns
• What Actually Lives Inside the Model
• Why External Knowledge Systems Are Necessary
  • Vector Retrieval: Finding Similar Text
  • Graph Retrieval: Recovering Structure
• Why This Matters

Why an LLM Is Not a Knowledge Database

A classical knowledge system stores information explicitly. A database entry might look like this:

(country="Austria", capital="Vienna")

If we ask the system for the capital of Austria, it simply performs a lookup and returns the stored value.

A language model does something fundamentally different. It does not retrieve a stored record. Instead it predicts the most probable continuation of a text sequence based on statistical patterns learned during training.

At the outermost level, the system produces tokens by sampling from a probability distribution over possible continuations. Mathematically this is often written as

which describes how likely a particular token is given the tokens that came before it. During training the model learns this probability distribution by adjusting its internal parameters to minimize a loss function. In practice this is typically the cross-entropy loss between the predicted probability distribution and the actual next token in the training data. Gradient descent is then used to update billions of parameters so that the model gradually becomes better at predicting the next token in a sequence. Over many training iterations this process shapes the internal representations of the network so that useful linguistic and conceptual patterns emerge. The model is therefore not explicitly programmed with rules or facts. Instead its internal structure is optimized purely through exposure to vast amounts of text.

However, this formula alone can be misleading. If language models were only performing classical statistical estimation over token frequencies, they would behave much more like sophisticated n-gram models or Bayesian predictors. Such systems can reproduce local statistics, but they cannot generalize well and they cannot discover deeper structures in language.

The crucial difference is the neural network itself. A modern transformer model contains many layers of nonlinear transformations and attention heads that dynamically route information across the sequence. These mechanisms allow the network to detect relationships between words, concepts, and symbolic expressions that may be far apart in the text.

The key mechanism that enables transformers to capture long-range relationships is called attention. Instead of processing tokens strictly one after another like earlier neural networks, the model dynamically decides which parts of the input sequence are relevant when interpreting a particular token. In practice each token generates queries that search for relevant keys among all other tokens in the sequence. The resulting weighted combinations of information allow the model to connect words that may be far apart in the text. This mechanism is what allows modern language models to track references, follow arguments across paragraphs, and relate mathematical symbols to explanations appearing elsewhere in the context.

The probability distribution above is therefore only the final sampling interface of the system. Behind it lies a very large nonlinear pattern recognition machine. During training, the neural network learns internal representations that capture regularities in language, mathematics, explanations, and reasoning patterns. Crucially, what the model learns are patterns, not explicit facts. The training process does not insert statements like “Vienna is the capital of Austria” into a memory structure. Instead it adjusts billions of parameters so that certain regions of a very high‑dimensional representation space correspond to recurring conceptual relationships observed in the training data.

When the model answers a question like “What is the capital of Austria?” it does not retrieve Vienna from a memory table. Instead the network transforms the prompt through these learned representations until the sequence of tokens corresponding to the word “Vienna” becomes overwhelmingly likely under the learned patterns. In practice most tokenizers do not even operate on full words, but on sub‑word fragments, so the model is assembling the answer piece by piece according to the patterns it has learned.

The difference might sound subtle, but it has deep consequences. Databases store facts explicitly. Language models instead learn structures in which certain statements naturally follow from certain contexts.

Even Scientific Formulas Are Patterns

This becomes even clearer if we look at something that appears to be very precise: a physics equation.

Consider the equation

At first glance it might seem as if the model simply memorized this formula, much like a bad student who has learned a line from a textbook without really understanding what it means. But that interpretation is misleading. The equation itself is not the knowledge. It is only a symbolic representation of a deeper concept: force describes the change of momentum over time.

To see why this matters, it helps to compare two kinds of understanding. A student who memorizes the formula $F = \frac{\mathrm{d}p}{\mathrm{d}t}$ may be able to reproduce the symbols on an exam, but the expression itself is just a sequence of characters to them. A physicist, in contrast, does not think primarily about the letters or the notation. For them the equation activates a much richer conceptual structure.

When a physicist sees this expression, it immediately connects to a broader pattern of how the universe behaves. Ideas about dynamics, momentum, and interaction come into play. In modern physics this also touches deeper principles: symmetries of space and time, conservation laws, and the structures described by symmetry groups. The equation is only one compact way of encoding these relationships. Mathematics is essentially the language we use to describe those patterns precisely.

In other words, the formula is not an isolated statement. It is a symbolic gateway into a network of concepts describing how physical systems evolve.

An LLM learns something somewhat analogous on the linguistic level. It does not store the equation as a static mathematical object. Instead it learns the linguistic and symbolic patterns connecting force, momentum, change, Newtonian dynamics and the notation used to express those relations.

Because these models are trained on an enormous portion of written human knowledge, they are exposed to a vast range of explanations, arguments, analogies, and reasoning styles. What the network therefore absorbs are not individual statements, but recurring patterns of thinking: how humans explain physics, how they derive formulas, how they reason about systems, and how concepts connect to each other. Over time the training process shapes a high‑dimensional representation space that reflects many of the cognitive patterns present in human discourse.

When it writes $F = \frac{\mathrm{d}p}{\mathrm{d}t}$ it is reproducing a learned mapping between these representational forms. The model has internalized the pattern linking them, not the equation as an isolated fact.

This is why language models are surprisingly good at rewriting equations into explanations and explanations back into equations. Because they have learned the structural patterns connecting these representations rather than memorizing individual statements, they can often generalize those patterns to new situations. When faced with a problem they have never seen before, the model can still apply similar reasoning structures it encountered during training, which is why LLMs are sometimes capable of solving entirely new problems that were never explicitly present in their training data.

What Actually Lives Inside the Model

Inside the neural network there are no explicit facts, rules, or entries. Instead there is a high‑dimensional parameter space that encodes regularities of language, concepts, and symbolic relations. One can think of this space as a vast hyperspace in which related meanings, explanations, equations, and narratives occupy nearby regions. During training the model gradually shapes this hyperspace so that patterns that frequently appear together in human communication become geometrically aligned.

Importantly, the intermediate layers of the neural network learn these structures largely on their own during training. No human explicitly tells the model which internal neurons should represent which concepts. Instead the network discovers useful internal patterns because doing so improves its ability to predict the next token. These internal features therefore do not necessarily correspond to the neat conceptual categories humans might use to organize knowledge. The model may capture many subtle correlations present in the training data — sometimes meaningful conceptual relationships, sometimes statistical associations that humans would not consciously describe.

Modern architectures are also intentionally designed to prevent the model from simply memorizing the training data. Neural networks contain narrow information pathways and compression steps that force the system to represent information efficiently. If the network could simply memorize every sentence it saw, it would fail to generalize. This problem is known as overfitting: the model would reproduce training examples perfectly but perform poorly on new inputs.

Because of these architectural constraints, the model has little choice but to learn reusable patterns instead of storing individual facts. In other words, the structure of the network itself encourages the discovery of general relationships rather than direct memorization.

A useful mental model is that the weights define a transformation landscape. Certain prompts push the internal state of the network into regions where specific continuations become highly probable. If a prompt mentions “capital” and “Austria” the internal representation of the prompt moves into a region of this hyperspace where the continuation corresponding to the word “Vienna” becomes highly probable, most likely activating zones representing cities, capitals, governmental systems, vacations, etc. on the way. But this is not a discrete memory. It is more like an attractor in a probability field.

One of the most striking discoveries in modern AI research is that the capabilities of these models follow relatively predictable scaling laws. As the size of the neural network, the amount of training data, and the available computation increase, the performance of the model improves in a smooth and often surprisingly regular way. Larger models tend to discover richer internal representations and capture increasingly subtle patterns in language and reasoning. At certain scales new capabilities appear that were not obvious in smaller systems. This phenomenon, sometimes described as emergent abilities, is one reason why very large models can perform tasks that smaller models struggle with, even though they are trained with the same fundamental objective of next-token prediction.

The model therefore behaves less like a database and more like a system that has learned how concepts tend to follow each other.

Why External Knowledge Systems Are Necessary

Because LLMs operate through pattern reproduction rather than fact retrieval, they are not ideal sources of authoritative knowledge.

The model can generate extremely plausible statements that were never true in the first place if those statements match the typical reasoning or explanation patterns found in human language. In other words, the model can produce answers that sound exactly like something a knowledgeable human might say even when the underlying statement is incorrect or incomplete.

Interestingly, something very similar happens in human thinking. People sometimes believe they understand a topic because they can reproduce the usual explanation pattern associated with it. Only when they try to verify the statement or derive the result do they discover that their understanding was incomplete. In that sense, the failure mode of LLMs is not entirely foreign - it mirrors a common limitation of human reasoning as well.

For applications where factual accuracy matters, the model therefore needs access to external information sources. This is the motivation behind Retrieval Augmented Generation (RAG).

In a RAG system, the language model does not rely solely on its internal patterns. Instead, it receives relevant documents retrieved from an external knowledge base and reasons over them while generating the answer. The architecture then becomes conceptually simple. A retrieval system finds relevant information, and the language model acts as a reasoning engine that interprets and synthesizes that information.

This division of labor mirrors how humans work. A scientist does not memorize every paper ever written. Instead they consult references and then reason about the information they find. Humans routinely perform their own form of retrieval augmented reasoning: they look up articles in encyclopedias or on Wikipedia, consult textbooks or lexica, and use formal tools such as mathematics to verify whether a statement is actually correct.

Another remarkable property of large language models is in-context learning. Even though the models weights remain fixed after training, it can temporarily adapt its behavior based on examples provided directly in the prompt. If a prompt includes several demonstrations of how a task should be performed, the model often continues the pattern correctly for new inputs. In effect the model performs a form of short-term learning inside the context window. The internal representations inferred from the prompt guide the generation process without requiring any permanent update to the model parameters. This ability further illustrates that the model operates by reproducing patterns rather than retrieving stored rules.

Vector Retrieval: Finding Similar Text

Most RAG systems rely on vector embeddings to retrieve relevant documents.

Text passages are converted into vectors in a high‑dimensional space. When a user asks a question, the system computes the embedding of the query and searches for passages whose vectors are nearby.

What does “nearby” mean in this context? During training, embedding models learn to place pieces of text that are used in similar contexts close to each other in this space. The geometry of the space therefore begins to encode meaning. Sentences that talk about related ideas tend to end up in neighboring regions, even if they use different words. At the same time, this space often captures stylistic and rhetorical patterns as well. Technical explanations cluster differently from casual descriptions, and scientific writing occupies different regions than narrative text. You can spot the difference in the vector embeddings of this blog

In other words, the high‑dimensional embedding space simultaneously encodes aspects of semantics, style, and conceptual associations. Similarity between two vectors is typically measured using cosine similarity or related metrics, which essentially check whether two vectors point in a similar direction in that space. Performing a nearest neighbor search in this space yields all semantically similar statements inside the knowledge base (I use this for example for the suggested articles on the bottom of every page)

It is also worth noting that embedding vectors themselves are often generated using transformer models very similar to the LLMs that later consume them. These models learn to map text into this geometric representation so that related meanings occupy nearby regions of the hyperspace.

This approach works very well when the answer is contained in text that is semantically similar to the query.

However, similarity is not the same as structure.

Many difficult questions depend on relationships between entities rather than simple textual similarity.

If the relevant information is spread across multiple documents that describe different parts of a system, vector retrieval may return fragments that are individually related to the question but fail to capture how those fragments connect to each other.

Typical backends are databases like pg_vector in PostgreSQL or database systems like ChromaDB.

Graph Retrieval: Recovering Structure

Graph‑based retrieval addresses this limitation by representing knowledge as a network of entities and relationships. Instead of storing only text chunks, the system builds a graph where nodes represent concepts or objects and edges represent relationships such as causation, dependency, or hierarchy. When a query arrives, the system retrieves a relevant subgraph rather than a collection of independent text fragments. This explicit structure makes complex reasoning easier. If the system already knows that

component A depends on component B

and that

component B failed during a redesign

then the reasoning path connecting those events is already encoded in the graph.

The language model can then focus on interpreting the structure rather than reconstructing it from scattered prose. The model can traverse such graph structures iteratively: it can follow relationships from one node to the next, interpret the intermediate results and then decide which connections to explore next. By repeating this process over multiple steps, the model can perform multi‑hop reasoning across the graph.

The relationships stored in such graphs often resemble what is known in the semantic web world as RDF triples. These triples represent knowledge as simple subject‑predicate‑object statements, for example:

("Vienna", "is capital of", "Austria")
("Electron", "has property", "charge")
("Component B", "failed during", "redesign")

When many such triples are connected together they form a rich knowledge graph that captures relationships between entities. Graph databases such as Neo4j are commonly used to store and query these structures efficiently.

Interestingly, the extraction of these triples from unstructured text is often performed using the same kind of transformer models discussed earlier. LLMs can read documents, identify entities and relationships, and convert them into structured graph representations that can later be used for graph‑based retrieval and reasoning.

Why This Matters

The deeper lesson is that modern AI systems work best when we separate three roles.

• External systems store information.
• Retrieval systems locate relevant pieces of that information.
• Language models reason about the retrieved material and transform it into explanations, summaries, or decisions.

When these components are combined, the language model becomes something closer to a cognitive engine operating on a structured information environment. In practice this means that asking a standalone LLM factual questions without any grounding is often the wrong way to use the technology. The model itself is not designed to be the authoritative storage location of knowledge. Its real strength lies in interpreting patterns, combining ideas, performing reasoning steps, and synthesizing information once the relevant data has been supplied by retrieval systems such as RAG or GraphRAG.

Interestingly, the much‑discussed phenomenon of “hallucinations” is closely related to this capability. The same mechanism that allows the model to generate plausible statements beyond its training examples is what enables creativity and generalization. If the system were restricted to reproducing only statements that appeared verbatim in its training data, it would behave like a database lookup and would be incapable of solving new problems or combining ideas in novel ways.

In that sense hallucinations are not purely a bug, they are a side effect of the very property that makes these systems powerful. When they appear problematic, it is often a sign that the system is being used without proper grounding. Once external retrieval systems provide the factual information and the LLM is used primarily for reasoning and interpretation, the architecture begins to resemble a much more robust cognitive system.

This same mechanism can deliberately be used as a feature. Because the model can generate plausible variations and speculative ideas, it can be used to explore creative solution spaces. If the system is connected to verification tools - for example a mathematical proof assistant, a symbolic solver, or a simulation toolkit - the model can propose candidate ideas while the external tool checks whether they are actually correct. In this way hallucination becomes a generator of hypotheses while external tools provide validation. This pattern is increasingly used in research systems where LLMs propose conjectures, derive candidate formulas, or sketch solution paths which are then verified automatically.

It is therefore helpful to think of the LLM as a machine that has learned how ideas move - how arguments unfold, how explanations are constructed, and how pieces of knowledge connect to each other. And once this reasoning engine is connected to reliable sources of information and verification tools, its ability to analyze, explore, and synthesize knowledge becomes extraordinarily powerful.

This article is tagged: Artificial Intelligence, Tutorial, How stuff works, Machine learning, LLM