Neural networks can recognise images, generate language and find patterns at a scale no individual mind can match. Yet they also forget. Every useful representation keeps some distinctions and drops others.
I once called the limit created by that loss an “information entropy wall.” The name is vivid, perhaps too vivid. It sounds like a theorem already established by information theory. It is not. In this essay, the wall is a thought experiment: a way to ask what an intelligent system loses when it turns the world into a finite internal model.
The question becomes more interesting when we place three subjects beside one another: Shannon's idea of entropy, the strange efficiency of the human brain and Gödel's proof that certain formal systems cannot settle every statement expressible within them.
They do not combine into one grand proof. They do, however, illuminate the same discomfort. A system can be astonishingly capable and still fail to contain the whole world it is trying to describe.
What the “entropy wall” is trying to name
In information theory, entropy measures uncertainty in a probability distribution. It does not measure meaning, intelligence or disorder in the everyday sense. A high-entropy source has many possible outcomes that are difficult to predict. A low-entropy source is more predictable.
Neural networks transform input through layers of learned computation. Those transformations are useful precisely because the model does not preserve every feature equally. A vision system may ignore the texture of a wall in order to recognise the person standing in front of it. A language model compresses patterns across many texts rather than storing the intention behind every sentence.
This creates a trade-off. Remove too little noise and the representation remains cluttered. Remove too much and the signal you later need is gone.
That trade-off is the core of my wall metaphor. Once a model has discarded a distinction, later processing cannot recover it unless the missing information enters again through another channel.
The information bottleneck, carefully stated
The information-bottleneck framework studies how a representation can compress an input while retaining what matters for predicting a target. It provides a clean language for the tension between remembering and forgetting.
It is tempting to turn that framework into a universal story about deep learning: every layer compresses, compression always improves abstraction and the sequence eventually strikes a fixed mathematical wall. The evidence is not that simple. Researchers continue to debate when ordinary deep networks actually compress in this technical sense, especially for deterministic networks and continuous variables.
So the cautious claim is better. Finite models face capacity and representation limits. Noise can be indistinguishable from signal when the available evidence is weak. A model can overfit randomness, underfit structure or preserve the wrong detail. More parameters may expand what the system can express, but they do not guarantee that training will find the representation a new situation requires.
Attention mechanisms, residual connections, external memory and larger context windows can move the boundary. They do not abolish the need to choose what receives attention and what becomes invisible.
Why the brain feels different
The human brain is also finite. It forgets names, invents memories, misreads probability and spends most of its time working with crude approximations. Any account that gives the brain magical freedom from information theory is already wrong.
Still, the brain differs from a one-way computational pipeline in several important ways.
It is recurrent. Higher-level expectations alter lower-level perception. Attention changes the data that receives further processing. A first interpretation can be challenged by a second look.
It is embodied. We can move our eyes, touch an object, ask a question or change the environment to obtain information that was missing from the first input.
It is plastic and redundant. Information is distributed across changing networks rather than carried along one clean path. Damage can sometimes be compensated for, although never perfectly and not in every case.
It is social. Language, writing, culture and science let a biological memory borrow structures built by other minds and other centuries.
It also has metacognition, at least in an uneven form. We can notice that we are confused. We can question the frame of a problem, not merely search harder inside it. That ability often fails, but when it works it allows the system to request new evidence or rebuild the representation.
Artificial systems can also use recurrence, tools, external memory, feedback and interaction. The difference is therefore not a mystical essence reserved for carbon. It is a question of architecture, embodiment, training and access to the world.
Pattern recognition is not the whole of understanding
Artificial neural networks learn statistical regularities from data. Human perception also depends heavily on learned and evolved regularities. The contrast is not between statistics on one side and pure understanding on the other.
The practical gap appears in how context is assembled. A person looking at a complicated image can identify objects, infer relationships, recall a similar event, notice an emotional atmosphere and revise the interpretation after one sentence of explanation. The result draws on a life of multimodal experience and on a body that has acted inside comparable situations.
Modern AI has narrowed this gap sharply. It can integrate text, images, audio and tools. Yet fluent output still does not guarantee that the system has preserved the distinction relevant to the next unfamiliar case. A model can be locally brilliant and globally brittle.
That is another version of the wall: not a fixed bit limit, but a mismatch between the representation learned yesterday and the question that arrives tomorrow.
What Gödel actually proved
Gödel's first incompleteness theorem applies to consistent, effectively axiomatized formal systems strong enough to express basic arithmetic. Roughly speaking, such a system contains statements it can neither prove nor refute from its own axioms. The second theorem places a related limit on the system's ability to prove its own consistency.
This was a devastating result for the dream of one complete, contradiction-free formal foundation for mathematics. It is also frequently stretched far beyond its domain.
A neural network is built with mathematics, but that does not automatically make a trained network the kind of proof system Gödel studied. Classifying a picture or predicting a token is not the same as deriving a theorem from explicit axioms. Before applying incompleteness directly, we would have to define the formal system, its statements and its internal notion of proof, then show that Gödel's conditions hold.
Without that work, saying “Gödel proves the limits of neural networks” is not mathematics. It is analogy.
Why the analogy still matters
An analogy can be useful without pretending to be a theorem.
Gödel warns us against the fantasy of a sufficiently elaborate closed system that can certify every truth available in its own language. The warning belongs to a precise mathematical setting, but its intellectual pressure travels further. It asks us to be suspicious when a model is treated as if expansion alone could make it complete.
The same caution applies to human beings. A mind can step outside one frame, adopt a new rule and examine its former assumptions. The enlarged frame will have limits of its own. Openness moves a boundary. It does not erase every boundary.
Human beings also cannot prove their own reliability from inside their own minds. We use other people, instruments, institutions and repeatable experiments because introspection is not enough. Science is powerful partly because it turns cognition into a system larger than one brain.
Closed systems and open loops
My original contrast placed the neural network on the closed side and the brain on the open side. That division is now too clean.
A deployed model can call tools, search for evidence, receive corrections and update memory. A person can become intellectually closed, repeating a familiar model no matter what enters through the senses. Openness is not guaranteed by biology and closure is not guaranteed by software.
The more useful distinction is between a closed processing loop and an open one.
A closed loop receives an input, transforms it and returns an answer without any mechanism for noticing that the representation has failed.
An open loop can seek another observation, compare incompatible explanations, expose uncertainty and alter the terms of the task. It still has finite time and memory. It simply has a route by which missing information can return.
This is where metacognition matters. The valuable ability is not always knowing the correct answer. It is recognising when the current way of knowing is inadequate.
Where AI might go next
If this diagnosis is roughly right, progress will require more than larger static models.
Multimodal systems need to integrate perception without flattening every modality into the same convenient token stream. Memory needs to be revisable rather than merely longer. Systems need ways to test their own outputs against the world, identify uncertainty and ask for the observation most likely to change the answer.
Open-ended learning matters too. A model trained once and sealed away from later experience will age badly in a moving world. Interaction can help, although continual learning also creates new problems: unstable memory, manipulation, privacy risk and the possibility of absorbing error faster than truth.
Human-machine systems may be more productive than a contest over which side is superior. Machines can search and calculate at extraordinary speed. People can supply goals, lived context and responsibility. Neither contribution is pure. Both can fail. The design problem is to make their errors visible to one another.
A smaller conclusion
The “information entropy wall” is not an established law of neural networks. Human feedback does not let the brain escape entropy. Gödel does not prove that biological minds transcend artificial ones.
What remains is less triumphant and more useful.
Every intelligent system builds an incomplete representation. Every representation loses something. A system becomes dangerous to itself when it mistakes that necessary incompleteness for a complete picture of reality.
The pride of human intelligence is not that we have already left the cave. It is that, occasionally, we notice another wall, distrust the shadow we once called truth and start looking for a way out again.