The Memory Deadlock of Large Models: Why Can AI Remember All Human Knowledge but Never 'Yesterday'?

Author: Hugo, Robonaissance

Compiled by: TechFlow

TechFlow Editor's Note:From early Atari game agents to large language models with multi-million-token context windows, artificial intelligence has always faced a fundamental physical contradiction when making continuous decisions across time: models can easily absorb the vast static knowledge already accumulated by humanity, yet in dynamic interactive environments they cannot carry every detail of the past losslessly with them.

Under the hard constraints of limited compute and context windows, how to compress the past while accurately extracting the core state that determines the future is not only a technical breakthrough for solving perceptual aliasing, but also the underlying logic by which today's autonomous agents move toward truly persistent memory.

The State Paradox in a Frozen Frame

This is Issue 3 of the series 'Memory, Forgetting, and the Self in Machines': when an agent cannot carry its entire past with it, how does intelligence actually act across time.

Try freezing a classic game of Pong at any single frame.

The frame contains only two paddles, a ball, and the current score. If you were asked to say where the ball is flying, you could not do it; even if a machine were shown the same static frame, it likewise could not tell.

Researchers Matthew Hausknecht and Peter Stone, in their 2015 paper on agents learning to play Atari games, put it bluntly: 'The current single-frame screen reveals only the positions of the paddles and the ball, but not the ball's velocity.'

They then explained why this mattered: 'Knowing the direction of the ball's movement is a key factor in determining the paddle's optimal position.'

This frozen frame is accurate: every detail in it is exactly where it should be. But this single frame alone is still not enough to play the game.

What the sensor captures in the present instant is merely an Observation.

The true global picture on which the agent's next action depends, however, is the State.

In Pong, Observation and State are separated in every single-frame slice, because the ball's direction of movement is a fact about several past instants, and no isolated instant alone contains it.

When the immediate input in front of the system is not enough for decision-making, a well-performing system must carry some information from its past into the present.

But what it carries can never be the complete past. It carries only a summary, and the choices in that summary depend entirely on what it needs to predict next.

That choice determines the informational basis available at later moments. Any detail omitted by the summary can never be acted upon by those later moments.

The Four-Frame Memory Compromise

The system Hausknecht and Stone were responding to was the DQN agent published in 2015 in Nature by DeepMind's Volodymyr Mnih et al.

DQN could learn to play dozens of Atari games purely from screen images, and the paper's authors had already encountered the frozen-frame problem during early development.

They wrote in the methods section: 'Since the agent observes only the current screen, the task is partially observable, and many simulator states are perceptually aliased (Perceptually Aliased).'

In other words, the current screen image alone cannot tell the agent the full situation.

Perceptual aliasing is a precise academic description of an everyday problem. Two very different situations, for example the ball moving toward your paddle or the ball moving away from your paddle, produce exactly the same image in a static frame.

A system that sees only images is forced to treat them as the same situation; if the two situations require opposite actions, the system will inevitably fail in one of them.

Their remedy was direct. Instead of receiving a single screen image, the network took the most recent frames stacked together as input.

The default setting uses 4 frames, although the algorithm also adapts to 3 or 5 frames.

Once four frames are placed side by side, the ball's motion appears directly as a change in position. Direction of movement is no longer an abstract memory concept, but a visual property that can be seen directly.

The paper did not call this stack of frames memory. It was essentially just an input channel slightly wider than a single moment.

Perceptual Aliasing and Physical Hard Limits

This input width has an extremely hard physical boundary, and Hausknecht and Stone precisely described where this boundary lies.

They wrote: Because DQN is trained using inputs composed of the four states most recently experienced by the agent, “if a game requires the player to remember events older than the last four screens, DQN will not be able to master the game.”

For DQN, the history beyond the fifth-to-last screen simply does not exist within its input window.

This fixed sliding-window mechanism directly reveals the core contradiction in machine intelligence's time-series processing: no matter how wide the window is expanded, information beyond the boundary is physically erased.

Source: TechFlow