Every few years, AI seems to reinvent itself. Expert systems gave way to machine learning. Machine learning gave way to deep learning. Deep learning gave way to transformers, foundation models and agents.

From the outside, it feels like everything changed.

Underneath, much of the mathematics didn't.

One mathematical idea I keep rediscovering is eigenvalues and eigenvectors. They arrive early in linear algebra as an abstract concept: a matrix stretches some directions and shrinks others, and those special directions are its eigenvectors. It doesn't sound like something that would shape the internet.

Four vectors before and after a transformation. Three of them rotate. The fourth, drawn in gold, points in the same direction as before but is twice as long. That direction is an eigenvector, and how much it grew is the eigenvalue.
Most directions change. Eigenvectors don't — they only scale.

It did.

In 1990, Latent Semantic Indexing projected documents into a lower-dimensional concept space so that a search for car could find automobile. Eight years later, PageRank used the same mathematics to answer a different question: which page deserves to appear first? Instead of analysing words, it analysed links.

Different problem. Same mathematics.

We rarely talk about eigenvectors when we build AI systems now, but they never left. Dense embeddings still rely on the idea that meaning lives in a lower-dimensional space. PCA still compresses data by finding the directions that carry the most information. LoRA assumes that adapting a large model often requires only a low-rank update.

Different tools. Different scale. The mathematics stayed.

Here's the part I find more interesting than the history.

Almost everything above relies on the largest eigenvalues, because they point to where the data spreads the most.

But the small end of the spectrum answers a different question.

Not where does the data spread...

Where does it hardly spread at all?

That turns out to matter whenever structure matters more than magnitude.

It made me wonder whether the same idea applies to agent memory. An agent's history is a trajectory through embedding space. Some stretches are routine. Others represent genuine changes in reasoning or intent.

I've started exploring whether the small end of the spectrum can distinguish between those two, and help decide what an agent should keep, what it should summarise, and what it can safely forget.

I don't know yet whether it outperforms simply summarising every few turns.

Whether this idea turns out to be useful or not, revisiting it changed how I think about agent memory. Sometimes the interesting question isn't whether the mathematics is new. It's whether we've found a new place to use it.