Why Better Language Models Make Worse Models of Human Reading
As language models have gotten better at guessing the next word in a sentence, they have gotten worse at predicting how people actually read. That reversal sits at the center of a new paper by CDS Associate Professor of Linguistics and Data Science Tal Linzen and Byung-Doh Oh, a former CDS Faculty Fellow now an assistant professor at Nanyang Technological University, who argue that today’s strongest models have become too capable to serve as models of the human mind.
Like LLMs, human language comprehension uses prediction. When a word is easy to anticipate, people read it faster, sometimes skip it, and their brains respond to it with less activity. We can hear the beginning of a warning and start to react before the speaker even reaches the end of the sentence. Language models, for their part, produce a probability distribution over the next word as a matter of course. Linzen and Oh’s interest lies in the degree to which language models’ language abilities can stand in for humans’.
In the early years of LLMs, the similarity to humans increased as models improved. However, at some point around 2022, the trend turned. Models momentarily matched human performance, but then quickly overtook us, and ever since, as models keep getting better, the divergence continues to grow.
Linzen and Oh call this property superhumanness, and they trace one of its main sources to memory. Their paper, “To Model Human Linguistic Prediction, Make LLMs Less Superhuman,” published in Trends in Cognitive Sciences, the leading venue for reviews and perspective articles in cognitive science. It reviews the evidence and lays out a research agenda.
The first kind of memory where models outperform humans is long-term: the store of facts and phrasings a reader brings to a text. Consider the fragment “Two days later, the British astronomer Richard…” Most people have no idea how it continues. But a model knows there is exactly one famous British astronomer named Richard, and confidently finishes the sentence with “Carrington.” This is due to quantity of words seen. A child has encountered at most 100 million words by the age of twelve; a modern model like Llama 3 trains on 15 trillion words. Models also forget far less than people do, and can reproduce long passages verbatim after only a handful of exposures. In short, says Linzen: “Language models’ predictions are better than humans’ because language models simply know a lot more than humans do.”
The second kind of memory is short-term, and the mismatch here is even starker. In, e.g., a sprawling 19th-century Russian novel, humans often struggle to keep track of every character by name; models do so excellently without effort.
A third difference concerns ambiguity. When a sentence admits several readings, people tend to commit to one and discard the rest. A model, with far more capacity to spare, can hold the competing interpretations open longer. That makes it better at predicting the next word and, again, less like a human reader.
As models continue to improve, all of these trends will continue, making AI ever-less useful as a tool to understand humans. This is because the primary goal of LLM development, for the big commercial labs, is to simply make better models, which, since 2022, is at cross-purposes with modeling human cognition. “It’s not the goal of a company like OpenAI to develop a model that has the same resource limitations as a human,” said Linzen.
But it is the goal of cognitive scientists to model human cognition, and to this end, Linzen and Oh argue for models intentionally built with human-like memory, whether by limiting training data along the lines of the BabyLM Challenge, which caps models at 100 million words, or by returning to recurrent architectures that compress the past into a fixed-size representation rather than storing it whole. They also call for training objectives that spread probability across words with similar meanings, and for model-editing methods that strip out knowledge a typical reader would never have. Because the existing evidence is largely observational, they also press for new human experiments designed to measure memory’s role in reading directly.
And in fact, they project, this kind of work will be helpful for more than just modeling human cognition. A sharper account of how people read does indeed help us understand humans better, with possible applications like informing the design of more readable text and a better understanding of reading disorders. But additionally, on the engineering side, Linzen has been pursuing a separate line of work on models that simulate a human learner, limitations and all. If you train a teacher model against that kind of simulated student, he said, it might explain things in a way a person can actually follow, instead of producing a wall of text — ultimately producing more useful AI.
By Stephen Thomas
