Why Can’t You Say “One Hour Was Lasted by the Meeting”? Language Models Help Reveal the Answer

English speakers never explicitly learn that some verbs resist the passive voice, yet they know it anyway. Say “the meeting lasted one hour” and everything’s fine; flip it to “one hour was lasted by the meeting” and native speakers recoil, even though no teacher ever taught them the rule. A new study from NYU Linguistics PhD student Cara Leong and CDS Associate Professor of Linguistics and Data Science Tal Linzen uses neural network language models to probe how learners might pick up this kind of implicit knowledge from the language they hear around them.

The paper, “Manipulating Language Models’ Training Data to Study Syntactic Constraint Learning: The Case of English Passivization,” published in the Journal of Memory and Language, tackles a specific kind of learnability problem related to passivization. Passivization is productive in English — speakers freely convert “John loves Mary” into “Mary is loved by John,” and they can do it even for verbs they’ve barely encountered. Hear “defenestrate” once and you can produce “the editor was defenestrated by the writer” without hesitation. But some verbs, despite being common and transitive-looking, refuse the passive: last, cost, resemble. How do learners know?

Two prominent hypotheses have circulated in the literature. The entrenchment hypothesis says learners track statistical patterns: if you hear a verb thousands of times in the active but never in the passive, you eventually conclude it can’t be passivized. On the other hand, the affectedness hypothesis points to semantics: verbs whose objects undergo real change (hit, drop, break) tend to passivize, while verbs with unaffected objects (last, cost) don’t. The trouble is that in natural English these two factors are tangled together, making it nearly impossible to study them separately in human learners.

Leong and Linzen trained small GPT-2 models on 100 million words of English — roughly the amount a 13-year-old has been exposed to. The models’ judgments of which verbs sound acceptable in the passive correlated with human ratings at r = 0.9.

“I was actually pretty convinced that language models were going to be really different from people initially,” Leong said. “Their learning process is not super similar to the way people learn, and they seem to be more general conclusion-drawers than people. So it was really interesting to realize they were able to do this in a way that was pretty similar to people.”

The researchers then ran a series of interventions by altering the training data to disentangle the entrenchment hypothesis from the affectedness hypothesis. In one experiment, they took a normally passivizable verb like drop and deleted most of its passive occurrences from the training corpus, matching its active-to-passive ratio to that of an unpassivizable verb like last. The models trained on this altered data rated drop as less passivizable — evidence that frequency asymmetries do shape the learned pattern. But the effect only went so far: the mutated verbs never became as unpassivizable as their targets, suggesting frequency alone doesn’t tell the whole story.

A second experiment tested the semantic side. The researchers took sentences originally containing an affected-object verb like drop and swapped in last, so the model saw last co-occurring with the kinds of subjects and objects typically found with more agentive verbs. This shifted the models’ judgments too, though the size of the effect depended on the particular verb. A third experiment introduced an entirely novel verb into the training corpus, varying both how often it appeared and whether it showed up in high- or low-affectedness contexts. Both factors mattered, and they operated independently of each other.

The findings speak to a long-running debate in linguistics about how much of grammar is innate versus learned from input. If a neural network without explicit grammar rules can pick up these restrictions from the same amount of text a child hears, the case for innateness on this particular point weakens.

“There’s a broad question about how people acquire negative exceptions,” Leong said. “You might have an intuition that some of this is innate, because there’s no explicit evidence in the language. But these exceptions vary across languages — why does English have this weird constraint in the passive while other languages don’t? It might seem to be a mystery where this knowledge comes from. This paper starts looking at where.” Methodologically, the paper also offers a way forward for studying language acquisition questions that can’t ethically or practically be tested on children — by running the experiment on a model instead.

By Stephen Thomas