Researchers at Brown University tested whether modern language models capture real-world knowledge. Their work was presented at the International Conference on Learning Representations in Rio de Janeiro, Brazil. Michael Lepori, the PhD candidate who led the project, says the study found "some evidence that language models have encoded something like the causal constraints of the real world," and that these encodings predict human judgements.
To investigate, the team showed sentences that varied in plausibility, from everyday events to impossible or nonsensical lines (for example, cooling a drink with ice, snow, fire or "yesterday"). They inspected the models' internal mathematical states using mechanistic interpretability, an approach that tries to reverse-engineer what a model stores.
The experiments used multiple open-source models and found that sufficiently large models developed internal vectors corresponding to plausibility categories. These vectors matched human uncertainty on ambiguous statements and emerged in larger models.
Difficult words
- researcher — person who studies topics and does scientific workResearchers
- encoding — internal representation a model stores of informationencodings
- causal — showing that one thing causes another
- constraint — a limit or rule that controls behaviorconstraints
- plausibility — how likely something is to be true
- mechanistic interpretability — method to study what a model stores
- reverse-engineer — work backwards to find how something works
- vector — a list of numbers used inside a modelvectors
- ambiguous — not clear, having more than one possible meaning
Tip: hover, focus or tap highlighted words in the article to see quick definitions while you read or listen.
Discussion questions
- Do you think it is important that language models encode real-world causal constraints? Why or why not?
- Write one sentence that is plausible and one sentence that is impossible. Explain why one is plausible and the other is not.
- Would you trust answers from larger models more than from smaller models, based on this article? Explain your reason.
