This post is also available in Dutch .
Prophecy of phrases
Your brain is a lazy know-it-all, or perhaps better put: very efficient. This pubescent trait is most apparent when we process sentences. When processing language, the brain continuously predicts what word could follow the current context and thinks “hah! I already told you so!” in case it gets it right. Correctly predicted words are processed less thoroughly than ones predicted incorrectly. This is because we merely use it for testing our word-hypothesis, and less for deeper analysis of its meaning. This saves us a lot of interpretational labour: you get a quicker grasp at what words are meant to convey, as you already understand their context. You might recognise differences in such prior understanding from moments when you still have to get “into your book” or need to get used to the way in which your new colleague formulates their sentences.
Linguistic shortcuts
Considering the function of language, this is not all that surprising. Its essence is conveying information—getting a message across—like a bicycle’s essence is getting people from point A to B. In general, we’re merely interested in the endpoint of such trajectories; the final information conveyed, or the destination you’re biking towards. If there exists a shortcut to reaching your endpoint, we like taking it. Your brain prefers to make as little effort as possible to achieve its goals and thus constantly seeks simpler routes. It automatically picks up all kinds of patterns from incoming information and gets better and better at predicting what’s to come.
Informative eyes
Cognitive neuroscientists discovered this for language processing during reading by analysing the eye movements of readers. They found that the predictability of a word correlates with how long you look at it. For instance, you look much shorter at the word “teeth” in the sentence: “She promised the dentist to brush her …” than in the sentence: “She gifted her neighbour a new set of … “.
Implicit surprises
Such predictions are made unconsciously and do not involve any thinking. They are fundamental to the way we process information. Your brain is inherently curious and attaches value to faulty predictions, because they show what’s left to learn. As such, unpredictable words are processed more thoroughly than predictable words. This is not only reflected in the eyes, but also in the brain, such as in the so-called N400-response: an electrical signal in the cortex that’s stronger when evoked by more unpredictable words.
Wisdom from word prediction
Curiously, we’re not the only ones to foretell words. Artificial neural networks named “Large Language Models” or LLMs, such as the famous chatterbox ChatGPT, are trained specifically for next-word-prediction. They get their knowledge by learning to predict what the most probable next word would be, given the preceding context. For this, they process colossal amounts of text and store knowledge in the billions of connections their network architecture has. The way we learn language is different; we need much less text and store knowledge more efficiently. Nevertheless, these artificial models process language in ways strikingly similar to what we do. In fact, the language “knowledge” of such LLMs is shown to be very predictive of how brains respond to words.
Logic of language structure
Wait a minute! Does that mean ChatGPT can read my thoughts? Don’t worry, it cannot. The fact that LLMs are able to predict some of our brain activity during language processing is actually quite reasonable. This is because we also process language using models of it, woven into our own neural networks. Just like LLMs, they train on heaps of text, from which they pick up a language’s structure and words’ logical relations. Because of this, LLMs “understand” that sentences such as “Orc, orc, orc, soup is eaten with a …” confuse the brain. Just like us, they’ve learned that the words soup and spoon often co-occur, as well as a strong poetic link between orcs and forks—and not spoons. We just learn such connections much more efficiently; a GPT needs billions of words to figure it out. Nevertheless, they make great tools for investigating language processing in the brain, as they allow us to compute how surprising a word is at various levels, such as its meaning, syntax, or rhythm.
Author: Wieger Scheurer
Buddy: Natalie Nielsen
Translator: Wieger Scheurer
Translation edits: Helena Olraun / Xuanwei Li
Header image by Markus Winkler via pexels.com
About The Author
