AI is altering scientists' understanding of language learning, raising concerns about innate grammar
The language of everyday interaction, in contrast to the carefully scripted dialogue found in most books and movies, is messy and incomplete, full of false starts, interruptions, and people talking over each other. Authentic conversation is chaotic, from casual conversations between friends to sibling bickering to formal discussions in a boardroom. Given how random language learning is, it seems like a miracle that anyone can learn a language at all.
People think that children must have a grammar template hardwired into their brains to help them get past the limits of their language experience.
This template, for example, could include a "super-rule" that governs how new phrases are added to existing ones. The only thing left for children to learn is whether their native language is one where the verb comes before the object (as in "I eat sushi") or one where the verb comes after the object (as in "I eat sushi" in Japanese).
However, novel insights into language learning are emerging from an unexpected source: artificial intelligence. After being exposed to massive amounts of language input, a new breed of large AI language models can write newspaper articles, poetry, and computer code, as well as answer questions truthfully. Even more amazing, they all do it without the use of grammar.
Grammatical language with no grammar
Even if their word choice is occasionally strange or nonsensical or contains racist, sexist, or other harmful biases, one thing is certain: the vast majority of the output of these AI language models is grammatically correct. Even so, they don't have any grammar templates or rules hardwired into them. Instead, they only learn grammar through experience, no matter how messy that may be.
The most well-known of these models, GPT-3, is a massive deep-learning neural network with 175 billion parameters. It was trained on hundreds of billions of words from the internet, books, and Wikipedia to predict the next word in a sentence given what came before. When it made a wrong guess, it used an automatic learning algorithm to change its settings.
Surprisingly, GPT-3 can generate believable text in response to prompts like "A summary of the most recent 'Fast and Furious' film is..." or "Write a poem in the style of Emily Dickinson." GPT-3 can also answer SAT-level analogies, questions about reading comprehension, and even simple math problems by learning to guess the next word.
The resemblance to human language does not end there. According to research published in Nature Neuroscience, these artificial deep-learning networks appear to use the same computational principles as the human brain. The researchers, led by neuroscientist Uri Hasson, first compared how well GPT-2-a "little brother" of GPT-3-and humans predicted the next word in a story taken from the podcast "This American Life": people and the AI predicted the same word nearly 50% of the time.
While listening to the story, the researchers recorded the volunteers' brain activity. The best explanation for the patterns of activation they observed was that people's brains, like GPT-2, relied on the context of up to 100 previous words when making predictions rather than just the preceding one or two words. "The fact that we found spontaneously predictive neural signals while people listened to natural speech suggests that active prediction may be the key to how people learn languages throughout their lives," the authors write in their conclusion.
One possible source of concern is that these new AI language models are fed a large amount of data: GPT-3 was trained on linguistic experience equivalent to 20,000 human years. However, a preliminary study that has not yet been peer-reviewed discovered that GPT-2 can still model human next-word predictions and brain activations even after only 100 million words have been trained. That is well within the range of how much language a typical child hears in his or her first ten years.
We are not claiming that GPT-3 or GPT-2 learn language in the same way that children do. Indeed, these AI models do not appear to comprehend much, if anything, of what they are saying, despite the fact that comprehension is essential to human language use. Nonetheless, these models demonstrate that a learner—albeit a silicon one—can learn language well enough from simple exposure to produce perfectly good grammatical sentences in a manner resembling human brain processing.
Many linguists have long believed that learning a language without a built-in grammar template is impossible. The new AI models demonstrate otherwise. They show that the ability to produce grammatical language can be learned solely through linguistic experience. Similarly, we contend that children do not require innate grammar to learn language.
The old adage goes, "Children should be seen, not heard," but the latest AI language models suggest that nothing could be further from the truth. Instead, children should be immersed in conversation as much as possible to help them develop their language skills. Linguistic experience, not grammar, is essential for becoming a proficient language user.