The state in which artificial intelligence is thinking
Kanerva’s book has faded from view, and Hofstadter’s star has waned, except when he occasionally raises his head to criticize a new AI system. In 2018, he wrote about Google Translate and similar technologies: “There is still something sorely missing in the approach, which is conveyed in a single word: to understandBut GPT-4, released in 2023, produced Hofstadter’s transformative moment. “I’m mystified by some of the things that systems do,” he told me recently. “This would have been unimaginable even just ten years ago.” And the strongest deflationists could no longer shrink. Here was a program that could translate as well as be an expert, make measurements, improvise, and generalize. Who are we to say he didn’t understand? “They’re doing things that are like… Pretty much thinking.” “You can say they are We are Thinking, just in a rather strange way.
LLM holders seem to have a “seeing as” machine at their core. It represents each word with a series of numbers that indicate its coordinates – its vector – in high-dimensional space. In GPT-4, a word vector has thousands of dimensions, which describe shades of similarity and difference from every other word. During training, a large language model adjusts word coordinates whenever it makes a prediction error; Words that appear in texts together are rounded in space. This results in an incredibly dense representation of uses and meanings, where measurement becomes a matter of geometry. In the classic example, if you take the word vector for “Paris”, subtract “France” and then add “Italy”, the next closest vector will be “Rome”. LLMs can “transform” an image by encoding its interior, its mood, and even the expressions on people’s faces, in enough detail to redraw it in a particular style or write a paragraph about it. When Max asked ChatGPT to help him use a sprinkler in the garden, the model wasn’t just spewing out text. The plumbing image, along with Max’s message, has been compressed into a vector that captures its most important features. This vector served as a title to call up nearby words and concepts. These ideas, in turn, recalled others as the model constructed a sense of the situation. She crafted her response with these thoughts “in mind.”
A few months ago, I was reading an interview with Anthropologist Trenton Brecken, who worked with colleagues to explore the insides of Claude, the company’s series of AI models. (Their research has not been peer-reviewed or published in a scientific journal.) His team has identified groups of artificial neurons, or “features,” that fire when Claude is about to say one thing or another. Features become volume knobs for concepts; Lift it up and the model talks about little else. (In a kind of thought-control experiment, the feature representing the Golden Gate Bridge came up; when one user asked Claude for a chocolate cake recipe, his suggested ingredients included “a quarter cup of dry fog” and “a cup of warm sea water.”) In the interview, Brecken mentioned Google’s Transformer architecture, a recipe for building neural networks that form the basis of leading AI models. (The “T” in ChatGPT stands for “Transformer.”) He said the mathematics at the heart of the transformer architecture closely approximates a model that was proposed decades ago — by Pentti Kanerva, in “Sparse Distributed Memory.”
Should we be surprised by the compatibility between artificial intelligence and our brains? MBAs are artificial neural networks that psychologists and neuroscientists helped develop. What was even more surprising was that when the models practiced something routine—predicting words—they began to behave in a brain-like way. These days, the fields of neuroscience and artificial intelligence are intertwined; Brain experts use artificial intelligence as a kind of model organism. Evelina Fedorenko, a neuroscientist at MIT, has used her LLM degrees to study how the brain processes language. “I never thought I would be able to think about such things in my life,” she told me. “I never thought we would have good enough models.”
It has become commonplace to say that artificial intelligence is a black box, but arguably the opposite is true: a scientist can explore and even change the activity of individual artificial neurons. “Having a working system that embodies the theory of human intelligence is the dream of cognitive neuroscience,” Kenneth Norman, a neuroscientist at Princeton University, told me. Norman has created computer models of the hippocampus, the area of the brain where episodic memories are stored, but they were once so simple that he could only feed them with crude approximations of what might enter the human mind. “Now you can give memory models the exact stimuli that you would give a person,” he said.