Mapping the brain has come a long way. Far from phrenology and physiognomy, modern medical imaging has enabled things like mapping the connectome of living, healthy human brains and parsing out images from dolphin vocalizations. Now, using voxel-based mapping, a team of UC Berkeley researchers have pinned down the brain regions where we handle certain topics and ideas: a 3D semantic atlas of the brain. Da Vinci would be proud.
Subjects in the experiments behind this semantic map had to sit in an fMRI for hours at a time, listening to The Moth Radio Hour, a storytelling podcast. Later, the researchers would painstakingly match up timestamped fMRI images with timestamped transcripts of the audio stream. This accomplished two goals. First, instead of using single words in isolation to study how the brain navigates language, the study used natural language to create a broader map of semantic concepts. Second, the timestamping let the team model brain activity as a function of what word was heard, creating a predictive map.
In the team’s own public-facing writeup of their work, they said, “This challenges the current dogma (inherited from studies of language production, as opposed to language comprehension as studied here) holding that language involves only the left hemisphere.” They developed their own new method of analysis, called PrAGMATiC because it is a Probabilistic And Generative Map of Areas Tiling the Cortex, and used it to create the primary semantic map below.
At its heart, language is a system of symbols for conveying information. We rely on shared understanding of those symbols — phonemes and words — to bridge the explanatory gap. And human brains are built in close conformance with a genetically dictated plan, most of which is common to everyone. It seems appropriate, then, that different people might process the same words in similar areas of the brain. Consistent with that idea, lead author Alex Huth said, “The similarity in semantic topography across different subjects is really surprising.” The team found that people process the same kind of words in the same brain regions — and that about a third of the cortex is used for language processing.
As for how this advances science, medicine or culture — imagine your spouse could wear an actual mood ring. Imagine eliminating the spectre of locked-in syndrome, by enabling doctors to compare brain activity to a semantic map and find the person within. Imagine a real-time translator that could let you talk with anyone in the world, no matter what languages you both speak.
Personally, I want to see this study repeated but with a much larger set of subjects, and they should be subjects that come from materially different linguistic environments. With what we know about how humans process language on a cognitive level, we could compare semantic mapping between high-context and low-context languages, or cultures that strongly value individuality versus cultures that strongly value belonging. With high-fidelity imaging and careful experimental design, neuroscientists could even prod at the divide between linguistic universalism and relativism. (For more on neurolinguistics, read this interesting PDF put forth by another department at Berkeley.) The authors are eager “to map other aspects of language, such [as] phonemes, syntax and so on. […] To explore these issues we therefore plan further studies using different stories, different modalities and different languages.”
While the paper is currently paywalled at Nature, the authors are quite happy to give access to the full text including supplemental figures. Instructions on how to get it are on their site.