Uri Hasson, an assistant professor in Princeton's Department of Psychology and the Princeton Neuroscience Institute, is studying the underlying neural mechanisms of both the processes that allow the brain to integrate information over time and those that facilitate communication between people. To gain these insights, he designs complex, nontraditional experiments that mimic real-life situations — for example, studying the brains of people watching a Charlie Chaplin movie in real time.
Photo by Denise Applewhite
Princeton University neuroscientist Uri Hasson strives to make research conditions in his lab as true to real life as possible, using uncommon subject matter — including slapstick comedy and high-school melodrama — in his studies.
Hasson, an assistant professor in the Department of Psychology and the Princeton Neuroscience Institute, is exploring the underlying neural mechanisms of both the processes that allow the brain to integrate information over time and those that facilitate communication between people.
People's daily lives involve immeasurable amounts of sophisticated neural activity as individuals make decisions, interact with others and complete routine tasks. Most researchers who study human cognition, however, eliminate much of that complexity when designing lab experiments in order to control as many variables as possible.
Hasson's distinctive approach is to embrace such complexity by designing experiments that mimic real-life situations within the lab. His experiments are conducted under natural conditions, over longer lengths of time, as subjects engage in complex tasks. He can learn from how a person's brain reacts to a Charlie Chaplin film and how people tell and listen to tales of love and woe from a high school prom, just as others might use more traditional forms of research.
"We're starting from the complexity and messiness of real life and slowly trying to strip away some of the dimensions while making it more simple; most researchers are starting from the other side, going from controlled situations to more complex ones. The hope is that we meet at the middle," Hasson said. "If you start to work with natural stimuli, and you do not simply replicate what other people have done in a controlled setting, it really brings new questions to the table."
Thinking about time
All of Hasson's research involves people whose brains are scanned using magnetic resonance imaging (MRI) machines typically used to produce detailed images of organs and tissue. Using a process called functional magnetic resonance imaging (fMRI), the machine produces images showing the blood flow to various parts of the brain — indicating increased activity there — as the study participants perform specific tasks.
In keeping with Hasson's interest in real-life conditions, he is studying how the brain accumulates and integrates information over long periods of time — an example of a question that is difficult to examine within the confines of typical controlled experimental settings. Many experiments involving cognitive processing last for short intervals, with researchers often showing subjects visual or auditory stimuli for 1,000 milliseconds or less to examine what parts of the brain respond to the stimulus, rather than allowing for more time in which the mind can wander.
Using functional magnetic resonance imaging data, Hasson has mapped the brain areas that integrate information over short, mid-length and long time scales. Sensory areas, such as the primary auditory or visual cortex (red), have relatively a short time scale for processing (up to hundreds of milliseconds), while high-level perceptual areas (blue) accumulate information over many minutes. (Image courtesy of Uri Hasson)
Hasson has designed multiple experiments involving activities such as watching movies or listening to stories that allow him to study how the brain processes complex information over periods as long as 90 minutes. The studies, which have involved reordering segments of those narratives, have yielded results.
"The seminal finding is that the brain uses similar strategies for integrating information across space and throughout time," Hasson said. In other words, research had already shown that the basic organizing principle of the visual system is that neurons in higher-level areas receive input from low-level neurons with smaller receptive fields and integrate that information over space. Hasson's research suggests that the brain integrates information over time in the same way. Neurons in early areas have a very short window for processing information, whereas higher-level areas receive and integrate information from lower-level sensory areas over many minutes, he said.
In an exercise to test the brain's processing of films over time, Hasson presented study participants with two Chaplin films, in their original form and in three edited versions in which segments of four, 12 and 36 seconds, respectively, were placed out of order. While the audio and visual sensory areas of the brain had a high correlation for all viewings, the correlation in areas responsible for higher-level processing increased among the participants as the segments got longer.
These types of results could help researchers understand the time scale of processing for each brain area, Hasson said. The amount of time required for tasks such as interpreting the characters' motives, understanding the plot and predicting action rely on the ability to integrate information over many minutes, and such capacity may be one of the features that dissociates human brains from other species that live more in the moment.
In work supported by the National Institutes of Health, Hasson has also conducted experiments with audio-only stories — remixing the story at the word, sentence and paragraph level, and playing it backward. The results were similar to his experiments with movies and were published in The Journal of Neuroscience earlier this year.
As the field of cognitive neuroscience is still relatively new and uncharted, these findings help "fill in the gaps" of how researchers understand comprehension, Hasson said.
"Understanding the brain is a communal effort," he said.
Hasson and his colleagues used the brain responses of a person producing speech to predict the brain responses of the person listening to a recording of the speech. The analysis revealed that during successful communication the listener’s brain responses become similar to the speaker’s brain responses. This implies that people understand each other by mirroring each other’s brain responses. (Image courtesy of Uri Hasson)
Brain-to-brain communication
Hasson's other line of research focuses on social interaction, another process that is difficult to examine under controlled laboratory settings.
"Cognitive neuroscience experiments typically isolate human or animal subjects from their natural environment by placing them in a sealed quiet room where interactions occur solely with a computer screen," Hasson said. "In everyday life, however, we spend most of our time interacting with other individuals. … There is no way to understand the brain without understanding how it interacts with other brains."
Hasson added, "Think about language, for example — I did not invent English, but rather learned to speak by interacting with other native English speakers."
Interpersonal interactions shape how people think and act in the world by modifying the brain responses of others, Hasson said.
He has demonstrated this concept with a novel two-step experiment involving speaking and listening. In the first step, one of the graduate students in Hasson's lab, Lauren Silbert, told an unrehearsed, 15-minute story about one of her high school proms — a disastrous experience involving two suitors, a fistfight and a car accident — while undergoing fMRI. Then 12 other study participants were scanned via fMRI while listening to a recording of the story.
The results showed that not only did all of the listeners show similar brain activity during the story, the speaker and the listeners had very similar brain activity despite the fact that one person was producing language and the others were comprehending it. This brain coupling, or "mind meld," attracted media attention in outlets ranging from the website of the technology magazine Wired and public radio's Radiolab program to the pop culture blog Jezebel when it was published in the Proceedings of the National Academy of Sciences journal in 2010.
Hasson collaborates with researchers around the University, such as Christopher Honey (left), a postdoctoral research associate in Hasson's lab, and Greg Stephens (right), an associate research scholar in Princeton's Lewis-Sigler Institute for Integrative Genomics, to find novel approaches to studying cognition. (Photo by Denise Applewhite)
"You see a lot of overlap between the production and comprehension systems, to the extent that one can think that maybe the brain has one system, not two, that knows how to produce and understand language," Hasson said.
The 15-minute scans generated a mountain of data, which Hasson received help interpreting from physicist Greg Stephens, an associate research scholar in Princeton's Lewis-Sigler Institute for Integrative Genomics who has collaborated with Hasson on several studies. Stephens looked for the structure or patterns across individuals, using the speaker's brain activity to create a model to decode and predict the listeners' brain activity.
"Speaking and listening are different brain functions," Stephens said. "But one of the ways the brain might process a complex auditory signal such as language is to produce its own version of the signal and compare against what it hears. Indeed, our observations of broadly shared activity between the speaker and the listener provide evidence for such dual processing. This is exciting and there are many more questions raised by what we found."
On average, the listener's brain responses mirrored the speaker's brain responses with some time delays. The delays matched the flow of information, implying a causal relationship in which the speaker's words shaped the responses in the listener's brain.
In addition, the analysis also identified a subset of brain regions in which the activity in the listener's brain preceded the activity in the speaker's brain, suggesting that the listeners were actively predicting the speaker's upcoming utterances. And the ability to predict speech may also be tied to how well people understand each other, Hasson said.
"The stronger the coupling between the speaker and the listener's brain responses, the better the understanding," he said. "Sometimes when you speak with someone, you get the feeling that you cannot get through to them, and other times you know that you click. When you really understand each other, your brains become more similar in responses over time."
Foundations of studying the mind
Hasson began his studies of the mind as a philosophy major at Hebrew University of Jerusalem. When he became impatient with discussions about ideal worlds and hypothetical situations, he added a major in cognitive science so that he could study the real-world workings of the brain. He went on to earn a master's degree in cognitive science at Hebrew University and a doctorate in neurobiology at the Weizmann Institute of Science in Israel.
Hasson served as a postdoctoral fellow at the Center for Neural Science at New York University before arriving at Princeton in 2008.
Today, Hasson's colleagues and students recognize him as a leader in the burgeoning field of cognitive neuroscience — while also commending him for his dedication as a teacher and mentor — both for how he approaches research questions and for his results.
"Compared to the kinds of stimuli that most cognitive neuroscientists study, natural events are impossibly complex. Uri is all about embracing and taming that complexity," said Deborah Prentice, chair of the psychology department and the Alexander Stewart 1886 Professor of Psychology at Princeton. "He has become famous, even at this early stage in his career, for developing a method to study brain activity during free viewing of films and listening to stories. Nobody else is doing this kind of work."
Hasson continues to be involved in several projects involving the ways brains relate to each other. Now that Princeton has two MRI machines on campus, Hasson's lab is designing experiments involving simultaneous scanning of speakers and listeners in conversation.
Alexander Todorov, an associate professor of psychology and public affairs whose research interests overlap with Hasson's, said that his colleague is well known for his unique methods to study the large-scale organization of the brain.
"He applied his innovative methods to a topic that has been difficult to study in a science focused on the workings of individual brains," Todorov said. "After all, communication requires at least two minds. Uri thought outside of the box. He devised exquisite methods and paradigms to explore the dynamics of interacting brains. The results are extremely interesting and open new venues for research."
In May, Hasson co-organized a workshop at Princeton on the mechanisms of social interaction that drew American, European and Asian researchers.
"This workshop was Uri's brainchild," said Todorov, who was also a co-organizer. "He had very clear ideas of how to bring together people doing exciting research in a new and unexplored area. And the workshop was a great success."
Bringing research into teaching
Hasson is helping to train the next generation of neuroscientists by bringing his broad experience to the classroom. This fall, he is teaching "Brain Imaging in Cognitive Neuroscience Research," a course he has taught several times that trains undergraduates and graduate students in fMRI. He covers the basics of brain physiology, experimental design, and imaging physics and analysis, so that students understand the foundations of fMRI before embarking upon their own research.
Alana D'Alfonso, a member of the Class of 2011 whose senior thesis was advised by Hasson, took the imaging course in her junior year, and said it greatly helped her with thesis research. Her project examined whether the brain's visual cortex in the congenitally blind develops new functions such as complex linguistic processing.
"Uri always reminded me that the research I was doing was both interesting and worthwhile, and that one has to expect ups and downs," she said. "With Uri, there is no 'cannot' or 'impossible.' Instead, Uri focuses on finding solutions."
When D'Alfonso had trouble finding enough subjects for her study, Hasson helped her collaborate with a lab at Georgetown University. Hasson continues to mentor D'Alfonso, who is now a research fellow at National Institute of Neurological Disorders and Stroke.
Said D'Alfonso, "Having professors who go above and beyond for their students and inspire them to potentially change their life plans is what makes Princeton great. Uri is one of those professors."
