A modality independent proto-organization of human multisensory areas

Here, we discuss the results of our study published in Nature Human Behaviour and its more general outcomes. Our observations have implications for dissecting the “nature vs. nurture” contribution to human brain development and enhancing novel educational and rehabilitative strategies.
Published in Social Sciences
A modality independent proto-organization of human multisensory areas
Like

Share this post

Choose a social network to share with, or copy the shortened URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

In the paper entitled “A modality independent proto-organization of human multisensory areas”, we show that the superior temporal cortex and neighboring areas retain the ability to represent sensory correspondences across modalities, despite the complete lack of prior auditory or visual experience. Given this finding, we argue that the superior temporal cortex must be innately provided with a functional “scaffolding” enabling a shared neural representation of coherent auditory and visual sensory events. The relevance of this observation is that the demonstration of a preserved functional topography in the superior temporal cortex for basic audiovisual processing provides additional support to the theory that large portions of the human neocortex may possess a predetermined modality-independent architecture that sustains subsequent, experience-dependent functional specialization. In turn, this finding has important implications both for dissecting the “nature vs. nurture” contribution to human brain growth and functioning and for the development of novel educational and rehabilitative strategies in individuals who lack the sensory ability(ies).

The superior temporal cortex is acknowledged as being a pivotal hub for the processing of audiovisual information. Combining signals across different sensory modalities is essential for an efficient interaction with the external world. To do so, the brain must detect the information conveyed by different sensory inputs and couple coherent events in space and time. Although some evidence of basic multisensory computations appears to be present in newborns, the extent to which audiovisual experience is necessary for the superior temporal cortex to develop and become able to detect shared features between the two sensory streams remains unknown. The current study intended to address this issue by assessing the impact of sensory experience on the development of brain computations related to the processing of real-world events. We reasoned that adult individuals who specifically lack visual or auditory information due to congenital sensory loss represent an optimal model to determine to what extent sensory experience is a necessary prerequisite for developing brain functional properties.

In the study, we measured the similarity of brain responses between two groups of sensory-deprived (SD, with congenital lack of either sight or hearing) adults and two samples of typically-developed (TD) individuals, exposed to the auditory and/or visual versions of the same long-lasting narrative. Brain response was assessed using functional Magnetic Resonance Imaging (fMRI) while the subjects were watching and/or listening to the same, edited version of Walt Disney’s action movie “101 Dalmatians”. We adopted a methodological approach, called Intersubject Correlation (ISC) analysis, based on the concept that brain regions do “synchronize” across individuals when processing the same stimulus features. The figure below illustrates the experimental paradigm and the analytical approach. We reasoned that any evidence of synchronization within the superior temporal cortex across conditions and experimental groups would indicate that this region sustains shared representations of visual and auditory stimulus properties even when information from one of the two sensory channels is missing since birth. Additionally, we thoroughly detailed the sensory events (low-level perceptual and high-level semantic) driving the synchronization and characterized its temporal dynamics by depicting the chronotopic organization of multisensory regions.

The figure shows the experimental paradigm adopted in the study (naturalistic stimulation) and the experimental conditions (visual and auditory streams). Moreover, the panel illustrates a schematic representation of the intersubject correlation (ISC) analysis that consists in measuring the similarity across the brain activity elicited by movie watching and that evoked by listening to the corresponding auditory narrative in two different groups of participants.
Analytical approach. The figure shows the experimental paradigm adopted in the study (naturalistic stimulation) and the experimental conditions (visual and auditory streams). Moreover, the panel illustrates a schematic representation of the intersubject correlation (ISC) analysis that measures the similarity across the brain activity elicited by movie watching and that evoked by listening to the corresponding auditory narrative in two different groups of participants.

Results demonstrated that the congenital absence of visual or auditory experience does not affect the ability of the superior temporal cortex to represent unimodal auditory and visual streams of the same perceptual events. This finding indicates that the functional architecture of the superior temporal cortex, based on the extraction of common basic features from auditory and visual signals, emerges despite the lack of audiovisual inputs since birth and thus irrespectively of postnatal audiovisual sensory experiences. This observation favors the hypothesis that the human superior temporal cortex is endowed with a functional scaffolding to process basic perceptual features that define sensory correspondences across audition and vision and also carries significant implications for developing innovative educational strategies in sensory-deprived individuals.

Further reading 

  • Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu H-WC, Cohen L, Guazzelli M, Haxby JV (2004). Supramodal category-related representations of objects in the human ventral visual pathway. Proc Natl Acad Sci (USA), 101:5658-5663.
  • Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004). Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci7(11), 1190-1192.
  • Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science303(5664), 1634-1640.
  • Ricciardi, E., Bottari, D., Ptito, M., Röder, B., & Pietrini, P. (2020). The sensory-deprived brain as a unique tool to understand brain development and function. Neurosci Biobehav Revs108, 78-82.

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in