Topo-Omni — Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

Abstract

Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topographic models reproduce aspects of this structure but remain unimodal and spatially constrain each layer separately, yielding fragmented maps that capture neither the contiguity of cortical processing streams nor their integration across modalities. We introduce Topo-Omni, a topographic multimodal model in which visual, auditory, and language/cognitive processing share a single contiguous in-silico sheet. Built by fine-tuning a pretrained foundation model with a spatial smoothness objective, this architecture develops clusters across modalities that are consistent with human neuroimaging, from sensory to cognitive systems. Driving or suppressing a cluster selectively biases or impairs perception, paralleling human intervention studies. Finally, we use our model to screen for novel clusters in-silico and discover new natural landscape and animal networks which we validate in human data. A single spatial principle thus organizes representations across modalities and processing stages, yielding testable hypotheses about cortical organization.

Main Findings

Topo-Omni builds on Qwen2.5-Omni and learns a bidirectional projection of its units onto a single two-dimensional cortical sheet, optimized jointly with a task loss and a spatial-smoothness loss. Across every modality the model was tested on, this single objective recovers spatial organization that parallels the human brain — from category-selective patches to causal involvement in perception.

Overview of Topo-Omni: a multimodal architecture is projected onto a single contiguous in-silico cortical sheet, yielding category-selective clusters for vision, audio, and language/cognition that match human fMRI response profiles, plus a model-guided discovery pipeline validated on human data.

A single sheet for vision, audition & cognition

Topo-Omni projects the vision encoder, audio encoder, and language/cognitive module of Qwen2.5-Omni onto one shared in-silico cortical sheet. A spatial-smoothness loss, applied jointly with the model's self-distillation task loss during fine-tuning, encourages nearby units to develop similar response profiles — with no neural data or category labels supplied during training. The result is brain-like clustering that spans sensory and cognitive systems within a single contiguous map, from category-selective patches (b) to model-guided discovery of new networks validated in human fMRI (c).

Topo-Omni develops category-selective regions for faces, scenes, objects, and word forms in its vision encoder, alongside retinotopic maps of polar angle and eccentricity, each compared against human fMRI selectivity maps and response profiles from Marvi et al. 2025.

Visual category selectivity & retinotopy emerge unsupervised

Applying the EMFL functional localizers from Marvi et al. (2025) to the vision encoder recovers focal clusters selective for faces, scenes, objects, and visual word forms — paralleling the FFA, PPA, LOC, and VWFA of the human ventral stream. Response profiles correlate with human fMRI across ten stimulus categories (Pearson r = 0.61–0.89, mean r = 0.75), reaching significance for the face- and object-selective regions. Without any explicit supervision, the same encoder also develops smooth, continuous maps of polar angle and eccentricity, echoing the retinotopic organization of early visual cortex.

Topo-Omni's audio encoder develops speech-selective and voice-selective regions matching human superior temporal gyrus and temporal voice areas, plus a tonotopic map of preferred frequency, compared against human fMRI from Marvi et al. 2025, Pernet et al. 2015, and Hedger et al. 2026.

Speech, voice & tonotopic maps in the audio encoder

A speech-selective region emerges whose response profile mirrors the human superior temporal gyrus (r = 0.69): it responds broadly to intelligible speech but drops for quilted speech, which destroys linguistic structure while preserving acoustic energy. A separate voice-selective region responds preferentially to human vocalizations over non-vocal sounds, paralleling the temporal voice areas along the superior temporal sulcus. The audio encoder also develops a tonotopic map, where neighboring units share similar preferred frequencies — mirroring the tonotopic organization of human auditory cortex.

Topo-Omni's language/cognitive module develops spatially distinct language, multiple-demand, and theory-of-mind networks that parallel human fMRI activations from classical localizer contrasts (Marvi et al. 2025).

Language, multiple-demand & theory-of-mind networks

Passing linguistic stimuli directly as text tokens to the language/cognitive module reveals three spatially distinct networks, each isolated by a classical localizer contrast: a language network responding to meaningful sentences over non-words (d′ = 1.39), a multiple-demand network responding to arithmetic problems over narrative reasoning (d′ = 0.54), and a theory-of-mind network responding to false-belief over false-photograph questions (d′ = 0.15). These parallel the fronto-temporal language network, the frontoparietal multiple-demand network, and the temporo-parietal / medial-prefrontal theory-of-mind network in human cortex — all emerging from the same spatial-smoothness objective, with no brain data or labels supplied during training.

Topography preserves brain alignment & task performance

Imposing spatial structure could in principle come at a cost. We compared Topo-Omni against a non-topographic SFT-Omni baseline (same backbone, task loss only) and the original Qwen2.5-Omni-3B on the Natural Scenes Dataset (NSD) and on OmniBench. Across twelve ventral-stream ROIs, Topo-Omni's brain predictivity is statistically indistinguishable from both baselines in 11 of 12 regions, and it achieves the best overall OmniBench accuracy and the best Sound Event subtask score. The spatial constraint that produces brain-like topography therefore comes at no measurable cost to neural predictivity or downstream multimodal capability.

Benchmark	ROI / Subtask	Topo-Omni	SFT-Omni	Qwen2.5-3B
NSD Brain-Score (Pearson r, noise-corrected)	FFA-1 (faces)	0.742	0.740	0.738
	PPA (scenes)	0.821	0.820	0.821
	VWFA-1 (words)	0.689	0.689	0.689
	EBA (bodies)	0.740	0.739	0.739
OmniBench (accuracy, %)	Overall	43.78	43.70	43.35
OmniBench (accuracy, %)	Sound Event	40.75	39.25	40.38

Selected rows from Table 1 — see the paper for the full twelve-ROI comparison and significance tests.

Driving face-selective units in Topo-Omni increases perceived face frequency toward ceiling; suppressing the top 10 percent of face-selective units collapses face-identification accuracy while leaving body, scene, and object recognition intact; suppressing other category regions leaves face accuracy largely unaffected.

Face-selective units are necessary & sufficient for face perception

Because Topo-Omni's clusters are spatially compact, they can be directly driven or suppressed — like TMS or intracranial stimulation in neuroscience. Driving units in the face-selective region biases the model toward reporting "face" for almost any input, reaching near-ceiling perception at just 15% coverage of the region (a). Conversely, suppressing the top 10% of face-selective units collapses face-identification accuracy to near zero while recognition of bodies, scenes, and objects remains largely intact (b) — and suppressing those other regions instead leaves face recognition essentially unaffected (c). Category-selective clusters in Topo-Omni are therefore not epiphenomenal: they are causally implicated in category-level perception.

Model-guided clustering of video segments identifies an animals-selective network and a natural-landscapes-selective network in Topo-Omni's cortical sheet; both predictions are validated by right-lateralized prefrontal activation in human fMRI from the Spacetop dataset.

Model-guided discovery of novel cortical networks

Beyond recovering known regions, Topo-Omni can propose new hypotheses about cortical organization. Clustering naturalistic video segments by their selectivity profiles on the in-silico sheet revealed two networks with no clear precedent in the literature: one selective for animals (snakes, birds, primates) (a) and one for natural landscapes (beaches, rocky terrain, alpine peaks) (b). Testing these model-derived video contrasts on independent human fMRI data (Spacetop) reveals right-lateralized clustering in prefrontal cortex for both categories — turning an in-silico observation into a validated, testable prediction about human cortex.

Citation

If you find this work useful, please cite the paper:

@article{alkhamissi2026topoomni,
  title={Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model},
  author={AlKhamissi, Badr and Mehrer, Johannes and Marinov, Lara and Abdelaal, Ahmed and Gokce, Abdulkadir and Schrimpf, Martin},
  journal={arXiv preprint arXiv:2606.09770},
  year={2026}
}