Description
Spatial transcriptomics enables the study of gene expression patterns within the brain, providing critical insights into cellular organization and function. A challenge in this field is mapping spatial transcriptomic data onto existing 3D brain atlases. This mapping would facilitate multimodal atlas creation, enable brain area label transfer from the atlas to the spatial transcriptomics sample, and unify brain area annotation across different experiments. Existing approaches primarily rely on image registration, which performs well for 2D-to-2D or 3D-to-3D alignment. However, spatial transcriptomics often requires 2D-to-3D mapping, where a single transcriptomics slide must be integrated into a full-brain 3D atlas. To address these limitations, we propose an alternative deep learning-based, feature-driven approach to map transcriptomics data to 3D atlases.
We use public mouse brain data from the Allen Institute, including a histological atlas [1], and two MERFISH datasets [4, 5], all registered to a common coordinate system that provides ground truth for our experiments. Our strategy for integrating spatial transcriptomics data into histological atlases begins by learning unimodal embeddings for small patches. Histology embeddings are extracted using a pretrained foundation model [2]. For spatial transcriptomics, we first generate cell embeddings by applying PCA to the gene expression profiles. As a baseline, we obtain patch-level embeddings by averaging the PCA embeddings of all cells within a patch. In contrast, for the point cloud transformer [6], we represent each patch as a 3D point cloud, with spatial coordinates as point locations and PCA-derived cell embeddings as point features. The transformer is trained on multiple tasks, including brain area classification and coordinate regression at the patch level, as well as brain area segmentation at the cell level. After generating unimodal embeddings, we align transcriptomics with histology embeddings using optimal transport (OT) [3]. We compare standard OT with a version incorporating coarse brain area as additional information. Mapping quality is assessed via mixing scores and coordinate transfer accuracy from histology to transcriptomics.
Evaluation of unimodal spatial transcriptomics embeddings shows that both PCA and the transformer perform similarly in coarse brain area classification. For more fine-grained tasks, the transformer outperforms PCA, achieving an $R^2$ of 0.94 vs. 0.71 for coordinate prediction at patch-level, and a balanced accuracy of 23% vs. 13% in segmentation of more than 300 brain areas at cell level. For multimodal mapping, OT achieves good modality mixing (silhouette score 0.052) but shows poor performance in coordinate transfer from histology to transcriptomics ($R^2$ of −0.16). Incorporating brain area information slightly improves mixing (silhouette score of 0.047) and substantially enhances coordinate prediction ($R^2$ of 0.13), highlighting the need for anatomical supervision. Future work will focus on refining embeddings, improving alignment, and comparing our method to classical registration methods.
References
[1] Allen Institute for Brain Science. Allen Reference Atlas – Mouse Brain, 2004. Available from http://atlas.brain-map.org.
[2] R. Chen et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 2024.
[3] D. Klein et al. Mapping cells through time and space with moscot. Nature, 2025.
[4] Z. Yao et al. A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain. Nature, 2023.
[5] M. Zhang et al. Molecularly defined and spatially resolved cell atlas of the whole mouse brain. Nature, 2023.
[6] H. Zhao et al. Point transformer. arXiv preprint, 2021.
