Speakers
Description
Human brain atlases play a crucial role in providing a spatial framework to organize information derived from diverse research on brains, integrating multimodal and multiresolution images. The preprocessing stage is crucial for enhancing the quality of the images, addressing issues related to degradation, and ensuring that subsequent analyses and interpretations are based on reliable and accurate data. By employing sophisticated preprocessing techniques, researchers can mitigate the impact of unknown degradations in the scanned BigBrain Images, resulting in improved image quality and, consequently, more robust findings in neuroscientific studies.
We present a pipeline that depends on two models to effectively address actual image restoration and alignment when the relationship between high resolution and low resolution images is unknown.
We propose a blind super-resolution model to address the resolution upscaling scenario when the function for mapping high- and low-resolution images is unknown. Our solution relies on three training modules with different learning objectives: 1. a degradation-aware network (U-Net)[1,4] to synthesize the high resolution image, given a low resolution image and the corresponding blur kernel; 2. a pre-trained generative adversarial network (GAN) to be used as prior, bridged to the U-Net by a latent code mapping and several channel-split spatial feature transforms (CS-SFTs); and 3. a rational polynomial image interpolation[2] into deep convolutional neural networks (CNNs) to retain details.
This pipeline considers the generic problem of dense alignment between two images, whether they be two frames of a video, two widely different views of a scene, two paintings depicting similar content, etc. Whereas each such task is typically addressed with a domain-specific solution, Near-Duplicates interpolation or alignment is an interesting new application, but large motion challenges existing methods. To address this issue, we adopt a feature extractor that shares weights across the scales and optimizes our network with the Gram matrix loss that measures the correlation difference between features. Then the fine alignment is learned unsupervised by a deep network that optimizes a standard structural similarity metric (SSIM) between the two images. The results on BigBrain images show the performance of the proposed approach.
References
1. Amunts K, Lepage C, Borgeat L, Mohlberg H, Dickscheid T, Rousseau MÉ, Bludau S, Bazin PL, Lewis LB, Oros-Peusquens AM, Shah NJ, Lippert T, Zilles K, Evans AC. BigBrain: an ultrahigh-resolution 3D human brain model. Science. 2013 Jun 21;340(6139):1472-5. doi: 10.1126/science.1235381.
2. Wang, X., Li, Y., Zhang, H., & Shan, Y. (2021). Towards real-world blind face restoration with generativefacial prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.9168-9178).
3. Bian, S., Xu, X., Jiang, W., Shi, Y., & Sato, T. (2020, October). BUNET: blind medical image segmentation based on secure UNET. In International Conference on Medical Image Computing and Computer-AssistedIntervention (pp. 612-622). Springer, Cham.
