Speakers
Description
Introduction:
Among their many applications,interpolation methods have proven particularly valuable in neuroimaging for frame rate enhancement, detailed reconstruction of 3D datasets, and replacing missing data. The latter problem becomes critical when dealing with ultra-high (cellular) resolution brain images. One such application is with the BigBrain, a 3D reconstruction of 7404 histological brain sections at 20-micrometer isotropic resolution [1]. The original sections of the BigBrain were re-scanned at 1 micron in-plane, and images were captured through selected slices at different depths with an optical microscopy technique. To deal with the large size of data, slices were divided into patches which were aligned and stacked to form 6x6x6mm^3 volumes at 1-micron isotropic resolution.
In this work, we present a novel, deep learning-based method for near-duplicate image synthesis[2,3,4] with bi-directional Flows of Feature Pyramid (FFP)[4] and Adaptive Feature Learning (AFL)[2] algorithm designed to replace missing data and create seamless and smooth 3D blocks of 1-micron isotropic BigBrain.
Methods:
Two 6x6x6mm^3 blocks of the BigBrain were downloaded from EBRAINS (https://ebrains.eu/): one at 2-micron (25 GB), and the other at 8-micron isotropic as a proof of concept. All codes were run on two NVIDIA GeForce GTX 1080 GPUs, CUDA Version 12.4.
We adopt a multi-scale feature extractor based on a feature pyramid architecture to accurately model motions of varying magnitudes, ranging from subtle movements to large-scale displacements. Built upon this, a scale-agnostic bi-directional motion estimator is employed to effectively handle both small and large movement.
To ensure visually coherent synthesis, we integrate Gram Loss, Gradient Loss, and Perceptual Loss into the optimization process. Gram Loss facilitates global texture preservation, Gradient Loss retains local edge details,and Perceptual Loss emphasizes textures and overall appearance.
Additionally, adaptive loss functions are introduced to focus on high-frequency or critical regions, providing flexibility during optimization. This adaptability improves robustness and enhances generalization and performance across diverse scenarios.
Results:
Using the motion estimator leverages the feature pyramid to align frames by predicting motion vectors at multiple scales, and corrects for any residual motion in the 3D blocks (Figure 1). The use of a combination of losses enhances the quality and consistency of the interpolated frames to yield a smooth transition between frames (Figure 2).
