Description
Tomographic imaging enables the 3D reconstruction of objects from multiple 2D projections and has found widespread applications across diverse fields, including medical imaging [1], semiconductor inspection [2], and biological imaging [3]. Although supervised deep learning has emerged as a powerful tool for image reconstruction, several fundamental challenges limit its deployment in real-world scenarios. The main issue is the need for training data from the target ground-truth samples, which is often difficult or even impossible to obtain. In addition, deep learning models typically exhibit poor generalization, a significant obstacle for clinical and scientific applications where robustness is critical. Finally, while training deep networks for 2D reconstruction tasks is relatively feasible, extending these approaches to 3D reconstruction becomes prohibitively expensive, particularly for applications such as nanoscale imaging of biological samples and integrated circuits [2,3].
To overcome these challenges, we present a local deep learning reconstruction framework based on the physics of tomographic imaging. Instead of processing all projections to recover the full 3D tomogram at once, our approach leverages the inherent locality of tomography by reconstructing each coordinate independently. We process the associated measurements for each coordinate located on a sinusoidal curve in the projection space. The proposed framework brings several key advantages, making it an ideal choice for 3D reconstruction of large volumes:
-
Strong generalization: Thanks to locality, the proposed framework has strong generalization, allowing models trained on simulated data to be successfully applied to real experimental data. This capability is crucial in applications where ground-truth images are difficult or impossible to obtain, like cryo-electron tomography (CryoET) due to intensive noise and missing-wedge information, and laminographic imaging because of the missing-cone information.
-
Memory efficiency: The coordinate-based reconstruction enables training on small mini-batches of coordinates at each iteration, significantly reducing the required memory. As a result, the framework scales naturally to the reconstruction of large 3D volumes common in nanoscale imaging [2,3].
- Performance on real experimental data: We show that the proposed framework achieves strong performance on 2D tomographic reconstruction with great success on out-of-distribution data [4]. Furthermore, we have extended the method to 3D cryo-electron tomography (CryoET) [5], attaining state-of-the-art results. Preliminary experiments also show promising results for X-ray ptycho-laminographic imaging of integrated circuits, measured at the cSAXS beamline of the Swiss Light Source (SLS) at the Paul Scherrer Institute (PSI), Switzerland.
