Helmholtz Imaging Virtual Conference 2021

Scientific Programme

Keynote

• From fossils to human organs, development of synchrotron phase-contrast hierachical imaging at the ESRF

  Paul Tafforeau, ESRF

  The European Synchrotron Radiation Facility (ESRF, Grenoble, France) has a long history in X-ray full-field imaging, especially in propagation-based phase-contrast imaging. Among the many research topics involving this technique, palaeontology is a noticeable one, which began at the ESRF in 2000. During the last two decades, constant efforts have been made to investigate larger and larger fossils or archaeological remains, whilst being able to image regions of interest at high-resolution with local tomography. Imaging these larger specimens, especially those as dense as fossils, requires higher energies, longer propagation distances and higher beam coherence. In these respects, we had reached the technical limit with the 3rd generation synchrotron sources. The project of the ESRF-EBS, the first high-energy 4th generation synchrotron source, is a game changer. The highest possible coherence of this new lattice is obtained on the short wiggler sources that replaced the original bending magnet sources of the previous machine. A specific project (BM18) aiming at fully exploiting this coherence up to 300 keV, began in 2017. In parallel, the existing beamline BM05 was refurbished to exploit this coherence up to 130 keV. In 2020, with the emergence of the COVID-19 pandemic, the ESRF decided to join an international research effort collaborating with Imaging scientists at University College London and Clinical scientists treating COVID-19 patients in Hannover, Mainz, Heidelberg and Grenoble, to help understand the disease. By adapting specific scanning protocols originally implemented to image fossils, it was possible to develop a new approach on BM05 to image complete human organs, including those damaged by SARS-CoV-2 infection, with a level of precision unknown so far. The technique, termed Hierarchical Phase Contrast Tomography (HiP-CT), has already brought impressive results, but will reach much higher contrast, speed, resolution and sensitivity on even larger samples once implemented on BM18. This new EBS beamline is expected to start user operations mid-2022.

Talks

• Session block 1

  Hyper 3D-AI: Artificial Intelligence for 3D multimodal point cloud classification
  Dr. Sandra Lorenz, HZDR
  Independent of the application field, spatially detailed information is commonly provided in the form of image data. Accordingly, major developments in image processing and artificial intelligence (AI) for image data interpretation are based on an image-like data structure, i.e. a spatially two-dimensional data grid with a custom number of informative layers. This approach has major flaws when applied in any oblique-angle scenario, in particular as it inherently distorts the spatial characteristics of the observed target (virtual vs. real-world neighborhood relationships, occlusions). Todays’ most crucial image data applications (e.g., resources, energy, mobility, medicine), however, heavily rely on the accurate interpretation of the spatial relationship of objects in all three dimensions. The upscaling of 2D-images to multi-feature attributed 3D point clouds is not only beneficial for the fusion of image data with 3D-information (such as orientation, shape, and surface roughness), but also offers a straightforward solution for the fusion of higher dimensional multi-sensor data. Although point clouds or meshes are routinely used as 3D analogues of real-world targets, the processing of multi-feature point clouds in terms of clustering, classification or material characterization is still in its infancy. Innovative AI approaches such as PointNets or 3D-CNN have shown great potential for point cloud clustering using the spatial relationships of the individual points. However, classifications based on both spatial and auxiliary, high-dimensional point information such as spectral signatures or compositional characteristics are yet to be developed. The HIP project Hyper 3D-AI aims at the development of advanced machine (deep) learning approaches to fill this exact gap. These approaches comprise both the challenging fusion of multiple sensors as well as the subsequent classification and segmentation. Besides the algorithm design and testing, the creation of reusable benchmark datasets for the validation and future development of algorithms are major tasks of the project. Due to the versatility in application, the project outcome could support any process that requires a multi-sensor-based discrimination of objects and materials.

  Developing multi-scale satellite-based imaging platform for change detection: application for landslide hazard classification and early warning service
  Prof. Dr. Mahdi Motagh, GFZ
  Rainfall-induced landslides are a major type of natural hazard that cause significant human and economic losses in mountainous regions worldwide. Proper assessment of hazard and risk posed by impending slope failures is fundamental to formulating proper strategies to mitigate environmental damage and related societal disruption.
  Airborne and spaceborne remote sensing play a pivotal role in driving the innovation in detecting, monitoring and assessing landslide hazards. Automated and semi-automated approaches using optical images have already been developed to create dynamic multitemporal inventories and to supplement field surveys. However, methods that use remotely sensed optical data cannot reliably support near real-time hazard assessments and early warning systems (EWS), since clear sky images may not be readily available prior to and during a given landslide event. Moreover, the displacement accuracies for landslide motion retrieved from cross-correlation analysis and feature tracking of optical images are insufficiently high enough to be used for the detection of small-scale deformation rates (in the range of mm to cm) that can signal an impending failure. Consequently, the use of optical remote sensing data is very limited with respect to providing necessary information for fast crisis response and to predict the onset of landslides.
  In the MultiSat4SLOWS (Multi-Satellite imaging for Space-based Landslide Occurrence and Warning Service) project, we aim to develop a novel hybrid multi-scale data fusion approach that incorporates information from all satellite remote sensing sources, including optical images as well as, amplitude, coherence, polarization of radar data together with polarimetric decomposition parameters. The approach is designed for near real-time change detection and assessing landslide stability and early warning indicators. The ultimate goal is to develop prototype solutions in order to provide timely information to local stakeholders and decision-makers so that the development and implementation of suitable mitigation measures can effectively minimize associated risks for susceptible populations.

  Solar Image-Based Modelling
  Prof. Dr. Yuri Shprits, GFZ & Dr. Fadil Inceoglu, GFZ
  Space weather is a term used to describe hazardous events in the near-Earth space environment that can have adverse effects on ground-based infrastructure such as power grids and telecommunications, as well as space assets and can severely affect GPS- or GNSS-based navigation. In this presentation, we will elaborate on the solar physics behind our new project Solar Image-based Modelling (SIM), as well the project itself where we aim to utilise the growing number of high-resolution multi-spectral images of the Sun to improve the prediction of the solar wind close to the Earth and develop new methods, utilizing data-driven approaches. SIM will exploit the capabilities of classical computer vision tools and modern deep learning algorithms to identify relevant structures on the Sun, such as coronal holes and active regions.

  Geophysical Joint Inversion for Accurate Brain Myelin Mapping (JIMM)
  Prof. Dr. Tony Stöcker, DZNE
  Magnetic Resonance Imaging (MRI) is a routine tool in clinical diagnostics. In the past MRI was mainly used for subjective image reading by experienced radiologists, but the technology can be extended to quantitative biophysical tissue parameter mapping. Sensitizing the acquisition to the process of interest enables parameter estimation by subsequent model fitting of the MRI data. An important example is the estimation of myelin content. Myelin is a major factor for brain structure and function and brain demyelination is a consequence of many neurological diseases. Since myelin affects many physical processes, there exist different MRI-based approaches for myelin quantification. These approaches are based on model simplifications and assumptions, which result in biased estimates of myelin content. The idea of the proposed project is to simultaneously fit two or more of these models by utilizing joint inversion approaches well-known from geophysical imaging. Due to the inherently ill-posed nature of geophysical inverse problems, there exist long experience and a variety of mathematical tools to combine data of different physical nature to fit a single or even different material parameters. Such methodology was not yet applied in medical imaging applications. A massive improvement with respect to accuracy and reproducibility of myelin mapping is expected by combining complementary information drawn from several MRI acquisitions.

  HIP Modalities
  Dr. Philipp Heuser, Helmholtz Imaging
  Helmholtz Imaging Modalities is the heart of the Helmholtz Imaging Network. The platform makes it easy to get an overview on the diverse cutting edge imaging possibilities across the Helmholtz Association, finding the experts experienced in the application of these and will for the first time give an complete picture of the world of imaging at Helmholtz at one place. All Helmholtz imaging enthusiasts can create their profile, describe their instruments and modalities, and the respective field of application. Helmholtz Imaging Modalities facilitates finding partners and collaborators for novel projects, with just the right complementary expertise necessary to conduct research for grand challenges.
  As of today, we invite you to create your profile at HIP Modalities, to describe your applications, modalities and instruments, and are looking forward to bring HIP Modalities to life together with you!

• Session block 2

  Breaking resolution limit of electron microscopy for magnetic materials
  Dr. Xiaoke Mu, KIT
  Scanning transmission electron microscopy (STEM) is one of the few approaches enabling direct imaging the magnetic properties of nano-materials and devices. However, field-free imaging conditions are required to image the native field inside the sample, restricting the instrument optics, resulting a limitation of spatial. This project is aiming to break the resolution limit in magnetic imaging by implementing ptychography for the field-free STEM. The talk will firstly introduce the concept of STEM and its drawback for imaging the magnetic domain structure. Then we will introduce the idea of how to improve the resolution. The proposed technique is expected to open a new path for research of nanoscale magnetic phenomena.

  Advanced table-top XUV and soft X-ray microscopy
  Wilhelm Eschen, Helmholtz Institute Jena
  State-of-the art high harmonic sources nowadays provide coherent radiation in the extreme ultraviolet with high photon flux. This enables coherent imaging experiments that were so far only possible at large-scale research facilities (e.g. synchrotrons) on a laboratory scale. In this talk, we will present a new EUV microscope operating at a wavelength of 13.5 nm. It achieves a resolution of sub 20 nm and allows quantitative measurements of semiconductor samples. We aim at advancing this technique towards 3D imaging in the soft X-ray spectral region

  SATOMI – Tackling the segmentation and tracking challenges of growing colonies and microbial diversity
  Tim Scherr, FZJ
  in collaboration with: Johannes Seiffarth, Bastian Wollenhaupt, Oliver Neumann, Hanno Scharr, Dietrich Kohlheyer, Katharina Nöh, Ralf Mikut
  Seamless Training Data Creation, Segmentation and Tracking using OMERO.
  Bacteria grow and divide in expanding colonies, with morphologies ranging from simple cocci and rod shapes to complex filamentous networks. A reliable analysis of acquired microscopy image sequences is essential for fundamental biological discovery. The SATOMI platform intends to accelerate the time-to-insight in the field of microbial single-cell analysis by combining training data creation and simulation, adapted state-of-the-art deep learning-based segmentation methods and a probabilistic tracking module. For a seamless user experience, all software tools are designed to interact with each other and OMERO is used for file handling.

  SyNaToSe
  Dr. Dagmar Kainmüller, MDC
  Nanoscale 3d microscopy techniques are currently at the forefront of facilitating novel scientific findings across a range of research fields and applications. However, automated analysis of the terabytes of 3d image data acquired during a typical recording is still an unmet challenge. To this end, a number of groups within Helmholtz Health and Helmholtz Materials are working towards ML solutions specific to their individual data analysis problems. SyNaToSe aims at leveraging synergies across these efforts and groups by means of domain adaptive machine learning, with a focus on image segmentation problems.

  Interactive Machine Learning at Helmholtz Imaging
  Dr. Paul Jäger, Helmholtz Imaging
  Considering human interaction when designing machine learning (ML) systems bears great potential: On the one hand, decision-making in ML systems remains imperfect in practice, thus requiring human interaction for safety-critical applications such as clinical diagnostics. On the other hand, the burden of manual training data annotation can be alleviated by means of human-in-the-loop scenarios. Taking this human-centered perspective, the Young Investigator Group Interactive Machine Learning (IML) headed by Paul Jäger strives to pioneer ML research directed at real-life applications. This talk gives an overview over the covered topics including probabilistic modeling, explainable AI, user modeling, active learning, and interactive systems with a special focus on image analysis tasks such as object detection or segmentation. A further focus is set on the appropriate and application-oriented evaluation of ML systems.

• Session block 3

  Ultra Content Screening – a multiplex immunoassay
  Gabriel Crespo López-Urrutia, FZJ
  Single cell proteomics has the potential to revolutionize the field of systems biology and the associated applied sciences, including biomarker discovery and drug development. Ultra-content screening is an optical method based on cyclic accumulation of fluorescence data resulting in composite images containing several analytes. In this joint project, the partners from Research Center Jülich and the German Cancer Research Center aim to deliver proof-of-concept that UCS is capable of detecting a minimum of 30 analytes in single cells. To this end, we will develop a suitable surface chemistry for cell immobilization, as well as protocols for sample staining, imaging, and de-staining. The whole UCS workflow will be automated to a large extent in order to obtain a robust method that generates reproducible data, free of human bias. In the explorative phase, we will apply UCS for biomarker discovery in bone marrow samples from leukemia and enriched circulating tumor cells from breast cancer patients. We will benchmark and cross-validate UCS data versus transcriptomic analyses (CITE-Seq). Results of this interdisciplinary project will have an impact on major biological and biomedical fields, such as signaling network regulation and cell heterogeneity and will thus contribute to bringing single cell proteomics towards broad clinical application.

  The Hidden Image of Thawing Permafrost: Mapping Subsurface Properties​ and Taliks with Remote Sensing
  Dr. Inge Grünberg, AWI
  in collaboration with Julia Boike, Ronny Hänsch, Paloma Saporta and Irena Hajnsek
  Northern landscapes and infrastructure are threatened by degrading permafrost across the Arctic due to a rapidly warming climate. The future degradation of permafrost strongly depends on subsurface characteristics, which are largely unknown. We plan to quantify these characteristics, in particular the frozen/unfrozen state, ice content, and soil properties in a novel imaging pipeline. Innovative radar technologies are be used and paired with expert knowledge and ground data of permafrost properties. In our imaging pipeline we will use an applied understanding of processes with model-based conceptualizations (CryoGrid). We obtained high-resolution images from various, independent data sets (optical, LiDAR, radar, ground based, numerical model based) from aerial and ground surveys during 2018. The test area, Trail Valley Creek, is 15x7 km² large and located in the North West Canadian permafrost zone, at the forest - tundra boundary zone. As first step of our imaging pipeline, we map vegetation types. Vegetation serves as model input, covariate in the radar data interpretation, and as proxy for subsurface conditions. Preliminary results show, that LiDAR data alone can provide enough information for mapping vegetation types at high spatial resolution (up to 1m1m) when informed by training data from even higher resolution optical images (0.15m0.15m per pixel).

  Neuroimaging biomarkers for Restless Legs Syndrome: first steps
  Dr. Kaustubh R. Patil, FZJ & Prof. Konrad Oexle, HMGU
  In this project we propose to identify biomarkers for the Restless Legs Syndrome (RLS) based on non-invasive neuroimaging data. RLS is a common movement disorder affecting 5-10% of Europeans. It manifests as an urge to move the legs, resulting in severe insomnia and subsequent depression, and represents a major public health burden. Diagnosis and early detection of RLS remains challenging. We aim to use multimodal neuroimaging derived features together with machine learning approaches to build predictive models for RLS. As large neuroimaging RLS-specific cohorts are rare, we will use polygenic risk scores derived from the genotype data as a proxy target. For this purpose we leverage the large population based cohort UK biobank which provides genotype data for almost 500k subjects and neuroimaging data for 50k subjects. In the first phase we are actively developing modular and reusable software infrastructure for this highly interdisciplinary project. The challenges presented by UKB in handling big data and collaborative approaches and the resulting know-how and reusable software will be useful for other endeavours in HGF as well as this project will serve as a blueprint for further applications.

  Tools for image data solutions across scales
  Dr. Kyle Harrington, Helmholtz Imaging
  Image data analysis is performed with a number of different tools and programming languages. Sometimes these correlate with imaging modality and domain, which can reduce the transfer of knowledge across domains. This talk will discuss how we can unify image data solutions to promote the exchange of algorithms and tools across scientific disciplines.

Posters

Towards a deep-learning-based approach for improving landslide detection using Synthetic Aperture Radar Data
Prof. Dr. Mahdi Motagh, Wandi Wang, GFZ

Semi-blind deconvolution for extended depth-of-field images of marine plankton
Tomas Chobola, TUM/Helmholtz AI
We intend to present a deep-learning-based semi-blind deconvolution algorithm applied to extended depth-of-field (EDOF) underwater microscopy images. Unlike the standard microscopy imaging systems, which are limited by the inevitability of a trade-off between magnification and depth-of-field, images taken by the EDOF systems (that use an electrically tunable lens) include all in-focus image slices during one image exposure, as well as all out-of-focus blurry slices that decrease the resolution and contrast perceived. We integrate the classic Wiener filer to deconvolve the images with the network-generated kernel.

SATOMI
Johannes Seiffarth, FZJ
Software Ecosystem topic: We show the developed software tools within the project and present the workflow from raw image data to segmented cells.

Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations
Sophia Wagner, Helmholtz AI

Automatic Cell Counter
Ruolin Shen, Helmholtz AI Munich

Correlated Characterization of Glasswing Butterfly Wings
Janis Heuer, Richard Thelen, Hendrik Hölscher, Institute for Microstructure Technology, Karlsruhe Institute of Technology (KIT)
It is of high interest in many scientific fields to obtain insights into complex systems or materials by correlation. This is often achieved by the spatial combination of two or more data sets. Our study achieves a causal relationship by imaging the same surface spot of the transparent wing with multiple instruments, such as atomic force microscopy, vertical scanning interferometry, standard light microscopy, and optical spectrometry. We use reference markers, an in-house written coordinate calculation code, and Digitalsurf’s MountainsMap software to create high-precision 3D overlays. This improves our understanding of the surface structure and its resulting optical properties.

Patient-specific virtual spine straightening and vertebra inpainting: An automatic framework for osteoplasty planning
Christina Bukas, HMGU/Helmholtz AI

HIP modalities & image anlysis
Philipp Heuser, Helmholtz Imaging, DESY

MemBrain -- A Deep Learning-aided tool for the automated analysis of membranes in Cryo-electron tomograms
Lorenz Lamm, HMGU/Helmholtz AI
In Cryo-electron tomography, we have to deal with a low signal-to-noise ratio, as well as image distortions caused by the "missing wedge", making the manual and automated analysis of the tomograms very challenging. However, for understanding biological processes, e.g., photosynthesis, on a molecular level, we need some reliable methods for the analysis of several compartments of the cell. In particular, for our application, the analysis of membranes is very important.

We present our current developments, in collaboration with various Helmholtz partners, for a label-efficient segmentation of membranes, as well as an already implemented approach specialized on the automated detection of proteins that are embedded within membranes.