ATTENTION! On Wednesday, October 22 at 10:00 a.m., we will delete all events older than 6 months from the Test category. If you have regular events there, please move them in time to prevent data loss!

Mar 5 – 7, 2024
Julius-Maximilians-Universität Würzburg
Europe/Berlin timezone

Efficient Docker Container Management and CI Strategies for Research Software Engineering

Mar 6, 2024, 2:30 PM
20m
HS4

HS4

Talk (15min + 5min) Advanced Continuous Integration - Advanced

Speaker

Dominik Thoennes (Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU))

Description

This presentation focuses on the continuous integration (CI) infrastructure used by two simulation software frameworks with a strong emphasis on performance and scalability: HyTeG, a C++ framework for large-scale high-performance finite element simulations based on geometric multigrid, and waLBerla, a massively parallel framework for multiphysics applications.

During this talk, we will detail our approach to managing Docker Containers within our CI environment. As we develop software for research purposes, we often face the challenge of testing various combinations of software packages, especially different compilers. To tackle this, we employ Python scripting and the Spack software manager to create Docker Containers. These containers are then utilized within the CI for our research software. The container creation process is automated within the CI itself, enhancing consistency and manageability.

Furthermore, we will explain how Python scripting generates the CI specifications for our GitLab CI. With approximately 150 different jobs in our Pipeline, a generated CI significantly reduces the chances of errors compared to manual creation and maintenance. Moreover, we will shine a light on some of the less conventional uses of CI, such as analyzing build times or conducting code style checks.

Our goal for this talk is to share best practices and spread ideas about more efficient CI utilization.

Slot length other(help with comment)

Author

Dominik Thoennes (Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU))

Co-author

Harald Köstler (Friedrich-Alexander Universität Erlangen-Nürnberg)

Presentation materials