Speaker
Description
In computational neuroscience, systematically monitoring the performance of simulation software remains a major challenge due to continuous technological advances and the to the diversity of scales across relevant network models. Previous work by Albers et al. [1] introduced conceptual foundations and an open-source framework for benchmarking neuronal network simulators. However, two years of operation shows that individual setup and configurations still poses significant barriers to reproducibility and systematic use within and across laboratories. Here, we extend the earlier approach by developing the concept of continuous benchmarking based on principles of continuous integration. All required artifacts, including configurations, environments, and results of the benchmarks are generated automatically and stored centrally. By introducing a unified workflow specification, we decouple benchmark execution from individual researchers and hardware-specific configurations, thereby lowering the barrier of entry for first-time benchmarkers and improving reproducibility, repeatability, and comparability across platforms and code versions. We illustrate the benefits of this approach with use cases in collaborative simulator development, model execution and cross-machine benchmarking. The concept enables early detection of performance regressions, fosters collaborative model refinement, and supports the advancement of simulation technologies in brain research.
References
[1] Albers, J. et al. (2022), A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations, Frontiers in Neuroinformatics, 16, 837549
References
[1] Albers, J. et al. (2022), A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations, Frontiers in Neuroinformatics, 16, 837549
| Preferred form of presentation | Poster & advertising flash talk |
|---|---|
| Topic area | Interoperability, data and infrastructure |
| Keywords | continuous integration, benchmark |
| Speaker time zone | UTC+2 |
| I agree to the copyright and license terms | Yes |
| I agree to the declaration of honor | Yes |
