Alina Bazarova, Stefan Kesselheim
Helmholtz AI, Jülich Supercomputing Center, Forschungszentrum Jülich
When: 13:15 - 18:00
Where: GRS seminar room, building 16.15., room 2009 [coordinates]
Artificial intelligence (AI) techniques are constantly changing scientific research, but their potential to enhance simulation pipelines is not widely recognised. Conversely, Bayesian inference is a well-established method in the research community, offering distributional estimates of model parameters and the ability to update models with new data. However, traditional Bayesian inference often faces computational challenges and limited parallelisation capabilities.
Simulation Based Inference (SBI) presents a comprehensive solution by combining simulations, AI techniques, and Bayesian methods. SBI utilizes AI-driven approximate Bayesian computation to significantly reduce inference times and produce reliable estimates, even with sparse observed data. This approach allows any representative simulation model to inform parameter constraints, leading to approximate posterior distributions. Furthermore, SBI enables workload distribution across high-performance computing clusters, further decreasing runtime.
This tutorial explores the theoretical foundations and provides hands-on training for constructing tailored SBI frameworks for specific models. Through practical examples, participants will gain insights into different levels of model granularity, ranging from a simple black box approach to a highly customizable design. By participating in this tutorial, attendees will develop the skills necessary to implement Simulation Based Inference in their own research projects.
Topics to be covered:
- Key features of Bayesian Inference and examples when classical Bayesian approach fails
- Typical SBI pipelines: one-liner, flexible interface, summary statistics
- Different SBI methods: SNLE, SNRE, SNPE
- Neural network architectures behind SBI
- Parallelisation and distributing DBI over multiple nodes
Requirements
- Laptop is required to be able to access the HPC system.
- Basic familiarity with statistical and deep learning concepts is expected.
- Experience of working with HPC systems would be beneficial but not strictly required.
Latest details [link]
Course schedule
14.00 – 14.20 Introduction, tutorial overview, onboarding to HPC system
- Teaching content & Overview of the tutorial
- Conveying the learning objectives
- Onboarding participants to JUWELS Booster HPC System
14.20 – 14.40 Lecture: Basic concepts of classical Bayesian inference
- Key features of Bayesian Inference
- Bayes rule
- Concepts of posterior and prior distributions
- Issues emerging when performing classical Bayesian Inference
- Real life examples
14.40 – 14.55 Hands-on: warm-up example in a Jupyter notebook
- A simple coin-flipping example implemented within Jupyter notebook
14.55 - 15.10 Lecture: basic concepts of Simulation Based Inference
- A typical SBI pipeline
- Key SBI methods and algorithmic discussion of those
15.10-15.30 Hands-on: Converting classical Bayesian example into an SBI one, Jupyter notebook
- Using previously introduced example to write the first SBI pipeline
- Compare different SBI methods on the same example
15.30 - 16.00 Coffee-break
16.00 - 16.30 Hands-on: data example. MCMC vs SBI, Jupyter notebook
- Using a data example to run classical MCMC algorithm to get parameter estimates
- Put the example into the SBI framework
16.30 - 16.45 Lecture: Deep Learning component and Sequential estimation
- Estimation through normalizing flows: advantages and disadvantages
- Estimation through a Neural Network classifier: parallels with MCMC
- Concepts of Sequential Estimation
16.45 – 17.15 Hands-on: flexible interface of the sbi package, Jupyter notebook
- Utilise previously used data example to illustrate flexible interface of the SBI package
- Customise neural network within the SBI machinery
- Perform sequential inference on the same data example
17.15 – 17.35 Hands-on: Constructing a summary statistic
- Extract summary statistic from the previously used data example
- Compare the outputs given different parameters of the summary statistic
- Use an example of Stochastic Differential equations to extract more complex summary statistic
- Evaluate the results
17.35 – 18.00 Parallelization and distributing SBI over multiple nodes
- Parallelise the simulations when using one node only
- Distribute the simulations over multiple nodes by means of Ray backend
- Use appropriate slurm script for the corresponding batch job submission
