21–23 Mar 2023
LaBRI
Europe/Paris timezone

Steering Large Scale Ensemble Simulations for Online DNN Training with Adaptive Sampling

21 Mar 2023, 18:30
1h
Dinner venue

Dinner venue

Poster AI and ML/DL Poster Session

Speaker

Sofya Dymchenko (INRIA and UGA)

Description

Simulation-based training of deep neural networks (DNN), such as surrogates and inference models, is technically challenging and expensive both memory- and computational-wise.

Large-scale deep learning applications for sciences (fluid dynamics, climate prediction, molecular structure exploration) demand novel approaches. One of them is online training, where the simulations are generated during the training process and used as soon as they are available. It benefits from (1) file-free processing and (2) ensemble steering. The first (1) overcomes the I/O bottleneck and enables the generation of large datasets that couldn’t be stored on disk. For example, in the context of sensitivity analysis, Melissa framework’s [1] largest experiment processed 270 TB of data online. The goal of the second (2) is to accelerate the training process and improve data efficiency. By monitoring the training state, it controls the parameterization of the next set of simulations to run.

We investigate strategies for adaptive simulation sampling for DNN train data, which range from Bayesian Optimal Experimental Design (BOED) and Simulation-Based Inference (SBI) to reinforcement learning.

[1] T. Terraz, A. Ribes, Y. Fournier, B. Iooss, and B. Raffin. Melissa: large scale in transit sensitivity analysis avoiding intermediate files. In Proceedings of the international conference for high performance computing, networking, storage and analysis, pages 1–14, 2017.

Primary author

Sofya Dymchenko (INRIA and UGA)

Co-author

Bruno Raffin (Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LIG, 38000 Grenoble, France)

Presentation materials

There are no materials yet.