15th JLESC Workshop

libyt: a Tool for Parallel In Situ Analysis with yt

21 Mar 2023, 18:30

1h

Dinner venue

Poster I/O, storage and in-situ processing Poster Session

Speaker

Shin-Rong Tsai (University of Illinois at Urbana-Champaign)

Description

Aims

libyt provides researchers a way to analyze and visualize data using yt (a Python package for analyzing and visualizing volumetric data) or any other Python packages during simulations runtime. User can either use a Python script or enter Python statements to analyze the ongoing data in simulations and get feedbacks instantly. This improves disk usage efficiency and makes analyzing large-scale simulation feasible.

Methods

Connecting Python and Simulation

• Using Python C API and NumPy C API, libyt provides an interface for exchanging data between simulations and Python instance.

In Situ Analysis Under MPI

• Each MPI process contains one piece of simulation code and one Python instance. When launching N MPI processes, there will be a total of N Python instances working together to conduct in situ analysis. Since yt supports MPI parallelism feature, libyt use it directly.
• In in situ analysis, data are distributed in different processes. We use one-sided communication in MPI, also known as Remote Memory Access (RMA) for data exchange process during in situ analysis.

Applications

Analyzing Fuzzy Dark Matter Vortices Simulation using GAMER + libyt

We use GAMER to simulate the evolution of vortices form from density voids in a Fuzzy Dark Matter halo. In order to investigate the dynamics of these vortices, a very high spatial and temporal resolution in simulation is required.

Each simulation snapshot takes 116 GB, and a total of 321 simulation snapshots are required to capture their evolution clearly. Roughly 37 TB disk space is needed if we were to do this post-processingly. libyt provides a promising approach by using yt function covering_grid to extract our region of interest, which now consumes only 8 GB. It is 15 times smaller than that in post-processing per simulation step.

Analyzing Core-Collapse Supernova Simulation using GAMER + libyt

We use GAMER to simulate core-collapse supernova explosions. libyt facilitates closely monitoring the simulation progress during runtime. We use yt function SlicePlot during in situ analysis to plot the gas entropy distribution. Since entropy is not envolved in simulation's iterative process, these data will only be generated through simulation provided function only when they are needed in in situ analysis. libyt handles data transition between simulation and Python.

Discussion and Conclusion

• libyt is an open source software.
• libyt provides a promising method to analyze and visualize data in parallel during simulation runtime with minimal memory overhead and slightly faster than post-processing under the same environment.
• libyt focuses on using yt as its core analytic method, even though we can call arbitrary Python modules using libyt.