Speaker
Description
Aims
libyt
provides researchers a way to analyze and visualize data using yt
(a Python package for analyzing and visualizing volumetric data) or any other Python packages during simulations runtime. User can either use a Python script or enter Python statements to analyze the ongoing data in simulations and get feedbacks instantly. This improves disk usage efficiency and makes analyzing large-scale simulation feasible.
Methods
Connecting Python and Simulation
- Using Python C API and NumPy C API,
libyt
provides an interface for exchanging data between simulations and Python instance.
In Situ Analysis Under MPI
- Each MPI process contains one piece of simulation code and one Python instance. When launching N MPI processes, there will be a total of N Python instances working together to conduct in situ analysis. Since
yt
supports MPI parallelism feature,libyt
use it directly. - In in situ analysis, data are distributed in different processes. We use one-sided communication in MPI, also known as Remote Memory Access (RMA) for data exchange process during in situ analysis.
Applications
Analyzing Fuzzy Dark Matter Vortices Simulation using GAMER
+ libyt
We use GAMER
to simulate the evolution of vortices form from density voids in a Fuzzy Dark Matter halo. In order to investigate the dynamics of these vortices, a very high spatial and temporal resolution in simulation is required.
Each simulation snapshot takes 116 GB, and a total of 321 simulation snapshots are required to capture their evolution clearly. Roughly 37 TB disk space is needed if we were to do this post-processingly. libyt
provides a promising approach by using yt
function covering_grid
to extract our region of interest, which now consumes only 8 GB. It is 15 times smaller than that in post-processing per simulation step.
Analyzing Core-Collapse Supernova Simulation using GAMER
+ libyt
We use GAMER
to simulate core-collapse supernova explosions. libyt
facilitates closely monitoring the simulation progress during runtime. We use yt
function SlicePlot
during in situ analysis to plot the gas entropy distribution. Since entropy is not envolved in simulation's iterative process, these data will only be generated through simulation provided function only when they are needed in in situ analysis. libyt
handles data transition between simulation and Python.
Discussion and Conclusion
libyt
is an open source software.libyt
provides a promising method to analyze and visualize data in parallel during simulation runtime with minimal memory overhead and slightly faster than post-processing under the same environment.libyt
focuses on usingyt
as its core analytic method, even though we can call arbitrary Python modules usinglibyt
.
JLESC topic | C++ library, Embedded Python |
---|