Speaker
Description
Comprehensive simulation studies of dynamical regimes of cortical networks with realistic synaptic densities depend on compute systems capable of running such models significantly faster than biological real time. Since CPUs still are the primary target for established simulators, an inherent bottleneck caused by the von Neumann design is frequent memory access with minimal compute. Distributed memory architectures, popularized by the need for massively parallel and scalable processing for ML, offer an alternative.
We introduce extensible simulation technology for spiking networks on massively distributed memory using Graphcore's IPUs. We demonstrate the efficiency of the new technology based on simulations of the microcircuit model by (Potjans et al., 2014) commonly used as a reference benchmark. It represents 1~mm² of cortical tissue, and is considered a building block of cortical function. We present a custom communication algorithm especially suited for distributed and constrained memory environments, which allows a controlled trade-off between performance and memory usage. Our simulation code achieves an acceleration factor of 15x compared to real time for the full-scale cortical microcircuit model on the smallest device configuration capable of fitting the model in memory. This is competitive with the current record on a static FPGA cluster (Kauth et al., 2023), and further speedup can be achieved at the cost of lower precision.
With negligible compilation times, the simulation code can be be extended seamlessly to a wide range of synapse and neuron models, as well as structural plasticity, unlocking a new class of models for extensive parameter-space explorations in computational neuroscience.
Acknowledgements
The presented conceptual and algorithmic work is part of our long-term collaborative project to provide the technology for neural systems simulations (https://www.nest-initiative.org).
This research used resources of the Argonne Leadership Computing Facility, a U.S. Department of Energy (DOE) Office of Science user facility at Argonne National Laboratory and is based on research supported by the U.S. DOE Office of Science-Advanced Scientific Computing Research Program, under Contract No. DE-AC02-06CH11357.
This work is partly funded by Volkswagen Foundation.
References
Potjans, T. C. and Diesmann, M. (2012). The Cell-Type Specific Cortical Mi-
crocircuit: Relating Structure and Activity in a Full-Scale Spiking Network
Model. Cerebral Cortex, 24(3):785–806.
Kauth, K., Stadtmann, T., Sobhani, V., and Gemmeke, T. (2023). neuroaix-
framework: design of future neuroscience simulation systems exhibiting exe-
cution of the cortical microcircuit model 20× faster than biological real-time.
Frontiers in Computational Neuroscience, 17:1144143.
| Preferred form of presentation | Talk (& optional poster) |
|---|---|
| Topic area | Simulator technology and performance |
| Keywords | microcircuit |
| Speaker time zone | UTC+1 |
| I agree to the copyright and license terms | Yes |
| I agree to the declaration of honor | Yes |
