organized in cooperation with Helmholtz Information & Data Science Academy (HIDA) and Helmholtz AI
Overview of AI's Parallelization Methods in Supercomputers
This course introduces the essentials for running deep learning models on supercomputers and effectively scaling them. It provides foundational skills to enable efficient training of large models.
The course cover alternating sequences of theoretical input and hands-on exercises, during which the instructors are available for quick feedback and advice.
Learning Goals
Day 1: Supercomputer Access Basics
-
Understand what a supercomputer is.
-
Configure SSH keys.
-
Setup VSCODE.
-
Use software packages of the supercomputer.
-
Run your first job on the supercomputer.
-
Bonus: Blablador.
Day 2: Distributed Data Parallel (DDP)
-
The good practices before starting training.
-
Where to store your data and how to load it
-
Run your first PyTorch code on the supercomputer.
-
Understand what is a distributed training.
-
Understand DDP.
-
Transform your code to a distributed one with DDP.
-
Use Tensorboard on the supercomputer.
-
Check GPU usage with llview.
Prerequisites
To participate in this course, you need to know
-
How to code with Python and PyTorch
-
Machine Learning and Deep Learning
Target Group
This course is addressed for whoever wants to learn how to scale their models (students, researchers, employees …).
Course Days & Times
June 24, 2025, 1 pm - 5 pm
June 25, 2025, 1 pm - 5 pm
NOTE: Registration will open May 27, 2025, 12 pm.
Attendance & Certificates
The course content is coordinated, so we strongly recommend that you do not miss any part of the course. To receive a certificate we expect at least 70 % attendance and active participation.
Registration & Cancellation
This course is open to individuals affiliated with Helmholtz or a HIDA Partner only.
Your registration for this course is binding. If you need to leave/miss the course for a period of time, please let us know in advance via hida-courses@helmholtz.de.
If you have to cancel the course for any reason, please do so as soon as possible to allow time for others to take your seat. To cancel, please withdraw your registration on the course site or write an email to hida-courses@helmholtz.de.
Additional Information
There is no waiting list for this course! If someone withdraws from a course, their place is automatically reopened. We therefore advise you to keep an eye on the registration in case the course is fully booked and you would like to attend. Also, this course will be offered again in the future - you can check our HIDA course catalog for updates.
This course is free of charge.
