organized in cooperation with Helmholtz Information & Data Science Academy (HIDA) and Helmholtz AI
Overview of AI's Parallelization Methods in Supercomputers
This course introduces the essentials for running deep learning models on supercomputers and effectively scaling them. It provides foundational skills to enable efficient training of large models.
The course cover alternating sequences of theoretical input and hands-on exercises, during which the instructors are available for quick feedback and advice.
Learning Goals
Day 1: Supercomputer Access Basics
-
Understand what a supercomputer is.
-
Configure SSH keys.
-
Setup VSCODE.
-
Use software packages of the supercomputer.
-
Run your first job on the supercomputer.
-
Bonus: Blablador.
Day 2: Distributed Data Parallel (DDP)
-
The good practices before starting training.
-
Where to store your data and how to load it
-
Run your first PyTorch code on the supercomputer.
-
Understand what is a distributed training.
-
Understand DDP.
-
Transform your code to a distributed one with DDP.
-
Use Tensorboard on the supercomputer.
-
Check GPU usage with llview.
Prerequisites
To participate in this course, you need to know
-
How to code with Python and PyTorch
-
Machine Learning and Deep Learning
Target Group
This course is addressed for whoever wants to learn how to scale their models (students, researchers, employees …).
Course Days & Times
March 18, 2025, 1 pm - 5 pm
March 19, 2025, 1 pm - 5 pm
NOTE: Registration will open February 06, 2025, 12 pm.
Attendance & Certificates
The course content is coordinated, so we strongly recommend that you do not miss any part of the course. To receive a certificate we expect at least 70 % attendance and active participation.
Registration & Cancellation
This course is open to individuals affiliated with Helmholtz or a HIDA Partner only. You may register for the course allocating yourself to one of the following groups:
- All Helmholtz affiliations
- Helmholtz Information & Data Science School (HIDSS) affiliation
- HIDA Partner affiliation
Please note that after the first two weeks of the registration period the unbooked seats from categories 2 and 3 will be opened for all Helmholtz affiliations (category 1).
Your registration for this course is binding. If you need to leave/miss the course for a period of time, please let us know in advance via hida-courses@helmholtz.de.
If you have to cancel the course for any reason, please do so as soon as possible to allow time for others to take your seat. To cancel, please withdraw your registration on the course site or write an email to hida-courses@helmholtz.de.
Additional Information
There is no waiting list for this course! If someone withdraws from a course, their place is automatically reopened. We therefore advise you to keep an eye on the registration in case the course is fully booked and you would like to attend. Also, this course will be offered again in the future - you can check our HIDA course catalog for updates.
This course is free of charge.
