Telegrafenberg Transformer Workshop

23 Nov 2023, 10:00 → 24 Nov 2023, 14:00 Europe/Berlin

Haus H, Telegrafenberg, Potsdam

Danu Caus (Helmholtz AI, Hereon, DKRZ)

Since all places were booked, the registration is closed at the moment. We may reopen it in the future if some of the already registered people cancel their participation.

This is a two-day workshop on the theory and practice of transformers. You will learn how transformers work internally and this will help you decide if they could be useful in your Geoscientific research. We will present transformers in their original natural language processing context by building a mini-chatGPT from scratch to illustrate all the building blocks of a transformer. This way you will understand how transformers handle sequence data on a deeper level. First day will be reserved for theoretical aspects and to illustrate potential applications of transformers in different domains, whereas second day will be focused on coding. You should feel comfortable reading and understanding Python code. We will follow along, step by step, the relevant lines of code pre-written in a Jupyter notebook.

This event is organized and hosted by Helmholtz AI, Hereon & DKRZ (Danu Caus, Adeniyi Mosaku) and the ML@Telegrafenberg Interest Group (Jonas Kuppler, Marco De Luca & Hannah Zoller)

Thursday 23 November

10:00 → 11:00 Theory and applications of Transformers, Part 1

Speaker: Danu Caus (Helmholtz AI, Hereon, DKRZ)

11:00 → 11:30 Coffee break

You also have the chance to set up the Miniconda / Python environment for the next day practical session.

11:30 → 12:30 Theory and applications of Transformers, Part 2

Speaker: Danu Caus (Helmholtz AI, Hereon, DKRZ)

12:30 → 13:00 Set up and test Miniconda / Python environment for next day practical session

Optional session for those interested.

13:00 → 14:00 Optional lunch gathering

Friday 24 November

10:00 → 11:00 Building a mini-chatGPT transformer from scratch, Part 1

Speaker: Danu Caus (Helmholtz AI, Hereon, DKRZ)

11:00 → 11:30 Coffee break

11:30 → 13:00 Building a mini-chatGPT transformer from scratch, Part 2

Speaker: Danu Caus (Helmholtz AI, Hereon, DKRZ)

13:00 → 14:00 Optional lunch gathering