Telegrafenberg Vision Transformers Workshop

6 Jun 2024, 09:30 → 7 Jun 2024, 13:00 Europe/Berlin

Große Kuppel, A31 Michelson Haus, Potsdam

Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)

Machine learning is becoming a popular method for Geo-scientists. Domain researchers often face challenges when applying the ML tutorial concepts to actual geo-scientific data.

Therefore, this workshop is aimed at demonstrating the application of Transformer on Geo-scientific data. The course will be held in person, and the number of participants are limited to 21 persons. You should feel comfortable reading and understanding Python code. We will follow along, step by step, the relevant lines of code pre-written in a Jupyter notebook.

The course will be held from 6 - 7 June, 2024 09:30 - 13:00 at Große Kuppel, A31 Michelson Haus.

Coffee breaks will be provided during the workshop.

Target group: Domain scientist (Geo-scientists) with some Python experience

Deadline for registration: 15, May 2024

This event is organized and hosted by Helmholtz AI, Hereon & DKRZ (Adeniyi Mosaku, Danu Caus) and the ML@Telegrafenberg Interest Group (Jonas Kuppler, Marco De Luca & Hannah Zoller)

1. Introduction to Transformer.pdf

2. Attention.pdf

3. Vision Transformer I & II.pdf

4. BERT on Seismic Pre-processing.pdf

Thursday 6 June

09:30 → 10:30 Introduction to Transformer

• Introduction to Transformer
• Transformer as a Neural Network Model
• Vision Transformer (Focus)
• Relationship between Transformer and Geo-scientific data

Speaker: Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)

10:30 → 11:00 Introduction to PyTorch

• Building PyTorch Models

Speaker: Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)

11:00 → 11:30 Coffee Break

11:30 → 13:00 Attention

• Step by step code implementation

Speaker: Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)

Friday 7 June

09:00 → 11:00 Vision Transformer I

• Theory, Paper and Code

Speaker: Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)

11:00 → 11:30 Coffee Break

11:30 → 13:00 Vision Transformer II

• Theory, Paper and Code

• +++ If time permits: Pre-training of BERT Models on Seismic data (Introduction)

Speaker: Adeniyi Mosaku (Helmholtz AI, Hereon, DKRZ)