Telegrafenberg Vision Transformers Workshop
from
Thursday 6 June 2024 (09:30)
to
Friday 7 June 2024 (13:00)
Monday 3 June 2024
Tuesday 4 June 2024
Wednesday 5 June 2024
Thursday 6 June 2024
09:30
Introduction to Transformer
-
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
Introduction to Transformer
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
09:30 - 10:30
- Introduction to Transformer - Transformer as a Neural Network Model - Vision Transformer (Focus) - Relationship between Transformer and Geo-scientific data
10:30
Introduction to PyTorch
-
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
Introduction to PyTorch
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
10:30 - 11:00
- Building PyTorch Models
11:00
Coffee Break
Coffee Break
11:00 - 11:30
11:30
Attention
-
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
Attention
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
11:30 - 13:00
- Step by step code implementation
Friday 7 June 2024
09:00
Vision Transformer I
-
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
Vision Transformer I
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
09:00 - 11:00
- Theory, Paper and Code
11:00
Coffee Break
Coffee Break
11:00 - 11:30
11:30
Vision Transformer II
-
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
Vision Transformer II
Adeniyi Mosaku
(
Helmholtz AI, Hereon, DKRZ
)
11:30 - 13:00
- Theory, Paper and Code - +++ If time permits: Pre-training of BERT Models on Seismic data (Introduction)