HIDA Lecture Series "AI and Large Language Models" - A Short Introduction to Efficient Natural Language Processing

Monday 25 Nov 2024, 11:00 → 12:00 Europe/Berlin

Zoom (online)

Join our next lecture from our series on AI and LLMs with Simon Ostermann from the German Research Center for Artificial Intelligence (DFKI)!

A short Introduction to Efficient Natural Language Processing

Date

November 25 2024, 11:00 am - 12:00 pm 

Abstract

This talk will outline the challenges of developing efficient models with limited data and resources. It explores strategies to maximise data and model efficiency, emphasising the importance of managing large models that typically require significant computational resources and are predominantly trained on English data. Techniques like pre-filtering, online methods, data augmentation, and curriculum learning are discussed, as well as parameter-efficient training methods like adapters, prompt tuning, and prefix tuning to enhance model performance without extensive data requirements.

Speaker

Simon Ostermann is a computational linguist and Senior Researcher at the German Research Center for Artificial Intelligence (DFKI), where he leads a research group on Efficient and Explainable Natural Language Processing in the lab for Multilinguality and Language Technology.

His research interests are on trying to improve the accessibility of Large Language Models (LLMs) in several aspects. First, by making the parameters and behaviour or LLMs more explainable and understandable to both end users and researchers, second, by improving language models in terms of their data consumption and size.

Simon Ostermann is currently leading and participating in a range of national and European research projects. He has have been working both in academic research and industrial research and development, especially in the automotive sector.