Large language models

Description of this Post

1 Title

Slide

2 Outline.

Slide

3 Large language models

Slide

4 Why is this useful?

Slide

5 What can we expect this model to capture?

Slide

6 ELMo: Embeddings from Language Models

Datare at al 9N12 Maan rnntoyvtalizad winrdn ranrocaontatinne

Slide

7 The ELMo model

Slide

8 The contributions of ELMo

Slide

9 The rise of the Transformer

Slide

10 BERT: Architecture

Slide

11 BERT: Input representations

Slide

12 BERT: Pretraining tasks

Slide

13 BERT: Pretraining tasks

Slide

14 BERT: pretraining

Slide

15 BERT: fine-tuning

Slide

16 The contributions of BERT

Slide

17 Outline.

Slide

18 Generative language models: The GPT family

Slide

19 More than a language model?

Slide

20 InstructGPT and ChatGPT

Slide

21 An example from ChatGPT

Slide

22 Reinforcement learning from human feedback

Slide

23 Reinforcement learning from human feedback

Slide

24 Training a reward model

Slide

25 Training a reward model

Slide

26 Fine-tuning with reinforcement learning

Slide

27 Outline.

Slide

28 Instruction-tuned LLMs and multi-task learning

Sanh etal 2022 Multitasck Promoted Training Enables Zero-Shot Task

Slide

29 Multilingual LLMs

Slide

30 Multilingual LLMs: Intuition

Slide

31 Multilingual LLMs: Models

TLlRAtn4d4an lARAIIRARR ARKRRRARBA yathin nnn masAazl

Slide

32 Multilingual LLMs: Application

Slide

33 Can LLMs solve NLP?

Slide

34 Can LLMs solve NLP?

Slide

35 Outstanding challenges and future directions

Slide