Скачать с ютуб Mor Geva: Transformer Feed Forward Layers are Key-Value Memories, and Build Predictions в хорошем качестве

Mor Geva: Transformer Feed Forward Layers are Key-Value Memories, and Build Predictions

Talk given by Mor Geva to the Neural Sequence Model Theory discord on the 9th of May 2022. Thank you Mor! Papers and resources mentioned during the talk and following discussion: Main Papers: - (First half:) Transformer Feed Forward Layers are Key-Value Memories (Geva et al, 2021) https://aclanthology.org/2021.emnlp-m... - (Second half, starting at 00:36:42) Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (Geva et al, 2022) https://arxiv.org/abs/2203.14680 Other references: - The LM-Debugger tool: https://blog.allenai.org/introducing-... - Locating and Editing Factual Knowledge in GPT (Meng et al, 2022) https://arxiv.org/pdf/2202.05262.pdf , https://rome.baulab.info - resource for checking layernorm position in GPT2: https://github.com/huggingface/transf...