ALPHA Timepoint is in alpha Talk to Us
Sign In Get Started
P

Publication of "Attention Is All You Need" (2017)

The Google Brain team is presenting their groundbreaking paper 'Attention Is All You Need' at a research meeting, introducing the Transformer architecture that will revolutionize AI and machine learni

Setting

Google Brain headquarters conference room, Mountain View, California. A modern, spacious room with floor-to-ceiling windows overlooking the Google campus. The room is equipped with state-of-the-art presentation technology, including a large projector screen and multiple monitors.

Characters

The figures in this scene as an entity network — co-presence links everyone in the moment; speakers who trade lines are bound tighter. Turn the resolution dial to reveal depth the engine actually computed.

TNGF
SELECTED
Ashish Vaswani
primary
A man in his mid-30s with a lean build, short dark hair, and a neatly trimmed beard. His sharp, intelligent eyes are framed by rectangular glasses, and he has a composed yet approachable demeanor.
Noam Shazeer
primary
A tall, lean man in his late 30s with short, dark brown hair and a neatly trimmed beard. His sharp, intelligent eyes are framed by rectangular glasses, and he has a confident, approachable demeanor.
Senior Researcher
secondary
A middle-aged man with a slightly receding hairline, wearing thin-rimmed glasses. His face shows signs of deep thought, with faint wrinkles around his eyes from years of intense focus. He has a lean but not athletic build, typical of someone who spends long hours at a desk.
Junior Researcher
secondary
A young engineer in their mid-20s with a slim build, short dark hair, and glasses. Their face is animated with curiosity and excitement, and they have a slightly hunched posture from leaning forward intently.
Engineering Lead
background
A middle-aged man with a lean build, short-cropped dark hair with slight graying at the temples, and a neatly trimmed beard. He has sharp, observant eyes and a composed demeanor, exuding quiet authority.

Dialog

Ashish Vaswani To put it simply, the Transformer architecture eliminates the need for recurrence entirely—self-attention allows us to model dependencies regardless of distance.
Senior Researcher Help me understand—how does your positional encoding compare to traditional RNN-based approaches in handling long-range dependencies?
Noam Shazeer Think of it like this: instead of forcing the model to traverse sequences step-by-step, we give it a bird's-eye view of all positions simultaneously. The attention weights handle the rest.
Ashish Vaswani As the data shows on slide 12, our approach reduces training time by 75% compared to LSTMs—with better accuracy on WMT benchmarks.
Senior Researcher Those are impressive numbers. But does the quadratic memory complexity of self-attention limit your maximum sequence length in practice?
Noam Shazeer Ah—that's where our multi-head attention shines. By splitting the computation, we keep memory usage manageable while preserving the global context.
Ashish Vaswani Exactly. And early experiments suggest this architecture could scale far beyond what we've shown today—that's what excites me most.

Chat with Characters

You've used your 3 free turns

Sign in to keep chatting with characters from this moment — unlimited turns.

Sign in to Continue
— Sign in for unlimited

Causal neighbors · 114 linked moments

Publication of "Attention Is All You Need"
Publication of "Attention Is All You Need"
2017 · contemporaneous
R
Release of the Transformer paper "Attention is All You Need"
2017 · contemporaneous
G
Google AdWords launched
2000 · same figure
E
Establishment of the MIT Artificial Intelligence Laboratory
1959 · same figure
M
Moore's Law paper published
1965 · same figure
G
GPT-3 Paper arXiv Submission
2020 · same figure
F
First point-contact transistor demonstrated
1947 · same figure
R
Release of BERT (Bidirectional Encoder Representations from Transformers) Paper
2018 · same figure
R
Release of GPT-1 (Generative Pre-trained Transformer) Paper
2018 · same figure
P
Publication of BERT
2018 · same figure
X
XLNet Paper Presentation at ICML 2019
2019 · same figure
G
GPT-2 Paper arXiv Submission
2019 · same figure
Publication of "Attention Is All You Need"
Publication of "Attention Is All You Need"
2017 · same figure
A
AlphaGo Zero introduced
2017 · same figure
J
John McCarthy creates the LISP programming language
1958 · same figure
U
UK Lighthill Report criticizes AI research
1973 · same figure
A
Association for the Advancement of Artificial Intelligence (AAAI) founded
1979 · same figure
A
Attention Is All You Need Paper Presentation at NIPS 2017
2017 · same figure
P
Publication of the AlphaGo Nature paper
2016 · same figure
DeepMind publishes AlphaZero paper
DeepMind publishes AlphaZero paper
2017 · same figure
D
Dawon Kahng and Martin Atalla present the MOSFET
1960 · same figure
A
AlphaFold 2 paper published
2021 · same figure
P
PyTorch 1.0 Release
2018 · same figure
R
Release of BERT paper
2018 · same era
R
Release of BERT paper
2018 · follows
R
Release of "Attention Is All You Need" (Transformer) paper
2017 · contemporaneous
R
Release of "Attention Is All You Need" (Transformer) paper
2017 · same figure
N
NeurIPS 2017 Conference Begins
2017 · same figure
R
Release of BERT Paper
2018 · same figure
R
Release of GPT-3 Paper
2020 · same era
R
Release of GPT-3 Paper
2020 · follows
R
Release of BERT model on TensorFlow Hub
2018 · same figure
T
Transformer model arXiv submission
2017 · contemporaneous
T
Transformer model arXiv submission
2017 · same figure
N
NAACL 2019 conference opening
2019 · same figure
P
Presentation of Attention Is All You Need at NIPS 2017
2017 · same figure