ALPHA Timepoint is in alpha Talk to Us
Sign In Get Started
P

Presentation of 'Attention Is All You Need' at NIPS 2017

Researchers Ashish Vaswani and Noam Shazeer present their groundbreaking paper 'Attention Is All You Need' at NIPS 2017, introducing the Transformer model to a room full of AI experts. The audience li

Setting

Long Beach Convention Center, main conference hall during NIPS 2017. The spacious hall is filled with rows of seating, a large projection screen at the front, and a stage where the presenters stand. The backdrop features the NIPS 2017 logo and sponsor banners. The space is designed for large-scale academic conferences, with high ceilings and professional AV equipment.

Characters

The figures in this scene as an entity network — co-presence links everyone in the moment; speakers who trade lines are bound tighter. Turn the resolution dial to reveal depth the engine actually computed.

TNGF
SELECTED
Ashish Vaswani
primary
A young, clean-shaven researcher with short, dark hair and a focused demeanor. Wears rectangular glasses that reflect the projection screen's light as he presents.
Noam Shazeer
primary
A lean man with short, dark hair and glasses, wearing a slightly rumpled button-down shirt. His posture suggests a mix of intellectual intensity and casual confidence.
Senior Researcher
secondary
A distinguished AI expert with sharp features and a piercing gaze. Wears rectangular glasses that reflect the projector light. His posture suggests years of academic rigor.
Tech Journalist
secondary
A sharp-eyed reporter in their early 30s, with a professional demeanor and a keen observational gaze. They have a slightly disheveled appearance from rushing between sessions, with a press badge prominently displayed.
Graduate Student
background
A young researcher with a slightly disheveled appearance, wearing thick-rimmed glasses and a look of intense focus. Their dark hair is pulled back in a messy bun, and they have a laptop balanced precariously on their knees.

Dialog

Ashish Vaswani Our key insight is that self-attention allows the model to weigh the importance of different words in a sequence simultaneously — no recurrence or convolutions required.
Noam Shazeer (interjecting) And crucially, this parallel processing means we can train significantly faster than with RNNs — here's the math proving it scales better with sequence length.
Senior Researcher Wait — how does this address vanishing gradients in practice? Your theoretical analysis assumes perfect optimization.
Ashish Vaswani Excellent question. Our empirical results on WMT actually show... (clicks to next slide) ...that the attention heads learn to maintain gradient flow dynamically.
Noam Shazeer (smiling) Think of it like this — each head becomes a specialist at tracking different dependency types. The model's own internal version of divide-and-conquer.

Chat with Characters

You've used your 3 free turns

Sign in to keep chatting with characters from this moment — unlimited turns.

Sign in to Continue
— Sign in for unlimited

Causal neighbors · 233 linked moments

A
Attention Is All You Need Paper Presentation at NIPS 2017
2017 · contemporaneous
A
Attention Is All You Need Paper Presentation at NIPS 2017
2017 · same location
P
Presentation of Attention Is All You Need at NIPS 2017
2017 · same location
A
Apple introduces Siri
2011 · same figure
G
Google Maps launched
2005 · same figure
R
Release of the MOS Technology 6502 Microprocessor
1975 · same figure
R
Release of GPT-3 Paper
2020 · same figure
R
Release of "Attention Is All You Need" (Transformer) paper
2017 · same figure
M
Macworld 2007 iPhone Introduction
2007 · same figure
2
2015 World Internet Conference
2015 · same figure
M
Microsoft announces Cortana
2014 · same figure
G
Google announces Google Assistant
2016 · same figure
G
Google releases TensorFlow as open source
2015 · same figure
Publication of "Attention Is All You Need"
Publication of "Attention Is All You Need"
2017 · same figure
R
Release of BERT paper
2018 · same figure
J
John McCarthy creates the LISP programming language
1958 · same figure
I
IBM PC Launch
1981 · same figure
T
Tandy TRS-80 Model I Launch
1977 · same figure
A
Attention Is All You Need Paper Presentation at NIPS 2017
2017 · same figure
N
NeurIPS 2017 Conference Begins
2017 · same figure
T
TensorFlow 1.0 Release
2017 · same figure
O
ONNX Format Announcement
2017 · same figure
F
First Maker Faire Bay Area
2006 · same figure
C
Commodore PET 2001 Launch
1977 · same figure
A
Atari Video Computer System (VCS) Announcement
1977 · same figure
A
Apple Special Event September 2018
2018 · same figure
A
Announcement of the 2016 Turing Award to Tim Berners-Lee
2017 · same figure
R
Release of BERT Paper
2018 · same figure
A
Apple WWDC 2019 Keynote
2019 · same figure
G
Google Now Announcement at Google I/O 2012
2012 · same figure
S
Siri Introduction on iPhone 4S
2011 · same figure
A
Amazon Echo Launch
2014 · same figure
W
Windows 10 Announcement
2014 · same figure
I
IBM Watson Health launched
2015 · same figure
D
Dawon Kahng and Martin Atalla present the MOSFET
1960 · same figure
R
Release of BERT model on TensorFlow Hub
2018 · same figure